A Conversion Conversation with Edelman’s Nicola Ambrogio

I recently chatted with Nicola during one of her business trips to Toronto over coffee. We talked about her work at Edelman, and how she leverages data and experimentation to inform better product decisions and encourage clients to get over their usually less-than-perfect assumptions.

---

Rommil: Hey Nicola! Thank you for taking the time to chat with me today, how are you?

Nicola: I’m fully hopped up on hot chocolate and in full Christmas mode. I’m a huge fan of Experiment Nation and I’m so excited to be here!

You might officially be our first fan LOL! How about we kick this off with you telling us a bit about yourself and what exactly is a user experience researcher? More specifically, how is it different from UI and UX?

I work in User Experience Research (UXR) and web analytics on Edelman’s Insights & Analytics team. After my undergrad in political science, I started working as a copywriter at a small ad agency and had no idea UX was a thing. I started hanging out with some cool Search experts, UI designers and Web Analytics people and was inspired to take my Google Ads and Google Analytics Certifications. I fell in love with Google’s mantra of thinking like a user and I got really obsessed with UX and optimization.

But I knew very early on that I would never be a UX designer. I’ve never touched Adobe Creative Suite and I have no design skills whatsoever?—?but I knew I could make users happy by studying their behaviour, identifying their ambitions and frustrations and finding a solution that brings them from their current state to their desired state. My contribution to user happiness is not in interface/interaction design but rather in the primary and secondary research that informs those interfaces and interactions. UXRs are responsible for empowering UX/UI designers with all the audience research, measurement frameworks/KPIs and critical insights they need to successfully design and/or optimize interfaces. Otherwise, UI design just becomes a subjective, artistic exercise and may fail to adequately resonate with target audiences.

I fight for users and ensure their desires are not being treated as secondary to corporate influences/policies. At the end of the day, a UXR is responsible for ensuring the final product was built by data and characterized by intuition. Not the other way around.

That’s interesting. I think that’s becoming a very common theme these days?—?putting an end to bike-shedding by leveraging data. In your opinion, how do you view UXR fitting in with other functions like analytics and experimentation?

Seamlessly and enthusiastically. I work very closely with wildly talented analysts, UX designers, search specialists and developers. UX unites us all under the common objective of making our users happy at any given point along the user journey. This happiness should be measurable and quantifiable, and it’s why we work collaboratively at the onset of a project to create user personas and attach clear KPIs to each one. We ensure that the tagging of the site adequately matches the behaviours we’re trying to measure, and we all work off the same dashboards that visualize the KPIs that matter most. If I’m designing a usability test, a survey or an experiment, I ask for feedback from the whole gang?—?search experts, UX designers, web analytics experts and web developers. I keep everyone looped in from the beginning and take their feedback and unique perspectives seriously because it helps set us up for success right from the start.

How does a UXR leverage experimentation in their work? What kinds of hypotheses do you have going in?

Mainly in usability testing. When I studied UX at BrainStation, I was taught to do usability tests at every design sprint. I think it’s so critical to eliminate any biases and assumptions as early as possible. A lot of times, our clients come in with very strong opinions about what a landing page should look like, so a lot of times, we try to do research to either prove or disprove our clients’ hypotheses. We usually end up creating two landing page options- one that corresponds to exactly what the client asks for (usually where all key components are “above the fold” *cringe*) and another version that is based on the audience research and measurement framework that we’ve established. We’ll create prototypes of the two versions and can either create small in-person focus groups or run online usability tests through something like Usability Hub to see what performs best. Spoiler alert?—?while users generally spend 80% of their time at the top of a page and it’s best to have H1s fairly high up on the page for search purposes, the “fold” is fake news. People scroll on whichever device they’re on.

What other kinds of experiments do you run for your clients and what are some of the challenges you face with clients when it comes to running them?

We had a client in financial services who told us they wanted us to update their website, which hadn’t been updated since the early 2000’s. When we dug in to their broader organizational objectives, they mentioned they were hiring new financial analysts and expanding their team. When we asked if we could survey their recent hires to get a better understanding of their frustrations, ambitions and overall experience with the website, the client flatly refused, stating that because candidates are recruited directly from post-secondary institutions, candidates don’t visit the corporate website.

Don’t you love that? Opinions based on zero information ha! Go on.

We persisted and stated there was no harm in using the survey to uncover whether or not they visited the website and any other insights into what made them choose this organization over competitors in the industry. We ended up doing the survey and found out that 78% of recent hires visited the website AND visited competitors’ websites. This finding gave us more leverage to conduct more research and develop a more sophisticated digital recruitment strategy, which set the stage for future experiments. We had another client in the tourism industry that was selling tickets through their website. Most of our digital marketing campaigns were driving traffic to the “buy tickets” page, but our client wanted to increase the rate of donations and suggested we drive more traffic to the animal welfare “donations” page. To test whether or not this would actually drive more donations, we ran the donations campaign but also added the option for users to add a donation to their order during the checkout process for their regular tickets. We found that users were more likely to add a donation to their regular order than to just make a donation on the donation page.

Nothing is more convincing to clients than data. Love it.

Here’s an interesting question. Communication is such an overlooked aspect of experimentation?—?the data doesn’t always speak for itself (especially for busy executives). Do you have any opinions on results visualization that our readers could benefit from?

Yes! While it has its limitations, I’m a big fan of Google Data Studio (GDS) because it’s pretty intuitive and allows you to connect data from Google Analytics, DV360 and other sources relatively easily. I like being able to help show C-Suite execs the sales results they want to see plainly and simply. GDS makes it easy to add percent increases/decreases next to each metric, so a busy C-Suite exec can easily see how we’re performing in comparison to the previous quarter. I’m also a big fan of creating visual and dynamic audience demographic information into a dashboard to create a mini user persona. It helps remind my internal team and our clients that these sales results were generated by real humans with real lives, frustrations and ambitions and it’s our job to make them happy. I believe data visualization helps cultivate trust among all stakeholders by shifting the focus to objective figures themselves as opposed to the likability of the person who’s presenting those objective figures.

People love pictures, am I right?! Ok - it’s time for the Lightning round! Vancouver or Toronto?

Vancouver is beautiful but Toronto will always be bae.

Pop-ups? Best thing ever or the worst?

Wildly offensive.

Hamburger menu? Love it or hate it?

It has no business being on desktop sites but I think we should accept it as universal and ubiquitous on mobile sites.

What do you love most about UXR?

By eliminating bias, assumptions and egos, UXR unites and empowers us all to make users (measurably) happier.

Very cool. Well, thanks for taking the time to chat with me! Hopefully, we’ll cross paths again the next time you’re in Toronto :)

---