Fresh Egg’s Sebastian Larsson: Experimentation roadmaps begin with research

Sebastian Larsson

A Conversion Conversation with Fresh Egg’s Sebastian Larsson

We hear stories of how some companies run tens of thousands of tests every year, and how they test anything and everything. While testing a wide array of ideas that range from optimization and exploration is good— something that the best Experimenters often encourage is having an Experimentation roadmap. An Experimentation roadmap is, as the name implies, a plan of upcoming Experiments that is rooted in research and goal alignment. I recently spoke with Sebastian to understand how his agency works with clients to create them.


Rommil: Hi Sebastian, how are you? Thank you for taking the time to chat with me today!

Sebastian: Hey Rommil, I’m great thanks. Kids half term holiday here in the UK so have had a few days off work to look after my two girls.

https://www.freshegg.co.uk/

Can you share with our readers what the Fresh Egg Digital Marketing Agency is about and what you do for them?

Sure, Fresh Egg is a digital agency located on the south coast of England in a town called Worthing. We pride ourselves on our customers-first approach to digital marketing. We always start any client relationship by understanding our client’s customers and their experience with the brand. My role is Senior Conversion Strategist in our Conversion Services team.

How does Fresh Egg leverage Experimentation internally?

If I understand your question correctly, it’s something I admit we could be better at. We run experimentation for multiple clients, but we are not as good at using it on our own website. It’s usually a time issue as client work is always prioritized.

When working with clients, how do you go about helping them develop their Experimentation roadmaps and strategy?

READ  Twilio's Justin Coons shares what you need to know about SEO Experimentation

We always start with a CX Discovery. This involves qualitative and quantitative research. We do everything from internal stakeholder interviews and Audience Empathy Mapping sessions to user testing, user interviews, heuristic website analysis, competitor analysis and of course a thorough check of their quantitative data. All this information then informs the creation of an experimentation roadmap. We do everything from running the whole testing programme for clients to training up internal teams to run their own CRO.

Advertisement

Connect with members of the Experiment Nation Directory
PhotoNameLocationShort Bio / SpecialitiesLinkedIn URL
Jonathan Deleon Orlando Experimentation https://www.linkedin.com/in/jonathantdeleon/
Aybala Karadayilar Berlin Product Management, Design Thinking, A/B Testing https://www.linkedin.com/in/aybala-co%C5%9Fkun-450537110/
Johann van Tonder South Africa Experimentation, Data Science, User Research, CLV http://linkedin.com/in/jvtonder

One of the biggest hurdles for Experimentation is the sharing of learnings. Because you work with so many clients, how do you go about ensuring everyone shares and absorbs the learnings?

We are a small team, four of us, and we all sit together so it’s natural for us to discuss learnings or issues that come up daily. We also have a document where we keep combined test data for all clients.

What are some of the challenges you face when trying to launch experiments for clients and how have you overcome them?

The main challenge is usually technical. For newer clients, we don’t know their website as well as they do, so having a thorough technical handover before starting the work is crucial. We also try and get sign off during the design stage. That way we don’t end up making changes once the variations have been built as this usually takes a lot more time and ends up delaying test launch. Basically, agreeing to the process with the client before starting the work saves a lot of time for everyone.

READ  Optimization and the human mind with Guido X Jansen

So every client, understandably, wants to understand the return on their investment with regards to Experimentation. Often at times, the observed lifts you see from Experiments, don’t materialize when they are rolled out. How do you explain this to clients?

This is a tricky one. We often work with smaller clients who might not be used to running experimentation programmes. I believe that it’s best to have a discussion upfront that test results are only valid for that test period. We can say to a decent degree of certainty that for the duration of the test we have increased revenue/leads by a certain amount. Leaving a holdout group running in control or rerunning the same test down the line to see if the same results still stand is one way of seeing if the initial result holds up. I’m not saying that we have the perfect solution to this as it’s hard when clients want to see that ROI and it tends to be short term wins and not a long-term strategy that the client is after. We are constantly working on better ways of explaining the value of experimentation to our clients that is not just instant revenue uplifts but long term strategy that leads to customer satisfaction.

Finally, it’s time for the Lightning Round! Describe experimentation in only 3 words.

Research, Testing & LEARNING

Bayesian or Frequentist?

On the fence so far, I think there is room for both if you understand how and when to use each method but I’m leaning towards Frequentist until someone convinces me otherwise.

READ  Launching a world-class Experimentation program and how to spot a healthy Experimentation culture with Journey Further’s Jon Crowder

What is the Swedish word for Conversion Strategist?

Ah, it’s funny because I have only worked with CRO in the UK. I had to look it up as I’ve lived here for over 14 years now and it didn’t exist as such when I left Sweden. I now know it’s Konverteringsexpert

Sebastian, thanks for joining the conversation!

Thanks for having me. Really enjoy reading the interviews on Experiment Nation so keep up the good work!



Connect with Experimenters from around the world

We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.

You may also like

To build or buy? Solving the classic Experimenter’s dilemma with Convoy’s Chad Sanderson

A conversation with Convoy’s Chad Sanderson about Experimentation Those in this space eventually face the question of whether to use Read more

BMO’s Claire Yeh: Experimentation is key to Personalization success
Photo of Claire Yeh

A conversation with BMO’s Claire Yeh about Experimentation Personalization and Experimentation may seem like very unrelated practices — but this can’t be Read more

The how, what, and why of copy testing with Green Light Copy’s Eden Bidani

A conversation with Green Light Copy’s Eden Bidani about Experimentation I recently spoke to Eden about what conversion copywriting is, Read more

Bithika Mehra on partnering with HiPPOs and leveraging neuromarketing to drive conversion
Bithika Mehra

A conversation with Bithika Mehra about Experimentation Despite our best efforts, HiPPOs are key stakeholders that can make or break Read more