Journey Further’s Experimenting in an Agile Environment with Magnolia Carvajal

Magnolia Carvajal

A chat with Journey Further’s Magnolia Carvajal about Experimentation

I recently chatted with Magnolia to get her take on whether we should avoid overlapping tests and how to kickstart a culture of Experimentation.


Rommil: Hi Magnolia, thank you for taking time away from your busy schedule to chat with me today. How are you?

Magnolia: Hello, no worries at all thanks for reaching out, I am very well, happy to be back in the office after Covid and finally enjoying the warmer weather 🙂

Can’t complain about more sunshine, that’s for sure. So Magnolia, I’d love to hear about how you got into Experimentation?

It was really random actually, I studied marketing with Psychology at university and at my first job at Tesco and I was lucky enough to be invited to a discussion around reducing average handling time on the websites; from that meeting I asked if they would put me on a digital marketing course through Tesco Academy and they did! I learnt a lot and then I got put to work on the Tesco Clubcard website testing to reduce the calls coming into the call centre — it was all CRO from there on.

What are your most and least favourite things about Experimentation?

Most Favourite thing: When you have done all the research, got all the data behind your test, you run the test and it LOSES. Why??

Least favourite thing: As above.

Advertisement

READ  Bithika Mehra on partnering with HiPPOs and leveraging neuromarketing to drive conversion
Connect with members of the Experiment Nation Directory
PhotoNameLocationShort Bio / SpecialitiesLinkedIn URL
Gurudev Karanth San Francisco Bay Area One of the founding members of the experimentation team at eBay in 2005, helping build the digital experimentation platform ground up and then deployed at Paypal. Built the first omni-channel experimentation platform ground up at Target https://www.linkedin.com/in/gurudev/
Brendan McCook Washington, DC, USA AB Testing, Product Strategy, Consulting https://www.linkedin.com/in/brendan-mccook-1b535026/
Michael St Laurent Vancouver Experimentation Program Consulting https://www.linkedin.com/in/michaellstlaurent/

Changing gears to something more tactical — What’s your view on deciding what to test?

Analyse, test, analyse, repeat. Also, insights taken from previous tests is a real driver of what to test.

A common question I hear is, “How many changes should be made to comprise a variant?” What’s your take?

Very much depending. I generally will only make one change unless it is a split URL or landing page test. Otherwise — how do you know what it was that worked/didn’t work?

How do you go about selecting a success metric, or a set of metrics?

Depends on the hypothesis and what we want to get from the test.

Companies that catch the testing bug always want to run more tests. What’s your opinion on concurrent testing? How do you account for potential interactions?

My opinion is that there is no way to not do it in an agile environment, but this does not mean running 2 tests on the same page as that is a direct crossover. However, running tests on the homepage and then further down the customer journey like the checkout is necessary in an agile environment. We account for potential interactions by segmenting properly when doing post test research and examining the interactions between any 2 sets of A/B tests

READ  UI design is hard work — but Jakub Linowski's GoodUI is leveraging Experimentation to make that work easier

How do you know if a company has a culture of Experimentation?

You would know a company has a culture of experimentation if they had a data driven web development process and not just changing stuff from the top guy.

What steps can you take to help a company adopt one?

Keep testing — show them the results of failed tests including the money saved from dev work, show them that if you had released the failed test on to the website without testing it, it would have actually hurt the company’s conversion rate as well as losing money paying for dev work and therefore losing revenue.

Finally, it’s time for the Lightning round!

Bayesian or Frequentist?

Bayesian

Favourite experimentation tool.

My dev really likes SiteSpect

If you couldn’t work in Experimentation — what would you do?

Something outdoors, only experimentation will keep me inside!

Describe Magnolia in 5 words or less.

Analytical, Methodical, Creative, Innovative, Extrovert

Thank you!

And thank you 🙂



Connect with Experimenters from around the world

We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.

READ  Convoy’s Chad Sanderson: Experimentation not only protects your KPIs but your job as well

You may also like

Ana Catarina Cizilio tells us about CRO in Brazil and the mindset needed for CRO

Ana tells us about the differences between the US and Brazil, as well as about the mindset you need to Read more

Specsavers’ Melanie Kyrklund on How to run a high-impact optimization program

In this episode, Rommil and Tracy interview Melanie Kyrklund from Specsavers on how to run a high-impact optimization program, how Read more

SplitBase’s Raphael Paulin-Daigle talks about convincing clients to invest resources towards optimization

Hi Raphael, thanks so much for taking the time to chat with us! How have you been? I’ve been good! Read more

Widerfunnel’s Chris Goward on how Experimenters need to focus less on the tools and more on strategy

Experimenters put so much emphasis on tool selection that they forget the importance of taking a step back and thinking Read more