A Conversation with Specsavers’ Melanie Kyrklund about Experimentation

I recently had a great chat with Melanie about how she nurtures an Experimentation culture by focusing on being a true business partner and promoting a strong Experimentation process; how she structures Optimization teams, and what she looks forward to in the next 5 years.

---

Rommil: Hi Melanie! How are you? I hope you’ve been well!

Melanie: Hi Rommil. I’m good thanks. Finally emerging from a challenging few months, as is everybody!

Thank you for taking the time to chat today. How about we start with you sharing a little about yourself and your career journey thus far?

I am currently working as Group CRO Manager at Specsavers, tasked with optimizing the global journeys for online prescription glasses. It’s still a relatively small but growing market, which makes it a fulfilling area to work in. Prior to Specsavers I held a mix of optimization, product and analytics roles at Staples, Booking.com and Liberty Global?—?I have 12 years’ experience in experimentation and 20 in digital overall.

Having worked across the digital maturity spectrum, I bring an understanding of corporate structures along with experience in newer ways of working. Complex businesses that are over twenty years old with decentralized operations and legacy systems cannot adopt new organizational models and processes easily. For many companies it isn’t feasible to “test everything” or aspire to?—?the requisite traffic and resources aren’t there and what is available needs to be used discriminately. My skill is in helping complex organizations pragmatically embrace experimentation and along the way, discover more efficient ways of developing customer experiences.

Very cool. I’m curious, has the pandemic impacted your day-to-day? Are you still running Experiments? Have you had to make any adjustments?

In our case, the pandemic represented an opportunity for the eCommerce function. Customers turned to our online services in greater numbers once our stores went into lock-down. In view of this increased demand, we continued experimenting. I didn’t need to make any changes to the experiment pipeline, as the prioritized tests tackled fundamental aspects of our online proposition that would remain relevant moving forward. Now that our stores are opening again, we are keeping an eye on changes in behaviour and readjusting our budgets and focus.

What role does Experimentation play in helping shape product direction and innovation?

Experimentation plays a fundamental role in this at Specsavers. As mentioned previously, I’m focused on online prescription glasses which is a growing market. As a result, I regularly need to circle back to understanding the customer and the problem/solution space and how our product offering can align to it. This is more of the innovation piece. Parallel to that, I am tackling the current user experience, understanding the fundamental areas to address and testing pragmatic solutions that can feed into product development.

From a broader perspective, one of Specsavers’ key values is to continually strive to improve the way we do things?—?so experimentation is very much aligned with company behaviour.

How do you balance earning money in the near term and product iteration?

For me, the two are intertwined. With traffic limitations comes greater scrutiny, and as a result, I build impact and revenue considerations into my prioritization process.

Having worked at international companies, do you suggest Experimenters run the same tests in multiple countries or test in a single country first and then roll-out to the others?

I test in multiple countries before rolling out. Though it creates more effort it speeds up decision-making and instills confidence in the course of action to be taken. Taking the conversation one step back, many companies have grown through acquisitions or run decentralized operations to some degree. This creates a non-homogenous environment to operate in upfront. Global regions and countries are also intrinsically diverse. All these nuances in business strategies and customer behaviour need to be understood prior to formulating solutions, and downstream, can generate different responses to tests in markets.

As a leader in Experimentation, how do you decide how to structure your team?

It all starts with strong optimization managers who can partner with the business. I look for multi-disciplinary backgrounds?—?individuals who are autonomous with data, have an understanding of UX and are reasonably technical. Looking downstream at the diverse resources required to run experimentation I’m less dogmatic about structure. I’ve had success working in multi-disciplinary teams, partnering with internal departments and outsourcing.

What kinds of metrics do you track to ensure that your program is performing well?

Experiments launched, win rate and incremental revenue.

How do you nurture a culture of Experimentation?

This is a complex topic and the answer is?—?it all depends on the organization you’re working in. Maybe it is more tangible if you think of it as nurturing a process rather than a culture, as experimentation often disrupts existing ways of working and thinking. The cultural shift is essentially about accepting a rigorous exploration process in place of rigid project plans.

Having executive-level support and excitement for experimentation that filters down to teams and their goals is a big advantage. In the absence of that experimentation may remain siloed, and optimizers will need to work hard to influence the business beyond their area of remit.

My main piece of advice to anyone working in optimization would be to focus on becoming a valuable business partner first and foremost. Demonstrate to your stakeholders a deep understanding of their business, marketing and product goals and proactively help them to achieve them via experimentation. This will enable broader discussions about ways of working.

“My main piece of advice to anyone working in optimization would be to focus on becoming a valuable business partner first and foremost.”

You’ve been in CRO for a long time?—?what are some of the most interesting changes that you’ve seen during that time?

From a tech perspective, the rise of mobile has underpinned the past decade with obvious consequences for optimization. Also of note is cloud-based data storage which has enabled data to become a crucial component in the delivery of digital experiences. Data has increasingly become a product for optimizers to leverage in the frontend.

Overall, experimentation has grown into a more mature discipline focused on solving customer problems and imbedded into product development and marketing.

What are you looking most forward to in the next 5 years?

I would like to see the adoption of sustainable growth models in business. Amsterdam, the city where I live, will be adopting the “doughnut model” devised by the British economist Kate Raworth in their public policy development post-COVID 19. I’m curious to see what the output of those discussions will be. We urgently need to look at how we can thrive whilst remaining in balance with the planet.

It’s time for the Lightning round!

Bayesian or Frequentist?

Either?—?provided one methodology is used consistently in the business

Who do you follow for inspiration in terms of Experimentation?

My users ?

What are your biggest pet peeves?

Selfishness

Finally, describe Melanie in 5 words or less.

Impossible!

Thank you, Melanie, for joining the conversation!

---