A Conversation with Accenture’s Geetu Mehta about Experimentation

I recently had a chance to speak to Accenture’s Geetu Mehta about what a good Experimentation strategy looks like, how she develops one, and how she gets buy-in.

---

Rommil: Hi Geetu, how are you today, Thank you for taking the time to chat!

Geetu: I am good Rommil. Thank you for the opportunity.

Could you tell us a little bit about what you do today and your career path thus far?

I am the digital analytics and experience optimization SME Manager at Accenture Interactive working at the forefront of Ideating and Problem solving for and with our clients everyday across all industries. We do this by architecting innovative customer experience transformation solutions with the goal of improving overall business performance and customer experience.

I have been in the digital landscape for over a decade now with over 16 years spent within ANZ with roles across Digital Sales ( SAAS/ PAAS), Customer and Marketing Analytics Strategy, Marketing automation and Data-Driven Marketing across Agencies and Publishers.

Awesome. Let’s dive right in.

There’s a lot of confusion in this industry between the words, “optimization” and “experimentation”. Could you share with us how you would explain the difference?

Well, one is the outcome of the other, optimization is the outcome that’s enabled by experimentation. Optimization is the process of iterative performance improvement aligned to any one of the business value drivers based on business objectives for eg?—?process improvement to drive efficiencies, improve customer acquisition or reduce customer churn.

Whereas, Experimentation is how you do things, it is what enables data driven decisioning taking the guesswork out of the equation that leads to optimization. It’s about testing and validating every business idea or assumption to understand where our efforts and cost needs investing in order to address a valued business outcome.

“A good optimization strategy is one that identifies and validates the right problems from the start.”

So, as an optimization strategist, could you share with us what makes a good optimization strategy?

From my point of view -

A good optimization strategy is one that identifies and validates the right problems from the start. Identifying what is the experience that we are looking to create or solve for, how does it map or connect to your key strategic outcomes or how will it move/ improve your business metrics if solved. Customer obsession will always lead you to improving business metrics.

Often we enter businesses and they have these set of problems they think are their real problems and a set of solution traps (Ideas) against them. What we are not doing up front is trying to understand the real problems for e.g., sometimes thinking that a drop in customer base associated to personalized marketing messaging or relevant channels of engagement whereas if you dig deeper with analytics and make an effort ask your end-users( Quant + Qual framework) you might find the problem might be in your mobile checkout flow design. My approach would be to spend 90% of time researching and validating the right problem and 10% formulating the right hypothesis to wrap up a right solution.

So in summary, A good optimization strategy looks at

1. Customer obsession?—?Solving for the entire journey experience touchpoints and not for channel or messaging in silos
2. Measures what matters?—?Connect business metrics to enable it with experimentation strategy?—?research the right problems to solve and assign the right KPI’s against it.
3. Process rigour?—?mobilize a cross-functional team with a framework to share & operationalize insights that can connect back to strategy.
4. Celebrate, Share & Evangelise your Wins and Learnings( Fails)?—?There is a win-win situation in both as you learn something amazing even if you failed.

*PS: Always get an executive buy-in at the start- By default.

That last point is gold. Without that executive buy-in, you’re toast.

Could you walk us through how you design an Experimentation strategy?

There is an approach and then there is a framework that can flex with revisited goals and revisited business models

The general approach is:

• Assess Experimentation Maturity; Map the current state across people, process, technology, data, measurement and culture.
• Set up an experimentation program which includes, Governance, Mobilisation of a cross-functional skillset, An Experimentation/ Measurement framework, Tools/tech set up for success, Insights repository and collaboration tools to create a feedback loop.
• Make sure all goals connect to the business outcome or it just becomes another project with no business value extracted from it.
• Research and validate the problem and solution hypothesis
• Design, Build and Run Experiments
• Measure and Share Insights with a feedback loop internally
• Share and evangelize Wins and Learnings

Definitely. A lot of times, folks just focus on wins when Learnings are just as, if not more, valuable.

So, what are some of the challenges in getting buy-in on your strategies? Do you have any tips on how best to overcome these challenges?

That is the tough part :)

Especially in the ANZ market where experimentation or optimization are the words associated to CRO or A/B testing mostly. I would say that some of the organizations have now started to realize the value of it and started to implement experimentation broadly into the business and have started reaping benefits, but most of them are still in that education phase.

Well I am no master to preach in this area and I am still learning heaps myself as I go, but from what I have seen and learned here are my two cents:

• Invest in one or Be(if you already have one) an Experimentation Evangelist in your organization: The resource is scarce but immensely valuable when it comes to commercial benefits it can bring to your business by embedding experimentation into your culture.
• Partner with those who are in the business of embedding experimentation as a way of working in your DNA
• Prove and Scale: Start small and pick up some experiments within your team that do not take a major dip in your budget but ties directly to business objectives to show incremental gains. Share this insight in ideation sessions and strategic executive team meetings building excitement tied with value and gains.
• Speak Executive language: Create a business case for change that’s commercially viable and feasible to get investment
• Power of sharing: Creating that nudge by celebrating experimentation wins subtly creates a subconscious culture where people want to participate?—?so when the time comes to roll out the program and establish a cross-functional environment they are already mentally ready to go.

As we all know, things can change dramatically in a matter of months?—?especially this year?—?how do you keep a strategy relevant and impactful despite these changes?

Now is the time more than ever for business to start understanding the value of building a culture of experimentation. With so many businesses shifting their business models from brick and mortar to online; B2B to B2C?—?Organisations that experimented built resilience against this change. So, Always be testing, keeping customer experience at the heart of everything you do, keep re-evaluating your goals, Have a flexible framework that can pivot, Share wins and learnings even if they are small is the key to creating that culture to drive impactful change.

It’s one thing to create a great strategy, it’s another to execute on them. How do you ensure that your vision becomes realized?

Absolutely correct.

Therefore, we also run accelerator programs which include workshop & maturity assessment at the start to help some of our clients understand gaps in capabilities. This assessment is done across people, processes, tech, data, culture and measurement. This helps us prioritize our efforts on the areas that need attention before a program of such rigour is rolled out. In some cases we also run this simultaneously where we can run a POC (6 months) to show quick impacts whilst also addressing their maturity to help them build and scale parallelly. We also run on the board experimentation training program where we will also train your team to become savvy whilst slowly but steadily inducing that culture of experimentation.

Many clients have little patience. How do you ensure that they have enough patience to see the fruits of your labour?

By setting the expectations right at the start. Generally, that’s where it goes all wrong. Underpromise but over-deliver as they say, especially with experimentation?—?The concept of failing is nothing but learning things that you wouldn’t have known otherwise. When clients come and say can you deliver a POC in 3 months, I would say yes if you have all the pillars for the program in place and just need rigour/ strategy to support but that is usually not the case hence always help them understand why it takes longer time to establish that rigour to deliver impactful results.

Finally, it’s time for the lightning round!

Bayesian or frequentist?

Bayesian

Describe Geetu in 5 words or less.

Ideator, Problem Solver, Experience Seeker, Experimenter

What is your biggest pet peeve in crafting an experimentation strategy? HIPPO

If you couldn’t work in experimentation what would you do?

Re-Inventing Education Models and Wanderlusting (while driving that change)

Geetu, thank you for joining the Conversation!

---