Informa Market’s Tony Grant on getting people excited about Experimentation
A Conversion Conversation with Informa Market’s Tony Grant
When you ask folks whether Experimentation is a good idea, more often than less, they will agree. While agreeing is fine, it’s surprisingly challenging to many people to run their first Experiment. There’s often a fear of risk, uneasiness with statistics, or the perception that it is only for when there is spare time?—?rather than critical to delivering value. I recently spoke to Tony about how he approaches those who are hesitant to dive into Experimentation, his approach to documentation, and the food he misses most.
Rommil: Hi Tony, how have you been? Thanks for taking the time to chat!
Tony: Thank you for inviting me to talk about experimentation. All is as well as it can be in these unprecedented times. We have just come out of lockdown in the UAE, there is still a curfew but it is nice to start moving back to some form of normality?—?whatever that may be from now on.
I’m happy to hear you’re keeping safe!
Could you share with us a bit about what you do at Informa and your career journey thus far?
I’m a CRO specialist, enabling companies to improve their online sales and increase opportunities at lower costs. After completing an MSc in Marketing Comms, I joined a wonderful digital agency called CandidSky as an account manager. Very quickly started to notice that many companies didn’t make decisions based on data available to them. It was then I immersed myself in the world of ‘CRO’, reading whitepapers, taking courses, speaking to people in the industry and of course, trying and failing.
After a few years at payments company Worldpay based in London, I moved to Dubai to be the CRO specialist at Informa Markets. My main responsibility is to enable experimentation and drive a culture of test & learn across the division. I’m sure we’ll come on to the how.
Yup?—?we’ll definitely dive into the “how”!
As someone that is driving a change of culture towards a data-driven “Test & Learn” one?—?can you share with us how you convince skeptics and get people excited about Experimentation?
If your key business purpose is growing customer value, competitor advantage and sustainable profitability, there is a good chance experimentation will help. You have to be excited and enthusiastic about what you do and believe in it, which you can say about anything you do.
I try to understand and learn why that person might be skeptical by collating data such as, how mature their digital marketing strategy is, have they tried experimentation before? What did they do? How did they go about it? And then drilling down accordingly. Depending on the answers given this directs me down the right route to take. The key is keeping the questions open-ended.
For example, simply asked how their conversions are performing, 99% of people will say it could be better and/or improved. This leads to, what do they think the issue is? Do they know?
If they know, as an experienced optimizer you might be able to identify some areas to look at. If they don’t know, then simply demonstrate there are methods we can use to find out, provide previous examples.
Being honest, authenticity is absolutely key. Always leave people with a simple action/tip they can try quickly and easily with a low barrier to entry. Always extend the offer to help based on the information you collate.
“If your key business purpose is growing customer value, competitor advantage and sustainable profitability, there is a good chance experimentation will help.”
I love your approach. I really like your tip about giving people simple actions with low barriers to entry.
So I see you’ve spent some time at UserTesting. I’m interested to hear your perspective on the role that user testing plays in Experimentation?
User Testing, in my opinion, is underused by companies. Often in projects such as a new website or redesign project, this step is missed, or deprioritized due to time/budget constraints.
User testing compliments web analytics and is essential for any form of research. User testing helps to figure out ‘why’. Why are people doing what they are doing? What are the bottlenecks? You can then spot patterns and begin the process of minimizing perceived barriers creating a stronger hypothesis.
Definitely. I like to think User Testing is part of a winning trifecta along with Experimentation and Analytics.
How do you ensure that the two practices?—?Experimentation and User Research complement each other efficiently?
The two go together, you can’t really do one without the other (you can, but we all know how that goes). You need solid research to inform what you are going to experiment on, and you need to validate the research is correct.
Time is split differently to support these activities
- Enablement and adoption, leaning towards driving tests, ensuring tech stack is working properly, are we up to date with industry practices, improving frameworks, processes, alignment with business goals and the PR internally.
- Research, improving knowledge, up to date behavioural insights, to increase the quality of experimentation. We also provide training, workshops, and knowledge share.
Speaking of knowledge sharing, documentation is a cornerstone of a well-run experimentation practice. What are some of the key details you capture?
Agreed, an extremely important aspect of an experimentation programme. Primarily to aid adoption and scalability of a programme. Apart from the obvious variation screenshots, hypothesis, date and results. We tag the information in a certain way.
- Behaviour type
- Segment
- Template page type
- Product/Service
- Region
- Market
This allows teams to view what has or hasn’t worked. The main purpose for us at the moment is to help springboard teams who may be new to CRO. Providing inspiration and educating them to run their own tests and experiences, again reducing the entry barrier. As we mature, it will become a playbook for future initiatives.
I like your tagging. For sure, if people can’t leverage learnings from the past, what’s the point, right?
How do you decide how many Experiments to run per month? What is not enough and what is too much?
Completely depends on the show size. 6 months from the show obviously the marketing teams are in the planning & strategy phase. The traffic volume and conversions are much lower during this period of the cycle.
There are two metrics essentially and a few variables we look at.
- Metrics including the number of users (not sessions) and number conversions.
- Team maturity, bandwidth and what support do they need in place to be set up for success.
Based on the data above, we can begin adjusting the strategy accordingly. The number of tests, areas of interest, hypothesis etc.
There are guidelines in place to help teams inform test cadence to avoid false positives/ negatives.
As a practice leader, how do you measure the performance of an Experiment program?
Data is only powerful if it’s converted into insightful customer knowledge, business intelligence, and informing an ever-improving value proposition.
Formulate a maturity model and benchmark the programme against the criteria you deem important. We have quite a robust maturity model with criteria such as management buy-in, technology, people skills, experiment quality.
This assessment allows us to understand the different show’s needs in terms of support. Do they require more training? What are their awareness levels? Is technology holding them back? Holding these conversations with senior stakeholders helps us to strategically grow the programme.
In your opinion, what goes into an Experimentation roadmap or vision?
The experimentation roadmap needs to be underpinned by a business and behavioural objective. Business objective means staying close to the company and understanding what success looks like ‘defining success’.
The behavioural objective would be along the lines of are people utilizing the right methods, are experiments impacting behaviour and are we learning as a company.
The roadmap is then formulated to deliver that vision. Test quantity, areas of interest, type of tests, segments to target and aligning with marketing and product sprints to maximize ROI.
It’s time for the Lightning Round!
Bayesian or Frequentist?
Depends who I’m speaking to ;)
If you couldn’t work in Experimentation what would you do?
Sky Sports Anchorman (but not like Ron Burgundy, maybe the suit)
What is your biggest Experimentation-related pet peeve?
‘I did a test once and it didn’t work’
I see you’ve moved from the UK to the UAE. What food do you miss most from home?
Mum’s shepherd’s pie
What is your favourite discovery in the UAE?
The history of the region. I didn’t know much about the UAE before moving. The culture is fascinating.
Describe Tony in 5 words or less.
A man with no shins.
But a man with many talents. Tony, thank you for joining the conversation!