WeTeachCRO's Sam Irvine partners with clients and helps them not get bogged down with statistics Experiment Nation

A conversation with WeTeachCRO’s Sam Irvine about Experimentation

Rommil: Hi Sam, how are you? Thanks for chatting today! How have you been?

Sam: Hey Rommil! I’ve been good thanks, all things considered. The UK is in a lockdown again, so I’m adjusting to that at the moment. How have you been?

I’m still standing! So Sam, I took a peek at your LinkedIn profile and I couldn’t help but notice that you used to be a police officer. I’d love to hear a bit about your career journey!

Sure. It’s been a bit of a long-winded career path. It certainly isn’t the usual way of falling into CRO, I must admit.

I initially went to university to study Forensic Computing & Security in an attempt to merge my enjoyment of digital and desire to work in law enforcement. However, I found that I wanted to be on the frontline of policing instead and left university to follow that ambition!

So I spent a few years in uniform policing, including a couple of years on a response team in London – a team responsible for taking all of the emergency calls. Definitely one of the most challenging jobs out there and a real eye-opener to the world.

I then took my detectives exam and worked in a Major Crime Unit investigating serious and complex crimes such as stabbings, robberies, and serious drug offences – to name just a few.

Then, out of pure luck, the opportunity to move into CRO was presented to me. It turns out running criminal investigations has quite a few crossovers with optimisation!

Haha – I guess that’s true – minus the stabbings of course. Could you share with our audience a bit about what is WeTeachCRO and how it stands out from the pack?

Yeah sure. WeTeachCRO is an optimization agency based just outside London. We currently work with quite a variety of clients, from maternity brands through to a high-street restaurant chain working on their online platforms.

I think it’s our partnership mentality that helps us to stand out amongst our client base. We won’t work for a client but we’re more than happy to work with a client. It’s that partnership that enables us to become an extension of a clients’ team and allows us to fully understand the business.

Very nice. Always great to hear when CRO is seen as a partner. As someone working agency-side, can you share with us some of the common challenges you face when you work with new clients?

Sure, new clients can always be a challenge! There can be a variety of things to overcome.

…the main challenge is to try and establish trust and build a personal relationship…

At an individual consultancy level, the main challenge is to try and establish trust and build a personal relationship. Without it, the relationship will break down. So that’s one area that I try to develop as quickly as possible.

At a higher level, integrating processes and ways of working can sometimes be a challenge. Especially if the client isn’t used to working in a methodical and structured manner.

Definitely. And getting new places onboard to new processes is never an easy feat. How do you go about bringing them around to your side?

I tend to start with a kick-off session which goes through some gentle educational pieces and rapport/trust building.

There are a couple of layers to building trust, in my opinion. So I try to build the relationship up just by getting to know the client and having a bit of a chat in the kick-off. It’s important to understand their position in the overall business, the challenges they face, and what pressures they’re under etc.

Showing that you understand them and building up a rapport can go a long way to developing that relationship. The other side of trust is that client’s want to see good positive results, so you have to prove you can deliver. A couple of early wins can be a good way of developing trust on a professional level.

Of course – most tests don’t win. So explaining how insightful “losing” tests can be is useful. I also like to reframe a losing test as one that potentially saved x amount of money vs what it would have cost long term if it was blindly implemented.

Statistics is a big part of Experimentation, but statistics can be very scary. How do you address those challenges?

Yeah, absolutely. The statistics can become a bit of a minefield. Especially with clients who don’t have any previous exposure to statistics or are in their optimization infancy. There can be a lot going on within optimization programmes so I try not to make it any more complicated than it needs to be.

I see it as my job to get bogged down in the statistics on the clients’ behalf and then make recommendations.

I see it as my job to get bogged down in the statistics on the clients’ behalf and then make recommendations. Clients often have a hard enough job navigating the bureaucracy of their business without me adding in a massive learning curve.

How do you go about measuring the ROI of your work to clients?

At WeTeachCRO we tend to use the velocity of testing as one measure and then projected incremental revenue as well (calculated from RPV extrapolation).

Here’s something I’ve been thinking about lately and I’d love to get your thoughts on it. Do you feel that the time of year has any impact to how open clients are to testing? If so, why do you think that is?

Yeah, I’ve definitely seen some clients shift their openness to testing when the holiday season comes around. I think it generally comes down to a clients’ attitude to risk.

The impact of Black Friday tends to depend on the client – some see it as business as usual – others get more nervous about the potential downside of testing and missing out on sales in whichever variation performs worse.

Most clients I work with implement a code freeze during the Xmas period though. This is usually a decision from the top and is an organizational attitude to risk. So running tests tends to come to a stop for a short time.

I hear that! Finally, it’s time for the Lightning Round!

Bayesian or Frequentist?

Haha, I’ll leave the debate to those who have far more knowledge than me. But I prefer Frequentist.

If you couldn’t work in Experimentation what would you do?

It would probably be something within the marketing spectrum. I’m geeking out on product positioning & narrative at the minute – so maybe that. Or, I’ve always fancied owning my own gym.

Marketing or a gym. Ha! You are a bit of a renaissance person.

Describe Sam in 5 words or less.

Always learning.

Love it. Great motto for life as well. Thank you for joining the conversation!

Connect with Experimenters from around the world

We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.