How do I change people’s attitudes towards Experimentation?

Of course, there are a million ways to answer this question that range from demonstrating wins, being inclusive, and evangelism. This week, I thought of sharing a couple of the little things you can do that, if done consistently over time, can start to realize gains in the long run.

Words matter

Experimenters and CROs often say that learning matters, but at the same time, when seeing a test that didn’t go the way they wanted, say something like “At least we learned something.” Let’s take a second to reflect on that sentence. “At least” implies something minimally acceptable, like a consolation prize. I implore you to think about putting learning at the forefront of how you speak to your clients and stakeholders. Something like, “Now we know” is far more powerful. It implies that experimentation, regardless of the outcome, is a step forward. Running a test is in of itself progress.

Framing can help remove bias

Georgi Georgiev recently posted about the importance of “blinding” – where you essentially don’t share which treatment was the control and which was the challenger. They are rather presented equally as variants. While this won’t work in all cases, there is great value in doing this. More often than not, it helps to remove biases in interpreting results. One should not underestimate an audience’s attachment to solutions – especially those they were involved in building. Blinding helps you facilitate the focus on the proper interpretation of the results – which helps you and your place of work get that much closer to making more evidence-based decisions.

You might also like:   Widerfunnel's Chris Goward on how Experimenters need to focus less on the tools and more on strategy

See you in 2 weeks,

Rommil Santiago
Founder, Experiment Nation


Connect with Experimenters from around the world

We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.


Advertisment


How Europe leverages AI for optimization with Noémie Sauvage

Noémie Sauvage shares her industry knowledge about optimization in Europe. Check it out!

How Wordle proves that Experimentation works

If you haven’t tried it yet, Wordle is a popular online game where you have to guess a 5-letter word Read more

Pizza Dough CRO with Simon Clark

Optimization and Pizza Making are the same and Simon will help you understand how and why it is beneficial to Read more

Can you run tests on sites with little traffic?

From a pure experimentation perspective - one absolutely can forego using statistics and just see what happens. We can believe Read more

Rommil Santiago