How do I change people’s attitudes towards Experimentation?

Of course, there are a million ways to answer this question that range from demonstrating wins, being inclusive, and evangelism. This week, I thought of sharing a couple of the little things you can do that, if done consistently over time, can start to realize gains in the long run.

Words matter

Experimenters and CROs often say that learning matters, but at the same time, when seeing a test that didn’t go the way they wanted, say something like “At least we learned something.” Let’s take a second to reflect on that sentence. “At least” implies something minimally acceptable, like a consolation prize. I implore you to think about putting learning at the forefront of how you speak to your clients and stakeholders. Something like, “Now we know” is far more powerful. It implies that experimentation, regardless of the outcome, is a step forward. Running a test is in of itself progress.

Framing can help remove bias

Georgi Georgiev recently posted about the importance of “blinding” – where you essentially don’t share which treatment was the control and which was the challenger. They are rather presented equally as variants. While this won’t work in all cases, there is great value in doing this. More often than not, it helps to remove biases in interpreting results. One should not underestimate an audience’s attachment to solutions – especially those they were involved in building. Blinding helps you facilitate the focus on the proper interpretation of the results – which helps you and your place of work get that much closer to making more evidence-based decisions.

See you in 2 weeks,

Rommil Santiago
Founder, Experiment Nation