Ask the Experiment Nation - How do you respond to someone who says, "Let’s not experiment, it just slows us down. Can’t we just put it on our webpage?" Experiment Nation

Q: How do you respond to someone who says, “Let’s not experiment, it just slows us down. Can’t we just put it on our webpage?”

Petra Lylander asked our Slack community this exact question recently which inspired this post.

I have to start by clarifying that just putting something out there and seeing what it does, is still an Experiment. Just not an efficient or well structured one. The data produced will be biased, or will take a long time to collect. So my comments below, are for when someone says, “Let’s not run a well-designed experiment, it just slows us down. Can’t we just put it on our webpage?”

With all that said, I get this question quite often. Here’s my take. I feel it boils down to how one measures “slowing things down”. If you prioritize the time to make something live, regardless of its quality, then the answer is yes. Proper Experimentation will absolutely increase the time it takes to get a something out. However, it will almost certainly shorten the time it will take to release something that performs.

Here’s how I think about it. Imagine you own a bakery that serves a neighbourhood and you decide to start offering cakes for the first time, but you don’t have a recipe. Then, you decide you don’t want to waste any time and choose to start selling cakes anyways based on recipes you found on the internet. You figure, if customers come back to buy your cake again, then you know you got the recipe right. If they don’t come back, then you got it wrong. Unfortunately, the chances of you making the best cake ever on your first try are very slim, so the time it will take you to produce a cake that customers will come back for is much longer than it should be. While I feel that this should be obvious to most people, for some reason, many think this doesn’t apply to their products or campaigns.

Another related comment I hear is, “Our competition is doing it, so we have to do it. It’s table stakes.” Here’s how I respond to that, “When you look at your competitors, you actually don’t know if it’s working for them.”

Returning to our bakery example, just because your competition is selling croissants doesn’t necessarily mean that they are selling well or generating profit. Each bakery serves a unique community and type of customer – market factors that you may not have. The same applies to your business.

How would you respond?

See you in 2 weeks,

Rommil Santiago
Founder, Experiment Nation

Continue the conversation on our Slack channel