Where should I start with testing?

Like with most processes, starting is usually the most challenging part. You know the feeling. It’s when you don’t have your flywheel going quite yet and you’re not sure at what stage to jump in. When it comes to Experimentation, the usual places I see people jump in are at the Research, Hypotheses creation, orContinue reading “Where should I start with testing?”

Best Practices are a starting point – not an excuse to avoid testing

You hear it often, “follow Best Practices” – where a best practice is something that has been generally accepted as the standard way of doing things to help maintain a certain level of quality. Interestingly, you’ll also often hear CROs arguing about them. But why would CROs, who are dedicated to improving the quality ofContinue reading “Best Practices are a starting point – not an excuse to avoid testing”

How many tests should you run per month?

Collin Crowel recently asked the community, “Is test velocity overrated?“. Where test velocity is the number of tests launched over a period of time. About 70% of respondents said “False” and I’m here to tell you that I’m firmly in the “it depends” camp”. Generally speaking, the goal of any Experimentation program should be toContinue reading “How many tests should you run per month?”

Meta-analysis: Do I have to run my test yet again?

Recently, there’s been a bit of buzz within the experimentation community around the concept of meta-analysis. For those unfamiliar with the concept, in a nutshell, it’s about looking at similar experiments altogether to pull insights. In fact, meta-analysis is actually considered the most trustworthy type of analysis out there. That said, it’s still prone toContinue reading “Meta-analysis: Do I have to run my test yet again?”

Ask the Experiment Nation – Should we build our own Experimentation Platform?

Q: Should we build our own Experimentation Platform? Recently, Davis Treybig posted an article about the gap between the best Experimentation Platforms vs. those the rest of us have. In short, he writes about how Experimentation tooling hasn’t kept up with the industry. This got me thinking. Does this mean you should build your ownContinue reading “Ask the Experiment Nation – Should we build our own Experimentation Platform?”

Ask the Experiment Nation – How do you respond to someone who says, “Let’s not experiment, it just slows us down. Can’t we just put it on our webpage?”

Q: How do you respond to someone who says, “Let’s not experiment, it just slows us down. Can’t we just put it on our webpage?” Petra Lylander asked our Slack community this exact question recently which inspired this post. I have to start by clarifying that just putting something out there and seeing what itContinue reading “Ask the Experiment Nation – How do you respond to someone who says, “Let’s not experiment, it just slows us down. Can’t we just put it on our webpage?””

Ask the Experiment Nation – Can you run tests on sites with little traffic?

Q: Can you run tests on sites with little traffic? In a recent post, Magnet Monster posted an interview with Jeremy Horowitz about what to do when you don’t have enough data. You can check out the post here. Jeremy goes on to say that we aren’t in a science lab and we don’t haveContinue reading “Ask the Experiment Nation – Can you run tests on sites with little traffic?”

Question of the week: When is it OK to run 2 A/B tests simultaneously on a site? Is it ever OK?

Q: When is it OK to run 2 A/B tests simultaneously on a site? Is it ever OK? We recently posed this question to the Experiment Nation community on LinkedIn. This is what they said. Surprisingly, people can get quite passionate around this topic. As you can imagine, there are 2 camps of thinking: “Duh,Continue reading “Question of the week: When is it OK to run 2 A/B tests simultaneously on a site? Is it ever OK?”

Question of the week: How do I make the most impact in my role when I only own optimization for a small part of the user journey?

Q: How do I make the most impact in my role when I only own optimization for a small part of the user journey? We recently posed this question to the Experiment Nation community on LinkedIn and this is what they said. This is a common question I get from folks across all disciplines, beContinue reading “Question of the week: How do I make the most impact in my role when I only own optimization for a small part of the user journey?”