You hear it often, “follow Best Practices” – where a best practice is something that has been generally accepted as the standard way of doing things to help maintain a certain level of quality. Interestingly, you’ll also often hear CROs arguing about them. But why would CROs, who are dedicated to improving the quality ofContinue reading “Best Practices are a starting point – not an excuse to avoid testing”
Collin Crowel recently asked the community, “Is test velocity overrated?“. Where test velocity is the number of tests launched over a period of time. About 70% of respondents said “False” and I’m here to tell you that I’m firmly in the “it depends” camp”. Generally speaking, the goal of any Experimentation program should be toContinue reading “How many tests should you run per month?”
Challenging stakeholders are often hard to deal with. Here are 3 tips on how to make these interactions simpler.
Recently, there’s been a bit of buzz within the experimentation community around the concept of meta-analysis. For those unfamiliar with the concept, in a nutshell, it’s about looking at similar experiments altogether to pull insights. In fact, meta-analysis is actually considered the most trustworthy type of analysis out there. That said, it’s still prone toContinue reading “Meta-analysis: Do I have to run my test yet again?”
We’ve all been there. You have a problem in your funnel and can’t figure out how to fix it. Sometimes a minor mental-shift can help.
Q: Should we build our own Experimentation Platform? Recently, Davis Treybig posted an article about the gap between the best Experimentation Platforms vs. those the rest of us have. In short, he writes about how Experimentation tooling hasn’t kept up with the industry. This got me thinking. Does this mean you should build your ownContinue reading “Ask the Experiment Nation – Should we build our own Experimentation Platform?”
Q: Have you ever tried Wordle? I.e. How Wordle proves that Experimentation works I recently posted this on LinkedIn, but I thought I’d elaborate on the metaphor this week. What is Wordle?If you haven’t tried it yet, Wordle is a popular online game where you have to guess a 5-letter word within 6 tries –Continue reading “Ask the Experiment Nation – How Wordle proves that Experimentation works”
Q: How do I change people’s attitudes towards Experimentation? Of course, there are a million ways to answer this question that range from demonstrating wins, being inclusive, and evangelism. This week, I thought of sharing a couple of the little things you can do that, if done consistently over time, can start to realize gainsContinue reading “Ask the Experiment Nation – How do I change people’s attitudes towards Experimentation?”
Q: How do you respond to someone who says, “Let’s not experiment, it just slows us down. Can’t we just put it on our webpage?” Petra Lylander asked our Slack community this exact question recently which inspired this post. I have to start by clarifying that just putting something out there and seeing what itContinue reading “Ask the Experiment Nation – How do you respond to someone who says, “Let’s not experiment, it just slows us down. Can’t we just put it on our webpage?””
Q: Can you run tests on sites with little traffic? In a recent post, Magnet Monster posted an interview with Jeremy Horowitz about what to do when you don’t have enough data. You can check out the post here. Jeremy goes on to say that we aren’t in a science lab and we don’t haveContinue reading “Ask the Experiment Nation – Can you run tests on sites with little traffic?”
Q: When is it OK to run 2 A/B tests simultaneously on a site? Is it ever OK? We recently posed this question to the Experiment Nation community on LinkedIn. This is what they said. Surprisingly, people can get quite passionate around this topic. As you can imagine, there are 2 camps of thinking: “Duh,Continue reading “Question of the week: When is it OK to run 2 A/B tests simultaneously on a site? Is it ever OK?”
Q: How do I make the most impact in my role when I only own optimization for a small part of the user journey? We recently posed this question to the Experiment Nation community on LinkedIn and this is what they said. This is a common question I get from folks across all disciplines, beContinue reading “Question of the week: How do I make the most impact in my role when I only own optimization for a small part of the user journey?”