Learn A/B Testing Statistics, how to use Overall Evaluation Criterion, Twyman’s law, how much traffic do you need to run A/B tests, and other statistical concepts all Experimenters must know.
Category Archives: Announcements
Build or buy your Google Optimize replacement with Kenneth Kutyn
Are you looking to replace Google Optimize? On Experiment Nation’s recent podcast, Charlotte April Bomford chats with Kenneth Kutyn about replacing Google Optimize, how to select a new vendor, and whether it’s worth building your own Experimentation Platform (spoiler alert: the answer is usually no).
Sharing insights, AI and the future of CRO with Tim Thijsse
Tim Thijsse, a senior CRO (Conversion Rate Optimization) consultant at Orange Valley, discusses AI and the future of CRO, and the importance of incorporating qualitative research into company-wide processes. He emphasizes the challenge of linking user behavior and mindset to knowledge and information. Tim shares his background, starting as an interaction designer and later transitioning to UX design and CRO. He highlights the evolution of CRO, the need for a culture of experimentation, and the role of qualitative insights in optimization strategies. Tim also discusses the challenges companies face in adopting an experimentation mindset, such as aligning strategy, operations, and technical aspects, and the importance of having a common metric and collaboration among teams.
7 must-know CRO fundamentals based on running 22,000 tests with Ayat Shukairy
Ayat Shukairy emphasizes the importance of addressing the functional, social, and emotional aspects of customer needs when designing websites. She shares seven lessons learned from her experience in conversion rate optimization.
CRO Insights from a Meta-analysis of over 1000 A/B Tests with Georgi Georgiev
Georgi Georgiev performed a meta-analysis of over 1000 A/B tests and has a great deal to share about:
– What you should expect your win rate to be
– The benefits of sequential testing
– Why you should report on confidence intervals
Get Executive buy-in for your CRO program with Jonny Longden
Let’s face it, experimentation is completely misunderstood. The power and the benefit and potential of it is to innovate and develop a business. And when you think about it, that’s what experimentation has always been in relation to the scientific method. If you think about spaceflight, medicine, anything great that’s happened in the history of modern civilization, this has all happened as a result of the careful application of the scientific method.
A/B Testing Statistics issues CROs should ignore with Timothy Chan
Experimenters obsess about many A/B Testing Statistics concepts – but not all of them are worth worrying about. Statsig’s Timothy Chan shares his thoughts about test isolation, interaction effects and confidence levels – some of his thoughts may surprise you.
Client Relationship Tips from a CRO Agency Founder with Andra Baragan
In this presentation, Andra Baragan, the Founder of ONTRACK DIGITAL will walk you through what you need to know to build a strong, successful relationship with your clients and with your team.
You will learn:
– how to set expectations with your client from Day 1
– how to leverage operating procedures and automate your work
– how to communicate with clients and get stakeholder buy-in every time
What you need to know about Launching a CRO agency with John Ekman
10 lessons that helped John Ekman in Launching a CRO agency. John learned many lessons when launching his agency – he wants to share them with you.
Optimize Customer Lifetime Value – NOT Conversion Rates with Valentin Radu
After decades of acquisition marketing, companies are waking up to the fact that optimising the entire lifecycle of the customer is the way to go. That happens due to rising acquisition costs, cookie deprecation, siloes, company mentalities, etc.
In this session, Valentin will share insights from his 12 years of experience transitioning from CRO – website optimization to increase the conversion rate to CVO – experimenting towards improving the CLV.
Improve conversions with the Message-market Fit framework With Daphne Tideman
Message-Market fit is the idea that a message, whether it be a product, service or idea, needs to resonate with a specific target market in order to be successful. It involves finding the right message that effectively communicates the value proposition of a product or service to a particular audience, in a way that resonates with them.
Avoid AB Testing Statistics Mistakes with Max LI
A/B testing plays a critical role in decision-making – but so does AB Testing Statistics Mistakes. In data-driven companies, A/B tests not only prove positive impacts of product features (treatments) but also provide evidence to safely roll out neutral features that are not expected to move business metrics. When rolling out a neutral feature, we need to test if the treatment is non-inferior to control, and conventional A/B testing is not suitable in this case.
Stop having bad test ideas – with Bart Schutz
Add Exploration and Validation phases to your testing program – because most of your wins are fake.
How to be a good Experimentation LEADER with Cristina Pujol
Being an Experimentation leader requires a different set of skills. Vistaprint’s Cristina Pujol walks through the lessons she’s learned.
The Best CROs start with RESEARCH with Rommil Santiago
Where should you start testing, like with most processes, starting is usually the most challenging part.
Pragmatists will grow the CRO industry – not purists – with Matt Scaysbrook
This attitude has a negative impact on new entrants to the industry and sets completely the wrong tone for the next generation. The greater the participation in the industry, the stronger it will become – the wider that participation, the more ideas (and the more diverse those ideas) will come through. And that is only a good thing for us all longer-term.
The best CRO career advice
We asked the CRO community on LinkedIn and on our community Slack channel what is the best piece of career advice you’ve ever received. Here is what they said. Speak up “Always ask for more” – Khalil Guliwala “A closed mouth doesn’t get fed” – Anthony Wallace “You don’t find what you’re looking for becauseContinue reading “The best CRO career advice”
The top 3 things CROs forget to do with Alexandra Matheny
Many optimizers are good at using data to craft hypotheses. Great optimizers, however, build data-driven segments, analyze by business goals (not just test KPIs), and importantly, propose how to take action on campaign results.
The problem with the word, “Hypothesis” and the importance of HiPPOs with Nima Yassini
Having mature conversations with immature markets, the problem with the word, “Hypothesis”, the importance of HiPPOs with Nima Yassini
Experiment Nation’s most popular posts of 2022
Here are the most popular posts from the last 12 months!
The Top 20 Must-Meet CROs of 2022
Earlier this year, we asked the Experiment Nation community who were the CROs that they felt everyone should know because of their welcoming nature, openness to connecting folks with others, willingness to share their expertise, and being just all-around awesome folks. Here are the results along with what the community had to say about them.
We asked the CRO community: How do you document your experiments?
If you’ve ever wondered how CROs document their Experiments, we’ve collected some information that might be useful to you.
Introducing the CRO Qualification Certificate (Open Beta)
We are introducing Experiment Nation’s CRO Qualification Certification – an easy and FREE way for CROs to demonstrate their understanding of the industry that is managed by a neutral party.
The CRO Salary Report: 2022
Are you being paid fairly? Find out. Experiment Nation conducted a CRO Salary survey and we have the results!
Introducing the Weekly CRO Wordle
Do you love CRO? Do you love Wordle? You’re in luck. Every week, we publish a new CRO Wordle. We’ll do our best to keep the words 5 letters long, but we make no promises.
Learning Together
With intros from Shiva Manjunath, Andra Baragan, and Tracy Laranjo, CAPM, our new eBook compiles over 60 learnings from 2 years’ worth of interviews with Experimenters from around the world to help you get a leg up on becoming a better Experimenter. Learn how to get it FREE!
Looking back at 2021
Highlights from the last 12 months. Sure it’s a bit of a humblebrag, but it makes us feel good and we love feeling good.
We’ve updated the Directory, the Job Board, and Tools
We’ve been busy updating our site – hopefully making it easier for you to connect and interact with other Experimenters. Find out what we’ve been up to!
Experiment Nation: The Conference
Have you ever wished you could speak at a conference but speaking to a live audience was too intimidating? We’ve got an opportunity for you.
2021 sneak preview
I’ve decided to work with Experimenters from around the world to help give Experiment Nation a more intimate and connected feel. Here are the projects that are in the works.
Introducing the Experiment Nation Directory
Connecting to Experimenters from around the world just got a little easier
Experiment Nation is starting a podcast
What could go wrong?