How to measure your experimentation
program ROI? (In French)
Category Archives: Weekly Newsletter
Evolved Search’s Simon Clark talks about the importance of UX and his thoughts about Google Optimize
Simon shares how communication and UX are important to any optimization practice.
Microsoft’s Tim Mehta talks about managing an Experimentation Program
Tim Mehta talks about how he leverages both quantitative and quantitative research to find frustration points to optimize.
Série – Estruturando uma área de CRO na sua empresa – episódio 3 (Participação especial André Vieira)
Neste terceiro episódio contamos com a presença de André Vieira, fundador da Looptimize. Ele nos conta um pouco mais da sua visão sobre:
- Qual o ponto de partida para começar a investir em CRO?
- Popularização da experimentação: heurísticas e boas práticas — pontos positivos e negativos
- A qual área CRO melhor pertence dentro de uma empresa?
Dexter’s Joris Bryon on Common Practices, Starting a CRO agency, and how to spot a CRO imposter
Tracy Laranjo asks Dexter’s Joris Bryon about how we need to stop saying Best practices and start saying Common Practices His tips on starting a CRO agency and working with a partner And how to spot a CRO imposter.
Rohit Rayala on how he supported over 25 clients and drove an average growth rate of 154% in 12 months
Rohit talks to us about a process called PSIP that hoped him drive 154% growth for over 25 clients.
Aditi Gupta talks about influencing product roadmaps with experimentation
Manage expectation to avoid headaches when adjusting roadmaps based on experiment results.
Série: Estruturando uma área de CRO na sua empresa – episódio 1
Nesse primeiro episódio da série, vamos discutir como caímos no mundo de Otimização de Conversão (CRO), algumas experiências até posições atuais e diferentes perspectivas entre exatas/humanas e seus benefícios.
The weekly buzz: July 30, 2021
👉 8 Common Pitfalls of Running A/B Tests by Leihua Ye, Towards Data Science
👉 A Conversation with Ronny Kohavi hosted by Marylin Montoya on AB Tasty‘s podcast “1,000 Experiments Club”
👉 Experimentation Culture Awards 2021: Learnings by Bithika Mehra, MBA, MSBA
How Save the Children Italy’s Valerio De Biasio leverages CRO to help save lives
CRO not only helps generate sales, it can help save and improve lives.
We’ve updated the Directory, the Job Board, and Tools
We’ve been busy updating our site – hopefully making it easier for you to connect and interact with other Experimenters. Find out what we’ve been up to!
CR-No: How to speed up your Experimentation Program
Rommil, Kenya, and Shiva discuss the challenges of speeding up a CRO program, as well as some of the ways they overcome them.
Farfetch’s Luis Trindade on how to build a strong Experimentation Culture
Farfetch’s Luis Trindade on how to build a strong Experimentation Culture.
Frequentist & Bayesian: Comparison and Practical Application (R script attached)
By Andrea Corvi Canon vs Nikon and iOS vs Android are just two modern examples of “dualisms” that divide crowds. The long-lasting Frequentist vs Bayesian fight is not any different and like the other two examples above, it’s still far from finding a winner. This doesn’t affect just the CRO world but the same typeContinue reading “Frequentist & Bayesian: Comparison and Practical Application (R script attached)”
Experiment Nation chats with the folks behind CRO Memes
Who are the people behind CRO Memes? We don’t know either – but Tracy did recently chat with the elusive group to learn a bit about them.
Experiment Nation: The Conference
Have you ever wished you could speak at a conference but speaking to a live audience was too intimidating? We’ve got an opportunity for you.
Polestar’s Oliver Wyman on building an Optimization team and Experimentation culture
Learn about how Oliver measures the performance of his team, ensures optimization is connected across functions, and what he’d do if he didn’t run experiments.
ConvertCart’s Shaivi Sahay on figuring out what to test first
Learn about how Shaivi approaches new CRO challenges.
The weekly buzz: June 16, 2021
👉 How to Measure the Impact of Changes to the Digital Experience by Andrew Millett from Blast Analytics
👉 Don’t Be Intimidated By Sample Size, Learn About It! by Kelly Wortham from Search Discovery
👉 It takes a flywheel to fly by Aleksander Fabijan, Benjamin Arai, Pavel Dmitriev & Lukas Vermeer
Maulin Shah shares his key elements of an experimentation roadmap
Maulin Shah shares his key elements of an experiment-driven roadmap
What Experiment Nation is listening to this summer
Summer is upon us, and with it, longer days – and hopefully some time, here and there, to relax and enjoy the weather. While Experiment Nation seeks to share content that helps the community think about concepts it may have never considered before, we are here to also help connect us as a larger community.Continue reading “What Experiment Nation is listening to this summer”
Willful’s Tracy Laranjo on how the Experimentation Community has made her a better Experimenter
Tracy also shares a peek behind the curtain of what it’s like to work on Experiment Nation.
The weekly buzz: June 2, 2021
👉 How to Measure the Impact of Changes to the Digital Experience by Andrew Millett from Blast Analytics
👉 Don’t Be Intimidated By Sample Size, Learn About It! by Kelly Wortham from Search Discovery
👉 It takes a flywheel to fly by Aleksander Fabijan, Benjamin Arai, Pavel Dmitriev & Lukas Vermeer
Sam Media’s Kudakwashe Shamu on how designers have to get out of their comfort zones
Kudakwashe shares his advice to designers out there who haven’t yet embraced Experimentation.
The weekly buzz: May 26, 2021
👉 Upsides to Unshipping: The Art of Removing Features and Products by Reforge
👉 Browser Privacy Policy Updates – Optimization Effects by Kelly Wortham and Cory Underwood
👉 Visual Hierarchy: A 2021 Optimization Guide For Marketers by Unbounce
⏰ Two days to go.. If you haven’t already, submit your case in individual, team or organization wide categories for Experimentation Culture Awards by May 28th. 📢
👉 Upsides to Unshipping: The Art of Removing Features and Products by Reforge
👉 Browser Privacy Policy Updates – Optimization Effects by Kelly Wortham and Cory Underwood
👉 Visual Hierarchy: A 2021 Optimization Guide For Marketers by Unbounce
⏰ Two days to go.. If you haven’t already, submit your case in individual, team or organization wide categories for Experimentation Culture Awards by May 28th. 📢
Adventures in Experimentation: A deep-dive into CRO tools
John and Marion deep dive into their favorite CRO tools, and the importance of a diverse experimentation portfolio.
The weekly buzz: May 21, 2021
👉 Multiplication instead of division: Molly Stevens and Lukas Vermeer team up to tell us why research and experimentation are the “peanut butter and jelly” of customer insights.
👉 What you can learn from one extra experiment: Kevin Dunn explains how you may miss out if you are only doing A/B testing, or following the Change One Single Thing (COST) approach to experimentation.
👉 How does Failure help you in Experimentation? Craig Sullivan shares Tesla’s experience with experimentation and failure.
📢 If you haven’t already, submit your case in individual, team or organization wide categories for Experimentation Culture Awards by May 28th. ⏰
Principais diferenças da atuação de CRO em agência e dentro da empresa
A cultura da otimização de conversão e experimentação tem crescido no Brasil nesses últimos anos e reunimos aqui alguns erros principais de quem está começando a atuar na área.
Amazon’s Joey Herzberg on ensuring that work is always aligned with what the customer wants
Joey talks about how to always be aligned to customer needs, working across functions and his drone work.
The weekly buzz: May 14, 2021
Here’s the latest Weekly Buzz. 🐝 Links in the doc!
👉 Why we use #experimentation quality as the main #KPI for our experimentation platform from Booking.com
👉 A/B Testing and Experimentation: Tactics From Matt Gershoff
📢 If you haven’t already, submit your case in individual, team or organization wide categories for Experimentation Culture Awards by May 21st. ⏰ And remember…
“It’s not about that one experiment, it’s about improving the process, structure, trustworthiness, democratization and motivation for experimentation and data-driven decision making.”
Kilo Health’s Lukas Petrauskas on how website copy is costing companies 30% of their website revenue
Lukas shares how he evaluates specialists, and how companies are missing out on 30% of their revenue.
Reprise Digital Spain’s Silvia Serrano on applying Maslow’s hierarchy of needs to CRO
Silvia shares how to apply Maslow’s hierarchy of needs to CRO as well as how to think about testing for value propositions.
5 erros comuns de iniciantes em otimização de conversão
A cultura da otimização de conversão e experimentação tem crescido no Brasil nesses últimos anos e reunimos aqui alguns erros principais de quem está começando a atuar na área.
Spencer Gray on how CRO is not a standalone service but rather a team sport
Spencer Gray shares how he grew the CRO department at 97th floor and how successful CRO works with other services.
Ana Catarina Cizilio tells us about CRO in Brazil and the mindset needed for CRO
Ana tells us about the differences between the US and Brazil, as well as about the mindset you need to succeed at CRO.
Product Experimentation: A chat with Moshe Mikanovsky about building competency and fluency with data (Part 2)
Jaya and Sid chat with Moshe Mikanovsky, a Product Manager with over 30 years of experience, about the importance of getting closer to the data by getting familiar with data tools and working with business intelligence.
Product Experimentation: Building a competency and fluency with data (Part 1)
Jaya and Sid chat about building competency with and a fluency with data to get a better idea of what’s happening with systems and how products are helping customers.
Question of the Week: Which of these menu pages would generate more orders?
Find out what our LinkedIn community thinks.
Specsavers’ Melanie Kyrklund on How to run a high-impact optimization program
In this episode, Rommil and Tracy interview Melanie Kyrklund from Specsavers on how to run a high-impact optimization program, how to align the program with a product’s vision, and how she likes black licorice.
Question of the Week: How would you collect emails and why?
Find out what our LinkedIn community thinks.
CR-No: Addressing the lack of direction and primary calls to action on pages
This week, Siobhan Solberg, Kenya Davis, Shiva Manjunath and Eddie Aguilar talk about how to address the lack of direction and primary calls to action on pages.
Question of the Week: Which product description page would drive more orders?
Find out what our LinkedIn community thinks.
CR-No: How to manage stakeholders and HiPPOs
Shiva, Eddie, Siobhan, and Kenya talk about how to manage HiPPOs and stakeholders.
Question of the Week: Which product description page would lead to more cart additions?
Find out what our LinkedIn community thinks.
Adventures in Experimentation: The importance of the voice of customer to conversion optimization – as well as to the entire business
Eden and Positive John speak about how recording your sessions with clients helps reduce the need for taking notes, how qualitative research is key to not just CRO but for all things business, and how no one likes pop-up surveys except for Positive John.
SplitBase’s Raphael Paulin-Daigle talks about convincing clients to invest resources towards optimization
Hi Raphael, thanks so much for taking the time to chat with us! How have you been? I’ve been good! While the COVID situation here in Montreal is making things a bit boring, business has been stronger than ever, and we work with super fun clients, so can’t complain about that side of things –Continue reading “SplitBase’s Raphael Paulin-Daigle talks about convincing clients to invest resources towards optimization”
Product Experimentation: Aligning the need to deliver and the need to learn with OKRs
Jaya and Sid explore: What does a good product experiment look like;
Aligning the need to deliver and the need to learn with OKRs; Why aren’t companies adopting the practice
Widerfunnel’s Chris Goward on how Experimenters need to focus less on the tools and more on strategy
Experimenters put so much emphasis on tool selection that they forget the importance of taking a step back and thinking about the bigger picture.
CR-No: Balancing between optimizing the entire customer journey vs. just the website
This week, the CR-No panel of CRO veterans talk about balancing between optimizing the entire customer journey vs just the website.
Point/Counterpoint: Exploring different experiment prioritization frameworks with Marion Ravel, Patrick Buffum and Trisha Utomi
On today’s episode of Point/Counterpoint, Trish leads Patrick and Marion through a discussion about: Their CRO New Year’s resolutions, and the pros and cons of several experiment prioritization frameworks.
Twilio’s Justin Coons shares what you need to know about SEO Experimentation
Rommil interviews Twilio’s Justin Coons and they talk about: Not only can you run experiments for SEO, but you also should, Which is better, short-form content or long-form content, and What metrics to optimize for when running SEO Experiments
Product Experimentation: Jaya and Siddharth explore how to Experiment during the Product Development Life Cycle
Jaya and Siddharth explore how Experimentation fits into the Product Development Life Cycle, how to work with challenging stakeholders, share stories about how they’ve leveraged Experiments to generate results.
Adventures in Experimentation: Eden and John answer the question, “How many experiments should I run?”
On today’s episode, John Ostrowski and Eden Bidani talk about how they respond when someone asks them how many experiments they should run and why John keeps a personal list of shame.
CR-No: Experiments should be backed by hypotheses – leave your best practices for your processes
How best practices in CRO should only apply to approaches and workflows and not necessarily what you should test and the danger of copying others’ “best practices”.
2021 sneak preview
I’ve decided to work with Experimenters from around the world to help give Experiment Nation a more intimate and connected feel. Here are the projects that are in the works.
AWA digital’s Johann Van Tonder on the experimentation industry’s maturity, approaches, and attitudes
A conversation with AWA digital’s Johann Van Tonder
Softtek’s Angeles Alonso Leverages UX Research to get the most out of Conversion Rate Optimization
A conversation with Softtek’s Angeles Alonso about Experimentation Comment mesurer le ROI de mon programme d’expérimentation? How to measure your experimentation program ROI? (In French) Evolved Search’s Simon Clark talks about the importance of UX and his thoughts about Google Optimize Simon shares how communication and UX are important to any optimization practice. Microsoft’s TimContinue reading “Softtek’s Angeles Alonso Leverages UX Research to get the most out of Conversion Rate Optimization”
WeTeachCRO’s Sam Irvine partners with clients and helps them not get bogged down with statistics
A conversation with WeTeachCRO’s Sam Irvine about Experimentation Comment mesurer le ROI de mon programme d’expérimentation? How to measure your experimentation program ROI? (In French) Evolved Search’s Simon Clark talks about the importance of UX and his thoughts about Google Optimize Simon shares how communication and UX are important to any optimization practice. Microsoft’s TimContinue reading “WeTeachCRO’s Sam Irvine partners with clients and helps them not get bogged down with statistics”
Podcast: Gartner’s Shiva Manjunath on how Conversion Rate Optimization and Best Practices are terribly named
A podcast with Gartner’s Shiva Manjunath about Experimentation CR-No: Balancing between optimizing the entire customer journey vs. just the website This week, the CR-No panel of CRO veterans talk about balancing between optimizing the entire customer journey vs just the Read more Point/Counterpoint: Exploring different experiment prioritization frameworks with Marion Ravel, Patrick Buffum and TrishaContinue reading “Podcast: Gartner’s Shiva Manjunath on how Conversion Rate Optimization and Best Practices are terribly named”
Podcast: The anniversary episode
Experiment Nation’s new Canadian co-host interviews Rommil CR-No: Balancing between optimizing the entire customer journey vs. just the website This week, the CR-No panel of CRO veterans talk about balancing between optimizing the entire customer journey vs just the Read more Point/Counterpoint: Exploring different experiment prioritization frameworks with Marion Ravel, Patrick Buffum and Trisha UtomiContinue reading “Podcast: The anniversary episode”