Weekly Newsletter Archives - Experiment Nation

CRO não é um profissional e sim um processo (participação especial André Gillich)

Nesse episódio nosso convidado André Gillich, Head of CRO, compartilha sua experiência na Jussi crescendo a cultura de experimentação, começando pela nova abordagem em que “CRO não é um profissional e sim um processo.”Exploramos o impacto da cultura “Agil” e o desafio do foco em outputs não outcomes.Fechamos o podcast de forma prática compartilhando iniciativas tomadas em nossas empresas para acelerar a flywheel de experimentação.

Experiment Nation: The Conference RELOADED :: The Science of Business with Jonny Longden

Our industry is focused on the individual elements of what we do and how we do it, but not why we do it and the overall philosophy for it. This is part of the reason that businesses don’t give it the traction it deserves; senior business leaders are not interested in the execution, they’re interested in the overall benefit.

Specsavers’ Steph Le Prevost on growing a top UK CRO program

Steph walks us through the challenges of growing Specsavers’ CRO team and how she overcame them, her experimentation red flags, and her take on if CRO certifications are really worth it. Plus: A comparison of the CRO job market between Canada vs. the UK, including how well you can expect to be paid as an optimizer.

Microsoft Digital Stores’ Tim Mehta on strategies for low and high maturity experimentation programs

Tim shares his tips for running low and high maturity experimentation programs, what it’s like to optimize in-house vs. agency, when it is (and is not) ok to make compromises, and to how make your program’s results actually drive action. Plus: Tim’s take on those controversial UI polls on LinkedIn.

Conversation Starters of the Week – October 14, 2021

Every week, Rommil Santiago shares interesting LinkedIn posts that he runs across to help spark ideas and conversation. Here are this week’s finds:


1) From Diane Wiredu

Most landing pages suck.

Prospects are scrambling for the exit button because of sloppy copy, confusing design and all the value buried down the page.

All mistakes that could be easily avoided with some key fixes and a better understanding of how to approach messaging. I’ve got my theory…

But I want to know… what do you think is the most important part of a landing page?

Join the conversation: https://www.linkedin.com/posts/dianewiredu_copywriting-marketing-cro-ugcPost-6853260391917740032-VWd6/


2) From Yongi Barnard

Yongi Barnard

👉Best practices are killing your website’s conversions. So STOP! 🛑

Here’s what you should do instead:👇🏽

✔Use your analytics to see where the bottlenecks are

✔Research. Research. Research.

✔Talk to your sales team. Know their tricks. Then figure out a way to recreate that on your website.

✔Watch how your website visitors behave using a heat map

👉And most importantly, don’t get too attached to your web pages.

👉Acknowledge that your website is a living specimen. And it’s a testing ground.


3) From Tim Mehta

Is anyone using Jira (and Confluence) to manage their experimentation program?

Would love to learn how y’all are leveraging it!

Join the conversation: https://www.linkedin.com/posts/timmehta_experimentation-cro-activity-6852632733143961600-VrtV


4) From Alexander Atzberger

Check this out, this is very cool – if you like economics! Yesterday David Card, Joshua Angrist and Guido Imbens won the 2021 Nobel economics prize for pioneering “natural experiments” to show real-world economic impact. Then my good friend Lalitha Rajagopalan pointed out to me that Guido Imben’s wife Susan Athey teaches economics at Stanford and publishes on A/B Testing and experimentation. In one of her notes (link below) she writes: “The term (A/B Testing) is widely used in the tech industry, where NOTHING changes at tech firms like Google, Facebook, Amazon, etc. without the change going through an A/B testing process.”

Join the conversation: https://www.linkedin.com/posts/aatzberger_how-do-academic-economists-use-ab-testing-activity-6854073260305379328-KLcQ


5) From Gayle Allen, Ed.D., MBA

Why are experiments one of the most overlooked forms of innovation? Stefan Thomke and I talk about this on Curious Minds at Work https://bit.ly/3oab406 

Join the conversation: https://www.linkedin.com/posts/gayleallen_leadership-innovation-experimentation-activity-6853343684566544384-cM4Q

Conversation Starters of the Week

Every week, Rommil Santiago, shares interesting LinkedIn posts that he runs across to help spark ideas and conversation. Here are this week’s finds:


1) From Ronny Kohavi

“In software, we often abort A/B tests (randomized controlled experiments) that are clearly negative and hurt users (https://lnkd.in/eWuqBVw), but we never abort a positive experiment; if anything, we would either start a larger experiment or a replication run. 

Merck announced Oct 1st that the results of Molnupiravir, an Oral Antiviral drug for COVID-19, are so good, they cut the trial short (https://lnkd.in/gUwqeBVA).

Health care is known to move slow, but there is also a push for more evidence-based health care. Is this such an example where we could do better by continuing the trial, and/or running a larger trial UNTIL Emergency Use Authorization is given? I’d love to hear from experts about why this decision makes sense in health care but not in the software world.”

Join the conversation: https://www.linkedin.com/posts/ronnyk_experimentguide-abtesting-healthcaredata-activity-6850519831188516864-tHKl


2) From Jonny Longden

“I know it’s an old debate, but it’s an interesting one nevertheless: should CRO be called that or should it be called something else?

It’s not just semantics and detail – the issue is much deeper.

The fact is that most people view CRO as a kind of hacking and tactical ‘tweaking’ of small stuff, which is 100% wrong and means they are missing out on massive potential benefit.

What it really is, is simply using the scientific method to de-risk everything you do through proof. This can fuel big things and it can also improve a tonne of stuff which isn’t conversion.

So no, the name doesn’t do it any favours.”

Join the conversation: https://www.linkedin.com/posts/jonnylongden_should-cro-be-called-cro-activity-6850789848748449794-Gaor


3) From Aditi Gupta

“Product growth is not just top funnel acquisition, it happens at multiple touchpoints through out the customer journey. In my session for Product Drive this year I will talk about the 6 levers of growth and share some best practises around them.
Sign up here – https://lnkd.in/gazCg38k

Join the conversation: https://www.linkedin.com/posts/aditigupta02_product-growth-is-not-just-top-funnel-acquisition-activity-6850484089280180224-3HIz


4) From Jason Vana

“Can you fit your brand strategy on 1 page?

If not, it’s likely too complex.”

Join the conversation: https://www.linkedin.com/posts/jasonvana_brandstrategy-activity-6850763126988644352–Hao

Migrando para a área de experimentação vindo do Jornalismo (participação especial Fabricio Migues)

Hoje começamos um novo papo em nosso podcast: como migrar para a área de experimentação? Nosso convidado especial é Fabricio Migues, CRO Manager na Small World Financial Services, conta pra gente como foi sair da área de jornalismo e relações públicas e adentrar na experimentação, como aproveitar seus aprendizados e conhecimentos da área de comunicação dentro de CRO e como isso o tem ajudado.

Shopify’s Shanelle Mullin on tough buy-in conversations and how we can empower future optimizers

Tracy Laranjo chats with Shopify’s Experimentation & Analysis Lead, Shanelle Mullin, about tough buy-in conversations, her take on the “I don’t dream of labour” trend, making experimentation more representative of diverse voices, and how we can empower the next generation of optimizers.

Série: Estruturando uma área de CRO na sua empresa – episódio 2

Neste terceiro episódio contamos com a presença de André Vieira, fundador da Looptimize. Ele nos conta um pouco mais da sua visão sobre:

  • Qual o ponto de partida para começar a investir em CRO?
  • Popularização da experimentação: heurísticas e boas práticas — pontos positivos e negativos
  • A qual área CRO melhor pertence dentro de uma empresa?  

Série – Estruturando uma área de CRO na sua empresa – episódio 3 (Participação especial André Vieira)

Neste terceiro episódio contamos com a presença de André Vieira, fundador da Looptimize. Ele nos conta um pouco mais da sua visão sobre:

  • Qual o ponto de partida para começar a investir em CRO?
  • Popularização da experimentação: heurísticas e boas práticas — pontos positivos e negativos
  • A qual área CRO melhor pertence dentro de uma empresa?  

Frequentist & Bayesian: Comparison and Practical Application (R script attached)

By Andrea Corvi Canon vs Nikon and iOS vs Android are just two modern examples of “dualisms” that divide crowds. The long-lasting Frequentist vs Bayesian fight is not any different and like the other two examples above, it’s still far from finding a winner. This doesn’t affect just the CRO world but the same typeContinue reading “Frequentist & Bayesian: Comparison and Practical Application (R script attached)”

What Experiment Nation is listening to this summer

Summer is upon us, and with it, longer days – and hopefully some time, here and there, to relax and enjoy the weather. While Experiment Nation seeks to share content that helps the community think about concepts it may have never considered before, we are here to also help connect us as a larger community.Continue reading “What Experiment Nation is listening to this summer”

The weekly buzz: May 26, 2021

👉 Upsides to Unshipping: The Art of Removing Features and Products by Reforge

👉 Browser Privacy Policy Updates – Optimization Effects by Kelly Wortham and Cory Underwood

👉 Visual Hierarchy: A 2021 Optimization Guide For Marketers by Unbounce

⏰ Two days to go.. If you haven’t already, submit your case in individual, team or organization wide categories for Experimentation Culture Awards by May 28th. 📢

👉 Upsides to Unshipping: The Art of Removing Features and Products by Reforge

👉 Browser Privacy Policy Updates – Optimization Effects by Kelly Wortham and Cory Underwood

👉 Visual Hierarchy: A 2021 Optimization Guide For Marketers by Unbounce

⏰ Two days to go.. If you haven’t already, submit your case in individual, team or organization wide categories for Experimentation Culture Awards by May 28th. 📢

The weekly buzz: May 21, 2021

👉 Multiplication instead of division: Molly Stevens and Lukas Vermeer team up to tell us why research and experimentation are the “peanut butter and jelly” of customer insights.

👉 What you can learn from one extra experiment: Kevin Dunn explains how you may miss out if you are only doing A/B testing, or following the Change One Single Thing (COST) approach to experimentation.

👉 How does Failure help you in Experimentation? Craig Sullivan shares Tesla’s experience with experimentation and failure.

📢 If you haven’t already, submit your case in individual, team or organization wide categories for Experimentation Culture Awards by May 28th. ⏰

The weekly buzz: May 14, 2021

Here’s the latest Weekly Buzz. 🐝 Links in the doc!

👉 Why we use #experimentation quality as the main #KPI for our experimentation platform from Booking.com

👉 A/B Testing and Experimentation: Tactics From Matt Gershoff

📢 If you haven’t already, submit your case in individual, team or organization wide categories for Experimentation Culture Awards by May 21st. ⏰ And remember…

“It’s not about that one experiment, it’s about improving the process, structure, trustworthiness, democratization and motivation for experimentation and data-driven decision making.”

Adventures in Experimentation: The importance of the voice of customer to conversion optimization – as well as to the entire business

Eden and Positive John speak about how recording your sessions with clients helps reduce the need for taking notes, how qualitative research is key to not just CRO but for all things business, and how no one likes pop-up surveys except for Positive John.

SplitBase’s Raphael Paulin-Daigle talks about convincing clients to invest resources towards optimization

Hi Raphael, thanks so much for taking the time to chat with us! How have you been? I’ve been good! While the COVID situation here in Montreal is making things a bit boring, business has been stronger than ever, and we work with super fun clients, so can’t complain about that side of things –Continue reading “SplitBase’s Raphael Paulin-Daigle talks about convincing clients to invest resources towards optimization”

Point/Counterpoint: Exploring different experiment prioritization frameworks with Marion Ravel, Patrick Buffum and Trisha Utomi

On today’s episode of Point/Counterpoint, Trish leads Patrick and Marion through a discussion about: Their CRO New Year’s resolutions, and the pros and cons of several experiment prioritization frameworks.

Twilio’s Justin Coons shares what you need to know about SEO Experimentation

Rommil interviews Twilio’s Justin Coons and they talk about: Not only can you run experiments for SEO, but you also should, Which is better, short-form content or long-form content, and What metrics to optimize for when running SEO Experiments

Product Experimentation: Jaya and Siddharth explore how to Experiment during the Product Development Life Cycle

Jaya and Siddharth explore how Experimentation fits into the Product Development Life Cycle, how to work with challenging stakeholders, share stories about how they’ve leveraged Experiments to generate results.

Softtek’s Angeles Alonso Leverages UX Research to get the most out of Conversion Rate Optimization

A conversation with Softtek’s Angeles Alonso about Experimentation The CRO Meme of the Week(ish): #002 See the best CRO Meme of the week as voted by the community! Experiment Nation: The Conference RELOADED :: Five things that piss me off about eCommerce and how to fix them with Jon Ivanco A quick and dirty guideContinue reading “Softtek’s Angeles Alonso Leverages UX Research to get the most out of Conversion Rate Optimization”

WeTeachCRO’s Sam Irvine partners with clients and helps them not get bogged down with statistics

A conversation with WeTeachCRO’s Sam Irvine about Experimentation The CRO Meme of the Week(ish): #002 See the best CRO Meme of the week as voted by the community! Experiment Nation: The Conference RELOADED :: Five things that piss me off about eCommerce and how to fix them with Jon Ivanco A quick and dirty guideContinue reading “WeTeachCRO’s Sam Irvine partners with clients and helps them not get bogged down with statistics”

Podcast: Gartner’s Shiva Manjunath on how Conversion Rate Optimization and Best Practices are terribly named

A podcast with Gartner’s Shiva Manjunath about Experimentation CR-No: Balancing between optimizing the entire customer journey vs. just the website This week, the CR-No panel of CRO veterans talk about balancing between optimizing the entire customer journey vs just the Read more Point/Counterpoint: Exploring different experiment prioritization frameworks with Marion Ravel, Patrick Buffum and TrishaContinue reading “Podcast: Gartner’s Shiva Manjunath on how Conversion Rate Optimization and Best Practices are terribly named”

Podcast: The anniversary episode

Experiment Nation’s new Canadian co-host interviews Rommil CR-No: Balancing between optimizing the entire customer journey vs. just the website This week, the CR-No panel of CRO veterans talk about balancing between optimizing the entire customer journey vs just the Read more Point/Counterpoint: Exploring different experiment prioritization frameworks with Marion Ravel, Patrick Buffum and Trisha UtomiContinue reading “Podcast: The anniversary episode”