Seven lessons from running 22,000 AB tests with Ayat Shukairy - Experiment Nation

Seven lessons from running 22,000 AB tests with Ayat Shukairy

Home / Experiment Nation: The Conference RELOADED / Seven lessons from running 22,000 AB tests with Ayat Shukairy

The average success rate of most AB testing programs hovers around 12%. Yep, for every 100 experiments you run, only 12 of them will generate an increase in conversions. That is worse than a coin toss.

Too many companies start AB testing hoping that it will deliver a great increase in conversions. A few months later, they are bitterly disappointed.

In this session, I will share the lessons we learned by analyzing over 22,000 AB tests on 700 CRO projects. These lessons paint the common patterns that helped us deliver upwards of 50% success rates on AB testing programs.

The lessons provide a strategic roadmap for a CRO project as well as tactical details needed while running the day-to-day operations.

Key takeaways

  • Attendees will learn what it really takes to run a successful CRO program (Time, resource, management, etc).
  • Attendees will learn the right balance between strategy and tactics when running an experimentation program.
  • Attendees will see an actual project management board running a CRO program

We’d love your feedback. Tell us what you thought of the session and of our conference for a chance to win a free Customer Value Optimization course from and a chance to be featured on Send us your feedback here:


Connect with Experimenters from around the world

We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.

You may also like

Tips on running a Test-to-Learn Experimentation Program with Shiva Manjunath

Shiva will run through WHY you want to be running an experimentation program based on insight generation and not simply Read more

Making product decisions with bayesian analysis

Positive John dives into the suitability of Bayesian inference for product decisions.

Women in experimentation: What holds us back?

In this session – recommended for women in experimentation and those working with them – Bithika will help identify blind Read more

Migrando para a área de experimentação vindo do Jornalismo (participação especial Fabricio Migues)

Hoje começamos um novo papo em nosso podcast: como migrar para a área de experimentação? Nosso convidado especial é Fabricio Read more