Experimentation culture plays a pivotal role in shaping the decisions and progress of data-driven companies. From small startups to tech giants, the ability to effectively incorporate experimentation into product development and decision-making processes is crucial. In this article, we delve into the insights shared by Mark Eltsefon, a seasoned data scientist and experimentation evangelist, as he discusses his experiences in fostering experimentation culture at different companies and the challenges he has encountered along the way.
You may also like
From Data Novice to Experimentation Evangelist
Mark’s journey into the world of data science and experimentation began unexpectedly. After graduating from a renowned technical university in Russia, he embarked on his career as a software developer, with Excel as his primary tool. However, his passion for data science ignited when he came across Andrew Ng’s Stanford lectures. He realized that data-driven decision-making fascinated him, and this passion led him to his first job in a bank. It was there that Mark’s interest in experimentation took root. Over the years, experimentation became one of his core responsibilities, eventually propelling him to senior data scientist roles and an experimentation evangelist at various companies.
Experimentation Culture: TikTok’s Data-Driven DNA
At TikTok, Mark experienced a data-driven culture that revolved around experimentation. TikTok’s massive user base created an environment where data was not only a component of decision-making but the central pillar. Every new feature, even minor ones, couldn’t be implemented without data-backed experimentation. The experimentation culture at TikTok wasn’t confined to the data team; it extended across the entire company. Everyone had the opportunity to propose ideas that could be tested, fostering an environment where innovative concepts were encouraged and tested rigorously. This democratization of experimentation empowered employees and created a fertile ground for innovation.
Nurturing Experimentation Culture at Gelato
Mark’s journey continued at Gelato, an e-commerce company in the print-on-demand industry. The challenges and dynamics of building an experimentation culture differed from his TikTok experience due to the company’s size and infrastructure. Gelato’s smaller user base required a more tailored approach to experimentation. Mark highlighted the importance of building a shared understanding of data-driven decision-making, especially among non-technical stakeholders. Educating designers, product managers, and stakeholders about the significance of data in driving outcomes became a focal point.
Democratization and Quality: Key Pillars of Experimentation
Democratization of experimentation, allowing various teams and individuals to propose and test ideas, was a common theme throughout Mark’s experiences. However, he emphasized that while early involvement is valuable, maintaining the quality of experimentation is paramount. Quality hinges on three crucial aspects:
- Infrastructure: Accurate data is the foundation of experimentation. Without reliable data, the outcomes and insights drawn from experiments can be inaccurate or misleading.
- Business Quality: Clearly defined hypotheses, appropriate metrics, and a realistic understanding of the potential impact are essential for productive experimentation. Avoiding vague or baseless requests ensures that the experiments contribute meaningfully to the company’s goals.
- Experimentation Quality: Implementing rigorous statistical methods, avoiding biases, and conducting comprehensive health checks are critical to ensure the validity of experimental results.
Experimentation culture, while an abstract concept, is pivotal in shaping the trajectory of data-driven companies. Mark Eltsefon’s journey from a data novice to an experimentation evangelist sheds light on the challenges and rewards of fostering experimentation culture. Whether at TikTok with its democratized experimentation or at Gelato where education and quality were focal points, the common threads of democratization and quality emerged as foundational pillars. Embracing these principles can pave the way for companies to harness the power of experimentation and data-driven decision-making to drive innovation and growth.
Connect with Experimenters from around the world
We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.
- Adventures in Experimentation (9)
- Analysis (2)
- Announcements (48)
- Ask the Experiment Nation (11)
- Certified CROs (193)
- Community (2)
- Conversation Starters (2)
- CR-No (5)
- CRO Handbook (4)
- CRO Memes (18)
- Experiment Nation: The Conference RELOADED (41)
- Experimentation from around the world (2)
- For LinkedIn (121)
- Frameworks (2)
- Growth (14)
- ICYMI (2)
- Interviews with Experimenters (167)
- Management (1)
- Marketing (11)
- Opinion (8)
- Podcasts with Experimenters (15)
- Point/Counterpoint (1)
- Product Experimentation (5)
- Product Management (9)
- Profile (9)
- Question of the week (5)
- Sponsored by Experiment Nation (1)
- The Craft of Experimentation (1)
- The Cultures of Experimentation (1)
- The weekly buzz (13)
- The year in review (1)
- Uncategorized (352)
- Weekly Newsletter (183)
- Where I Started (2)
- Making the case for data-backed design featuring Brian Massey - September 30, 2023
- Announcing the 2023 Conference Awards - September 24, 2023
- How to Maximize Hypotheses Testing ft. Eduardo Marconi Pinheiro Lima - September 9, 2023