A Conversion Conversation with De Nieuwe Zaak’s Marlies Wilms Floet

Experimentation involves many skills. It lives at the intersection of a scientific approach, stakeholder management, technology, and psychology. So it should come as a surprise to no one that for a company to excel at Experimentation it requires a strong Experimentation Culture. I recently spoke with Marlies about her 10 tips on how to nurture such a culture, how to build effective Experimentation teams and how to ensure Experimentation is delivering value.

---

Rommil: Hi Marlies, thanks for chatting with me! How have you been?

Marlies: Thank you for this initiative. Doing fine, although it’s a rollercoaster time now during the COVID-19 situation. Different circumstances like working at home, staying safe, helping others as much as you can and the uncertain perspective in the long term. But also a chance, we can show even more the added value of customer insights and business experimentation.

Let’s start with you sharing with us what you do and a little bit about yourself.

As an economic psychologist from origin, I always have been fascinated by (online) human behaviour. Started as an international qualitative researcher and after that gained lots of digital (marketing) experience being the e-commerce manager of a travel company for more than 10 years. Basing our marketing budget on ROI, conversion of the website was key, so therefore about nine years ago, I started with conversion rate optimization (CRO). At this moment I work seven years at a full-service digital company in the Netherlands (De Nieuwe Zaak), where with a team of CRO specialists we use data-driven optimization in product development as well as optimization of websites and digital marketing. We have clients in B-2-B, B-2-C as well as Non-profit organizations.

As companies start to embrace business Experimentation, what is your perspective on how to build outstanding Experimentation and Optimization teams?

In our full-service digital agency, we do strategy, product development and –optimization as well as marketing.

About three years ago, we changed from discipline-teams to multi-disciplinary teams centred around client-groups. First, CRO-specialists were mostly represented in more marketing-centred teams, about a year ago, we also have expanded CRO-specialists permanently in all product development teams. There are big advantages of having CRO-specialists in both product- and marketing teams, such as:

1. In the early stage of strategy and product development, we use our CRO or business experimentation processes and add value right from the start
2. In the early stage of product development, we already account for future marketing strategies, such as the right basis for data, content-management and personalization
3. In (marketing) optimization we can react very quickly as we have the developers in-house and our clients do not have to move around different teams or companies
4. Last, but not least by being integrated into different teams, it is easier to have all colleges embrace the ‘growth’ or business experimentation mindset.

What are some of the qualities that make great Experimenters?

First of all, we have in common that we get a ‘kick’ from optimizing the business value for our clients whether it is in extra value from winning experiments, or ‘saved’ money from losing experiments, gaining valuable insights and making awesome products which the users love to use and show a large customer lifetime value. Other things we have in common that we have a shared vision based on three points:

1. Insights-driven (using mixed methods of qualitative and quantitative data)
2. Customer-focused (combining end-users needs and business goals)
3. Process-driven (always following the steps of setting targets, getting insights, make and prioritize hypothesis, experimentation and validation, implement, learn and archive).

Besides the shared vision, in our ‘T-profiles’, we have a good mix of specializations (such as strategy, qualitative research, user testing, data analysts, A/B-test management, personalization a business case performance management). Further, it is important to have a good variation in soft skills and put all experimenters in their strength.

Experimentation is one of many pieces of the puzzle when it comes to satisfying customer needs?—?how do you ensure that all the practices are aligned and delivering value?

We always start from our vision (see above) and apply these in an agile environment.

This means that we are always adding value in small steps. Another important thing here is that our client (eg product owner or other stakeholders of the company) is part of everything we do. In most cases, the product owner is part of our team (literally works in the team, often physically a couple of days a week) and we have regular demo’s to C-level, stakeholders and encourage them for example to follow user-testing.

Personalization is such an important strategic initiative for many places right now?—?how do you approach Experimenting on Personalization? How do you know when you’ve gotten it right?

Depending on the maturity of an organization we approach (experimentation on) personalization differently. A ‘mistake’ we often see is that a company bought an expensive personalization tool, but does not have the data, traffic or people available to manage it in the proper way. So the key is to first get an understanding of the company’s strategy and maturity level and make a roadmap of the steps to be taken to come at the desired level. This means that for some clients we started personalization on channel-level (eg e-mail) and for others, we are optimizing user stories in an omnichannel environment.

The process in optimization and experimenting on personalization is not different from our ‘regular’ process. Personalization is integrated into our whole optimization roadmap.

How do you measure the performance of an Experimentation/Optimization team? And how do you nurture an Experimentation culture?

Besides (A/B-)testing, where you can calculate the wins or losses in clear KPI’s and business value, it is sometimes a challenge to measure short-term or long-term performance and to convince our clients to put budget in for example user testing (so if you have any good and recent studies about that, please share!).

About nurturing an experimentation culture, I can share my 10 tips:

1. Get C-level on board: e.g. involve them in user testing, ideation
2. Involve HR: make a solid (onboarding) program, hire the right people
3. Inspire: show simple examples
4. Listen: to ‘what’s in it for me’ for every individual
5. (Help to) define the relevant KPI’s
6. Set up a scalable process, which fit in existing workflows
7. Take care of good governance
8. Support with (automated) data
9. Celebrate success
10. Encourage a culture of failure = learning.

These are amazing. I love this list!

Changing gears. It’s time for the Lightning round!

Bayesian or Frequentist?

Bayesian

Who do you consider as leaders in the business Experimentation space?

My basic hero is Daniel Kahneman. Leaders now are several, depending on the topic. I always get very much inspiration from ‘Conversion Hotel’, a yearly conference in the Netherlands.

If you couldn’t work in Experimentation, what would you do?

Making the world a better place.

Describe Marlies in 5 words or less

Passionated in psychology, online behaviour and data-driven optimization ;-)

Thank you, Marlies, for joining the conversation!

---