A Conversion Conversation with nib Health Funds’ Jess Vandenbruggen

There are no silver bullets in this world?—?not one thing can solve every problem. This is the case with Experimentation. Alone it is useful, but when it’s combined with other tools such as research and analytics, Experimentation can be powerful.

I recently had the chance to chat with Jess Vandenbruggen from down under (aka Australia) about what she thinks Experimentation brings to the table, her love for yeast extract spread, and how everyone is going to be Experimenting.

---

Rommil: Hi Jess, thanks for taking the time to chat! How about we start off with a little bit about yourself and what you do?

Jess: Hi Rommil?—?thanks for inviting me to the conversation. I am a Digital Optimisation Manager at nib which has me heading up our digital experimentation program, analytics and optimisation projects. I am driven by solving problems in creative ways, understanding the capability of the technology and how it can be used to deliver quality experiences to our users. Most importantly, I love learning and am constantly looking to grow my knowledge

“We were making decisions solely based on user research, what management thought would be a good idea, analytics or industry insights. None of these things work well by themselves.”

There’s been an increase in interest in experimentation lately to help companies make better decisions. How do you leverage experimentation at nib to make that happen?

This has been a key driver to how experimentation has evolved at nib. My goal in starting up this program was to enable the business to make better, data-driven decisions. We were making decisions solely based on user research, what management thought would be a good idea, analytics or industry insights. None of these things work well by themselves.

That’s so true. When you have a hammer, it’s easy to think everything is a nail?—?when really, you need a toolbox.

Experimentation has allowed us to take these validated ideas and prove out what users ‘actually’ do which has resulted in great wins, surprises and grand failures.

Whilst the initial focus has always been on how much the winning experiment has earned us, over the past year I have come to a realisation that experimentation has an equally strong case in identifying the opportunity cost of implementing change without testing. For example, research suggested users wanted to be asked a certain question up front, it seemed like a ‘no brainer’ but we tested it anyway. The experiment failed significantly and had we implemented without testing we would have cost the business $13m over the year.

Experimentation is making sure that we know the impact of the changes we are making to our digital experience and ensure we are spending money on the right things.

“My goal in starting up this program was to enable the business to make better, data-driven decisions.”

How do you define learning in an experimentation context? And how do you share your learnings?

Whether an experiment is a failure, success or indifferent there is something to be learned from it. You learn if you identify why something did or did not work and incorporate into your future approach to work.

Sharing learnings is key to building and maintaining buy-in to your experiment program. We share our results on a regular basis (still trying to understand the right ‘regular’ is for this) and we are working on building out a more detailed experiment findings ‘hub’ to support the learning of our wider group and partners.

“Sharing learnings is key to building and maintaining buy-in to your experiment program.”

With so many frameworks out there?—?how do you prioritize where to experiment?

There are so many frameworks! I haven’t found one to suit our needs as yet but rather have been piecing together parts of all of them. Being able to identify potential revenue associated with experiments has been quite valuable in getting stakeholders’ attention. We are still working on mastering our understanding of effort required for some experiments in our program.

Experimentation as a role is relatively new at companies?—?what characteristics do you seek out in job candidates?

Working so heavily in this as my ‘everyday’ it is easy to forget that experimentation is not everywhere. Finding people is hard, especially as we aren’t located in the major cities. Looking for someone who can grow into this role is key, I look for candidates that think laterally when it comes to solving problems and have a growth mindset where they are wanting to learn and throw themselves into challenges whether it be technology or projects.

It’s time for the Lightning round!

Vegemite: Best thing ever? Or worst thing ever?

BEST

Bayesian or Frequentist?

Bayesian

If you couldn’t be in Experimentation?—?what would you be?

Perhaps overthinking this one, but is there such a thing as not being in Experimentation anymore?

Leading Digital Strategy and Experience would involve placing bets and experimenting, all innovation based work would involve experimentation. Any line of work that involves finding different ways to solve problems would involve some level of experimenting.

I think once you’re in, you’re applying it to every facet of your life, haha!

Ha! Jess, thank you.

---