A semi-regular segment where I chat with experimentation professionals from around the world.

While there is an increasing awareness of the need for experimentation?—?it can still be very challenging to find roles in the field. But sometimes, when you see the opportunity, you have to just make it happen.

This week we chat with Jason Yang?—?a Product Owner of Digital Performance and Insights at Telus. He works at the TELUS Digital office in Vancouver. He’s lives at the intersection of business and data, helping both sides see eye-to-eye.

---

Rommil: Hi Jason, thanks for taking the time to chat. Why don’t you share with us how you got into experimentation?

Jason: I have always been a supporter of constantly testing things out with a clear hypothesis. I actually don’t have a data analytics background, but in my previous roles, I was always part of a digital business management team. After working for two major retailers, I saw a common pain point for companies. Business and data folks didn’t speak the same language. Business people will say, data people don’t have the sense of urgency and don’t care about the money. Data people will say business people only care about money and ask dumb questions.

I saw this as an opportunity and created a role at TELUS Digital. My job is to translate business requirements into actionable stories so data people will have a clarity on definition of done. Also, I make sure to provide space and time for analysts and devs to do what they do best.

Wow, that takes a lot of initiative! From your perspective, what key skills should someone who’s interested in getting into experimentation have?

Curiosity and stoicism. Always be curious yet never get too high or low.

Based on my experience, a person with business/economics background with statistics tend to do well.

Testing methodology seems pretty straightforward, but knowing which method to use is the difficult part.

Here’s a tougher question?—?what does a company culture of experimentation mean to you?

It’s different at different places.

At small sized companies, they have access to all the tools so they can test the shit out of everything?—?yet they might not fully understand why they are testing.

Medium sized companies tend to struggle a lot. They tend to over-complicate testing culture. Having a thought leader in the company who can influence senior leadership of the benefits of experimentation is vital.

Large corporates do well. They might not have the most progressive experimentation culture yet they all know the importance of testing things. What they struggle with is the program and/or governance around experimentation.

Why do you think some companies have a hard time embracing a culture of experimentation?

• Lack of data literacy and curiosity
• Instant gratification culture
• Misunderstanding the intention of experimentation. For example “Let me prove you wrong” is the wrong way to approach experimentation
• Fear

If you could name 3 things that companies should do to not just embrace experimentation but to also see the rewards of it, what would they be?

• Understand different value currencies: People always argue…what is the value of doing this and that. Having a better understanding of the different currencies of value aside from monetary such as operational efficiency, relationship management, culture and confidence booster, etc. Data itself is a very rational thing yet users are emotional non-rational beings.
• Sell it properly: I hate it when people say, “I’m just stating fact”. If I say, based on current society’s BMI scale, you are a fat, unhealthy human being….just stating the fact isn’t going to help you achieve your goal. It comes down to how data people can communicate the information in the most consumable way to trigger curiosity and appreciation.
• Empathy: Just be nice and kind. We are not saving kids from burning building or performing a brain surgery. Have some… ?? (I don’t know what this is in English. Ask your Korean friends lol). Once you achieve that state, you can be more empathetic and people will want to hear what you have to say.

If you could go back in time, what advice would you give your younger experimentation self?

Work hard and be kind (This is from Conan O’Brian).

Finally, Bayesian or Frequentist?

Big fan of economics. Believer in the ceteris paribus approach that leads to agile methodology of ongoing iteration to improve whatever you do. In other words, bayesian.

It could be my lack of statistics knowledge but I just don’t believe that frequentist approaches exist. A common example is pinball lottery in the glass box and trying to figure out the probability. Frequentist will say, count all the pinballs and make sure we have “accurate” information before analyzing the data. I would ask, how do you define accurate? Do you have to measure the weight of each pinball? Do we have to do an impact analysis on each pinball when it collides with glass surface, what’s the potential reaction happening the pinball surface that can trigger anomaly behaviour? But again, I do not have formal training in this field. All I have is… child-like wonder lol

---