Kotn’s Dan Gray: Experimentation doesn’t have to stay online when it comes to Growth Experiment Nation

A Conversion Conversation with Kotn’s Dan Gray

There are no certainties in life. Those with enough grit to make a go of it at startups know this fact very well. With limited time and limited resources, startups still know they have to Experiment?—?in every channel?—?to find Growth. Gone are the days of Growth Hacks?—?today’s Growth focuses on Profitability and well-planned tests. I recently spoke to Dan whose heading up Growth at Kotn about Experimentation at startups and our common dislike of the term “Growth Hack”.

Rommil: Hey Dan! Thank you for taking the time to chat with me today, how are you?

Dan: I am doing well?—?feeling very energized by the new year. Thanks so much for asking me to be a part of your project!

How about we start with you telling us a bit about yourself and what you do over at Kotn?

So I am currently the Director of Growth at Kotn. I actually just joined the company about a month ago, previously having worked at Well.ca and Opencare in various growth positions. My role at Kotn is to build a performance marketing team that is primarily focused on new customer acquisition across the business.

“…people have been embedding experimentation thoughtfully in their marketing for over a century.”

Very nice?—?I’m sure you’ll crush this as well!

So, a lot of people understand the idea behind A/B testing ads?—?but not many know how to use experiments to discover new channels altogether. Can you share with us how you leverage experimentation to discover new opportunities?

I think it’s worth noting that this notion of A/B testing is not new. Over the holidays I was rereading Claude Hopkins’ Scientific Advertising and it was a great reminder that people have been embedding experimentation thoughtfully in their marketing for over a century. The reason I mention this is because experimenting to find a new channel is actually not that different than testing a new ad or piece of copy. While the scale and cost is often a lot bigger, I generally use similar processes regardless of where I’m testing.

Available on Amazon (I swear this isn’t an affiliate link).

Specifically for new channels where I would encourage teams to test, is areas of the marketing landscape that they can potentially own. We’ve heard it over and over again about how Facebook and Google have a duopoly, but you still have companies that are winning by testing executions that aren’t currently owned. I actually think your team at your company is doing a terrific job of this! Continuously investing in owned properties like your stores or websites and building new ones is a great way to grow sustainably.

That’s so nice of you to say! We are definitely working hard at investing and Experimenting as you say. We’re always learning something new.

For the benefit of those relatively new to Growth, based on your experience, what are 3 of the top things that folks should absolutely know about experimentation in order to succeed?

Start with a hypothesis. I’m guessing that every year billions of marketing dollars are wasted on tests that are set up without a valid hypothesis. What ends up happening, is a quantitative result occurs and you have no idea why and even worse can’t leverage those dollars spent into valuable learnings.

Oh I totally feel this! If I had a dollar for every time I see Experiments run without having a hypothesis, or some sort of next steps in mind… Sorry, go on.

Get measurement right. This and your hypothesis make up the infrastructure necessary to run tests that can inform future decision making. I think this is the biggest reason why so many teams have a bias towards digital channels, because they don’t spend the time figuring out how to measure offline channels properly. If you can get measurement right both offline and online, your world of experimentation will expand instantly.

Ever hear that saying that if you have a hammer, everything looks like a nail? I feel digital marketers definitely fall prey to this. Far too often, the value of brand awareness and offline tactics are put aside because they aren’t “measurable”. Huge mistake. What else?

Be prepared to fail. Building a growth team is a really hard thing to do using a bottoms up approach for this exact reason. Companies that don’t naturally understand growth have a hard time grasping the cost of failure and why it’s necessary and unavoidable. Whenever you model out experiments (which by the way you should be doing), make sure the potential for failure and sunk costs are clear being proceeding.

Source: https://alphahq.com/blog/decision-makers-think-experiment-iteratively/

You’re so spot on. I like to frame experimentation as something that converts uncertainty into manageable risk.

Moving on?—?someone in your role has definitely seen a lot of Experiments. In your opinion, what is the one test or experiment you wished that people in growth would stop running?

I hate little hacks like re: in promotional email subject lines or the stories you hear about people putting fake cracks or hairs in ads to make people think it’s on their screen. No wonder we all hate the name Growth Hacking!

Amen. Time to bury it.

Source: https://www.pinterest.ca/pin/551550285614692914/

Continuing with the theme of startups. In the early days, they don’t have much traffic. Are there still any opportunities to leverage experimentation to drive growth?

Totally! While spend and traffic ultimately equate to speed of experimentation, there are tests that don’t require money to execute. Go buy a t-shirt with your logo on it and print some postcards and hand them out in a park. Talk to customers and understand what would make them use your product.

Love it. Kinda goes back to what you were saying about teams sticking to digital.

When you start out, can you run a $10k test on Facebook? No?—?but why would you want to? Tests like that are great when you have a strong foundation of what might work, but until then, test things that only cost you your time.

Finally, it’s time for the Lightning round!

Grow your user base or drive for profitability?

Profitability, and thankfully the whole market is headed this way.

Agreed. I was never a huge fan of growing the user-base alone. It never sat well with me. Now we’re seeing so many companies getting burned by not figuring out profitability.

Frequentist or Bayesian?

Is it bad that I had to Google this?! I think we need to grab coffee so you can explain this to me.

Yeah for sure! I’m no statistician?—?but I’m happy to share what I know. Like profitability, the field of Experimentation is definitely starting to head towards Bayesian.

Finally, why startups and not big corporations?

I’m obsessed with ownership and feel very uncomfortable when I can’t see a direct path from my work to revenue. Plus the speed of implementation and testing is too exciting.

Those early days are definitely addicting. I love that about startups?—?just getting stuff done.

I’m looking forward to seeing what you guys do this year?—?I’m sure it’ll be amazing!

Connect with Experimenters from around the world

We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.