Rommil from Experiment Nation: Hi Darian, happy to chat with you today! How about we start with a bit about yourself and what you do?

Darian: Hi, my name is Darian Rusow! I focus on conversion rate optimization at Curology, a personalized skin care company, for their two brands - Curology and Agency. I recently moved from Southern California up to the Seattle area and am loving the cold and rainy days.

You don't often hear folks loving the cold and rainy, but I do enjoy a nice thunderstorm. So, Darian, how did you get into Experimentation?

I got into experimentation on accident. When I was interviewing for a marketing internship they decided I’d be a better fit on the web marketing analytics team. From there I was introduced to A/B testing and fell in love - I’ve been testing and optimizing ever since!

Very nice. That's a common story - discovering it by accident LOL.

I’ve read that you’ve spent some time doing App Store Optimization (ASO). Could you tell us more about what testing for app stores is like considering app stores don’t share a whole lot of information?

App Store Optimization was so uncomfortable for me when I first started because I went from testing on my own channels where I had mountains of data to testing on external channels with almost no data. 

You lose the ability to gain little insights when testing on the App Stores but that’s where user testing or 3rd party tools can come in handy. Of course, both of those methods have their own cons but it helps paint a picture around why some of your creative assets maybe performed the way they did.

What tools do you use to perform ASO?

ASO encompasses keyword optimization AND conversion optimization. 

For conversion optimization I used Google Play’s native testing tool and StoreMaven to run A/B tests on our creative. StoreMaven was great about getting us more insights into user behavior (scroll behavior, time on page, video watch time, screenshot engagement), but in order to run these types of tests you need to send paid traffic to the platform. This opens up some biases in your data that you’ll need to be mindful of when analyzing the results

For keyword optimization I really loved the team at AppTweak - I found their platform was the most helpful in identifying new words to target and their consulting team was great to work with too.

What kind of results could one expect from ASO and how long does it take to see results?

You can see results from ASO pretty fast - it’ll be different depending on the platform but for keyword changes Apple will usually reflect any new results within a week and Google Play can take up to a month. 

The types of results you can expect from ASO are really just as varied as they are with web CRO. One callout that’s really cool to me about ASO is that your results on mobile can have a faster rate of diminishing returns. A lot of the top apps on the app stores are updating their assets monthly to make sure they’re always on top of their game, pun intended.

Changing gears. A common question I hear is when to use an A/B test vs. a Multivariate one? What are your thoughts on this?

In the end I think it boils down to time and goals of your company. Are you more interested in learning or optimizing? Do you have the traffic to sustain a multivariate test and get results in a meaningful time period?

I totally agree. Not only the traffic, but the resourcing to keep creating assets for MVTs.

OK, as someone that’s been knee-deep in SEO - is it possible to run tests for SEO? And if so, how at a high-level?

It’s been a while since I’ve been hands on with SEO but A/B testing SEO changes was always difficult for me. Essentially what we would do is find a group of pages that were similar enough in content and performance, roll out the SEO changes to half of them (our variant group) and measure changes in ranking between the variant group and the control group with a pre/post analysis. It takes a lot of time to reach a conclusion and even when you’ve got an idea of what’s winning it’s still best to roll it out consciously.

As someone who is part of a newly formed CRO team - in order of priority, what skills would advise our readers to hire for?

Tied for number 1 is the ability to communicate clearly to all levels within a company (yes, I’m talking about HiPPOs) and the ability to uncover insights. Even if you have a team filled with the best analysts, developers, or any other technical skill, if they can’t communicate your program isn’t going to go far.

Communication is such an undervalued skill in our field, I find. I always say, if you can't communicate the learnings well, no one will value your work.

Changing gears a bit now. It's time for the Lightning Round!

Describe yourself in 5 words or less.

Nosey in a good way.

Frequentist or Bayesian?

TBD - doing a lot of research into understanding Bayesian at the moment.

If you couldn’t work in Experimentation, what would you be doing today?

Product marketing or working with clay.

Ah yes, the classic marketing vs. clay choice. LOL

What do you have going on that you feel our audience should know about?

Curology launched a new brand this year called Agency that creates a customized anti-aging cream - you should check it out!

Very nice. And finally, who should we interview next and why?

Most of my saved LinkedIn posts are from Jeremy Epperson. He has 13 years of experience in CRO - super helpful for people starting out and those more experienced.

Awesome. Thanks so much for taking the time to chat with me today!