John Ostrowski: The path to a career in CRO is never straight Experiment Nation

A Conversion Conversation with LADDER’s John Ostrowski

What I found most striking during my chat with John, the head of CRO at LADDER, is that his story reminded me so much about my own?—?and of all the other Experimenters I’ve chatted to thus far. A career in Experimentation, especially digital CRO, did not exist not too long ago. Many of us started off wanting to do something completely different, but by chance, found ourselves working in this amazing space.

I, myself, wanted to be a Mechanical Engineer?—?designing life-changing products for the world. I didn’t aim to work on huge products necessarily. I would have been perfectly satisfied making better can-openers, or cool windshield scrapers. All I wanted to do was to solve real problems for real people?—?no matter what the size of the problem. Looking at where I am today, I’m realizing that life has a funny way of acting like a genie in a lamp. Sometimes it gives you exactly what you asked for, just not in the way you initially planned.

With that said, today we learn about John’s journey.

Rommil: Hi John?—?how are you? Thanks for chatting with me today!

John: Hello Rommil! It’s good to be chatting with you, I really like your initiative sharing more career stories within our Optimizer’s community. It’s great to be part of it!

For those of us unfamiliar with LADDER, can you share with us a bit about what they do and your role there?

Ladder is an award-winner agency focused on full-funnel growth. Our teams are split between NYC, Austin, London and Wroclaw (Poland) where I’m currently based on with most of our operations team. We’ve been working with many different companies, from start-ups to Fortune 500. Our mission is to have all departments?—?Strategy, Social, Search, Creatives and Messaging, and Conversion Optimization?—?working as independent agencies empowered and engaged from sales to operations with world-class specialists in their fields.

Before I joined the team as Head of CRO, Conversion Optimization was a supportive resource brought up as a “Black Ops Team” to assist in major campaigns through conversion audits. Over time we worked hard to better “productize” our services and establish clear processes.

What CRO sometimes feels like?—?minus the gun, the camouflage, the killing…

Clients are getting up to speed and button color testing is not cool anymore, they’re looking for a more scientific approach, they’re looking for process. We’re now receiving more dedicated projects and growing the department towards this direction over the past months.

Can you tell me about what the Conversion Rate Optimization (CRO) department does at LADDER?

Before translating small deliverables into proper packages and plans, most of our CRO effort was on delivering one-off conversion research audits which is common practice for junior teams starting with web optimization.

Nowadays, we work closely with our strategy team, they are responsible for a broader understanding of the client’s funnel and traffic channels. I get into the picture to coach the client and suggest a package structure for their conversion goals as planned in the strategy roadmap. In most cases, we still start with a one-off service that serves as research for diving into a testing cycle retainer.

Most of our projects in Q1 2020 are related to landing page optimization?—?including research, wireframing, and testing hypothesis?—?aligned to a performance marketing strategy that is also led by Ladder, we’ve been very successful streamlining Social and Conversion Optimization queues.

“…calculating how many experiments you have running compared to how many you could be running is an interesting way to report on “size of opportunity” up the chain of command.”

I couldn’t help but notice that you studied Mechanical Engineering?—?I studied that too lol. How did you end up in CRO?

Funny coincidence and possibly an interesting correlation to be studied! It is quite impressive the number of engineers that I see migrating from the shop floor to the technical marketing side of the force in the past years.

I took part of my engineering in the United States, and during an internship program at Georgia Tech, Atlanta, I got introduced to the world of data science. I really found joy in working with data and statistics. Back in 2015, machine learning wasn’t much of a buzz word as of today. I was carried away by the novelty effect.

Later in Hungary, where I finished my bachelor studies, I had a great opportunity to work as a marketing data scientist in a Shopify Plus account. They were a big toy trader in the country with operations expanding throughout the European Union with a brand new e-commerce platform. I ended up being in an environment that needed conversion optimization to stand out from the competition.

That’s interesting! I got into CRO when I was a consultant for Bell Canada?—?where we were looking for services to differentiate ourselves from other agencies. Do you remember your first-ever CRO project?

My first CRO related project was redesigning their menu (e-commerce nav bar). We weighed user interaction (event data) to promote more visits in the categories that generated higher sales margins while keeping a good SEO profile (organic keywords)?—?all this in Hungarian, I don’t speak the language at all, but data also talks (:

Hahaha! Sometimes I feel like I don’t speak English well myself

After digging deeper into the community for resources and starting my CXL studies, I found Conversion Optimization to be the perfect balance between applied statistics, data analysis, and behavioral economics. It was a perfect match.

I’d love to hear a bit about your journey in terms of consolidating that department? What are some of the struggles you faced and how you overcame them?

It’s a big work in progress, I prefer coming back to this question in a later post! Can we agree on that?

I’ll definitely take you up on that!

So, I hear that story a lot?—?where people seek a balance of various skills. Because CRO covers so much ground, how do you measure the performance of your team?

Due to our agency reality, I keep track of company indicators as well as account-based indicators. We recently started with cascading OKRs which is now the main objective tracking for the team. It’s a very exciting initiative, and I’m honestly still learning a lot when it comes to the managerial side of conversion optimization.

For individual accounts, I track customer satisfaction (qualitative and subjective), test win-rate and test coverage (quantitative). Since I attend most of the client-facing calls, feedback is immediate and promotes healthier relationships. An extra note on test coverage, calculating how many experiments you have running compared to how many you could be running is an interesting way to report on “size of opportunity” up the chain of command.

“Growth hack tactics” are not the solution, but a solid research process that continuously moves the growth needle in the right direction.

What lessons can you share with those who are also looking to build up a CRO practice?

I’m a very process-oriented person and I truly believe that the main difference between juniors and seniors in this industry is the understanding of how the process communicates with the strategy. “Growth hack tactics” are not the solution, but a solid research process that continuously moves the growth needle in the right direction. Therefore, my top suggestion for someone starting in this field is to understand the process of optimization, a good starting point that I always come back to is Tal Raviv’s “Please, Please don’t A/B test that”.

It is important to mention how important expectations management of stakeholders is, but this is a long discussion for another blog post. We found that sales team training was the most effective in that regard.

Definitely. I feel like the term “Growth Hack” needs to be retired. What else?

It is important to mention how important expectations management of stakeholders is, but this is easily another blog post. We found that sales team training was the most effective in that regard.

100%. To many, Experimentation is a new world. We always have to do what we can to help people understand what the possibilities are, and sometimes more importantly, what they aren’t.

With that said, it’s time for the Lightning Round!

Least favourite mechanical engineering class?

Reliability engineering got me fascinated by stats modelling failure rates of non-destructive systems.

Biggest difference between Brazil and the US?

In the US you can walk around with maps open in your iPhone for orientation during commuting, back home in Brazil I wouldn’t recommend it. I still miss Brazilian BBQ though.

Describe yourself in 3 simple words.

Positive, process-driven, nomad.

John, thank you!

Connect with Experimenters from around the world

We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.