The CRO Survival guide with Carlos Trujillo

Also available in audio format:

Transcript

Carlos Trujillo 0:00
Don’t be afraid of calling out bad practices and unrealistic CRO case studies. We all know there’s so much noise out there. And these might get people to have unrealistic expectations about what you can really achieve with CRO and how long it will take you to do so. So let them know this noise might not have into account basic statistical principles. And many of these things might be highly unlikely to happen to you and really, really discard all bad and color test. It might work for being and booking but it’s unlikely to work for you.

Carlos Trujillo 0:47
Hi, Experiment Nation. This is Carlos Ruhija. Speaking from Colombia, and today I’m going to share with you ideas on how to conquer your first 90 days as a new CRO a specialist. And I’ll share these based on my own experience working as an experimentation strategies at Spiro. But why you should listen to me. I started as a CRO full time practitioner working with leading names in the industry without prior experience working with enterprise clients. I evolved from doing program management into leading experimentation strategy in a short period of time, and I have been trained directly by the some of the smartest minds in the industry. However, I have also built dozens of sites myself for SMBs all over Latin America and the US at my previous experience as an agency founder and leader. After this talk, you will know a set of frameworks and 10 ideas, you will be able to start using today to know how to share your expertise and increase trust in your work plan even before you get to launch your first experiments, ideas on how to nurture a relationship with your stakeholders and clients and show them you’re the one who’s going to help them to succeed ideas on how to pitch and get approval on your first experiment ideas.

Carlos Trujillo 2:07
And shift the mindset to avoid doing spaghetti testing and focus on one kid really have an impact on the business ideas on how to set realistic but exciting expectations about the CRO program, and moving stakeholders mindsets from just testing to win to testing to learn and then winning as a consequence, and then ideas into how to shift mindsets to focus on the long term impact for the business to get more people to care about experimentation as a tool to achieve their own goals. But you might be wondering why 90 days multiple sources claim that the first 90 days that are rolled are key to determine your future success. This is the time you’re used to get to know how things are done in the new organization you’re joining. And this is your opportunity to try to shine to build trust, or to throw it all away if you don’t have a right plan. These are some blockers you might face in your first 90 days at the roll. These are things I have faced over and over again while starting working with a client.

Carlos Trujillo 3:10
And I’m pretty sure you might face some of these too. Number one misconception about what experimentation and CRO release from leadership interesting experimentation to false having unrealistic expectation expecting miracles from it in no time. The required tools not being available so that you are not able to start experimenting the first day, you might like research tools, an accurate analytics, implementation etc. Lack of user research so that you might not have an idea of what to start testing on. And there could be a big appetite to test only opinion based ideas, low traffic for testing nothingness from bandwidth, low statistical power, which will translate into inability to launch multiple experiments at once, or inability to get conclusive results at all. And the lack of framework to prioritize ideas and activities. This can be really common in companies that are new to CRO which we’re translating, having barriers to running research activities, or just appetite to testing the highest paid person’s

Carlos Trujillo 4:19
opinions, and so on. So now I’m going to share

Carlos Trujillo 4:23
with you 10 ideas you will be able to execute on month one, month two and month three to overcome some of these blockers. But first I want to show you where all these comes from. This is the spirit of experimentation or is it is a set of frameworks we use as the foundation to kickstart every experimentation program we run for our clients. It is a set of activities that gets divided into rituals and artifacts for assessment and integration planning and process tests on Learn decision in action It is not intended to be following that strict order. However, many of these ideas could be respectively applicable to month one, month two, month three. But again, you should go check it out yourself and adapted to your own use case. Here are the 10 ideas at the end. Number one, get familiar with the business goals, because people don’t really care about experimentation. Get used to translated the CRO, experimentation, geeky language into plain English. Get used to speaking the business language get to know where the company OKRs was a big, hairy, audacious goal for this year. Let them know you are not only going to optimize their website but to use experimentation as a way to move faster in testing the assumptions of the business strategy. Number two, be a reliable source of valuable information. Keep up to date with industry standards, never stop learning. Show them you know your craft. Teach them what experimentation is really about. Number three, identify how mature the company is and adapt accordingly. Not all companies are the same, your new experience might be very different from the previous one. And here, there’s a tool I want to recommend you to use. Go and check spearos maturity audit assessment in less than 30 minutes, you will get to know where your company is that in the maturity spectrum, and you’ll have the right tools to adapt accordingly. Number four, don’t be afraid of calling out bad practices and unrealistic CRO case studies. We all know there’s so much noise out there. And these might get people to have realistic expectations about what you can really achieve with CRO and how long it will take you to do so. So let them know this noise might not have into account basic statistical principles. And many of these things might be highly unlikely to happen to you and really, really discard all bad and color test. It might work for being in bookkeeping, but it’s unlikely to work for you or spend time and energy into what can really have an impact. Number five may others achieve their own success. CRO is a team effort for us to succeed we need other people to succeed and most likely get the credits, that’s fine. But you won’t be able to achieve anything just by yourself. Be a team player, you will get leadership buy in and you will get co workers to support you etc.

Carlos Trujillo 7:30
Number six, get people exposed to research find these as quickly as you can. In the case of spear wiki start every experimentation program with a big project called research Excel when we run a wider range of research activities in order to gather enough insights. To start experimenting. However, it might take between four and eight weeks from the time we start running these activities until the time we present these insights. If you’re in a case like this, make sure you show some progress. In the meantime, dig into what has been gathered before. Run low resource activities. Do whatever you need to do but show some progress about the data you are gathering. Number seven start by running non inferiority experiments. Think what you can take a way to add value and start testing that this will let you gain time while you cook more complex experiments. And these will get the experimentation machine up and running faster. Number eight, get the key people familiar with the CRO processes and technicalities. This will be key in order to set the right time expectations help the people knowing what’s the life of an experiment how long it will take to design death to a monitor and analyze and experiment. Get them familiar with the statistical principles so that they are not picking out experiments and wanting to stop them the day after you launch them. Number nine, work on improving your systems every day, make sure you are building a systematic, repeatable process. This is the key to achieve bigger business goals. And this is the mechanism that will allow you to show them what’s going on. at Spiro, we make sure we start with a program activities visual roadmap and an experimentation visual roadmap as soon as we can so that people can see the progress we are making, even before we get to deliver anything. And last but not least, number 10. Tie experiments to a specific business goals. This is the only way to get more people interested. Show them where are you learning from your customers that can be applied to other channels and business units. This is the best way to get noticed and to get more people to support the work you’re doing. So we just went over a lot of ideas today. But in summary, get familiar with the business goals and don’t be afraid to calling out bad CRO practices in order to Show your expertise and increase trust in your work plan.

Carlos Trujillo 10:03
Make others success your priority and be a reliable source of information in order to nurture a relationship with your stakeholders and clients. Start fast by running non inferiority experiments and get people exposed to research findings as quickly as you can in order to pitch and get approvals. On your first ideas. Identify where in the spectrum of maturity your company is in, adapt accordingly, and get the key people familiar with the CRO processes and technicalities in order to set realistic but exciting expectations about the CRO program. And work on improving your systems every day in tie experiments to a specific business goals in order to shift mindsets to focus on the long term impact for the business. That was it for today. And I know there’s a lot to process. So send me a DM in LinkedIn so that I can share this slide deck and the frameworks I share today with you. Thank you so much until the next time

If you liked this post, signup for Experiment Nation’s newsletter to receive more great interviews like this, memes, editorials, and conference sessions in your inbox: https://bit.ly/3HOKCTK