CRO isn’t just for marketers with Kelly Wortham

AI-Generated Summary

Kelly Wortham dropped some serious knowledge bombs in our recent Experiment Nation chat. Here’s what stood out: – CRO isn’t just for marketers: Lawyers, project managers, even developers can thrive in this field. It’s all about a love of learning and adaptability. – Don’t underestimate the power of mentorship: Kelly has THREE mentors, each focusing on different aspects of her career. Find your own advisors to help you grow! – Certifications are nice, but experience trumps all: While CXL is great, hands-on testing platform experience is the real deal. – UX psychology is CRUCIAL: Understand your users’ motivations and behaviors to create truly effective tests.

https://youtu.be/GQYG8Yb_ikQ

Audio

AI-Generated Transcript

(00:00) um and and those stories are fascinating because what happens is you know the the the the lawyer working on the the legal team with the experimentation team became fascinated with experimentation and was like I want to learn more about that and then ends up know taking some courses and then becoming a cro

(00:22) specialist right [Music] welcome to experiment Nations podcast I am your host Charlotte bonford and we’re going to have a chat with Kelly Wortham today I’m sorry I you know I’m hoping I said your last name correctly um Kelly you have an expens extensive experience on website testing optimization you know Kelly she loves to

(00:54) test and learn that’s why she named the community from the word Tes learn that’s very interesting Kelly would you like to tell us more about your experience um sure um so I’ve been uh in the uh experimentation industry for about 25 years and um and um I got into the industry from Academia and um part of the reason love to to

(01:23) teach and learn um and the test and learn Community is part of that it’s an opportunity for folks in the industry to teach and learn and Converse and um that’s what it’s all about um and so we wanted a a way to continue the conversation in between the events uh where we all get together and learn from each other but those don’t

(01:49) happen as often um and we wanted to continue that conversation in between so um that’s how it was born almost 12 years ago now um and so now um we have that that Community um and um that slack home that and that uh continuous conversation that we also have um once a month um online and continuously some

(02:18) online events and iners events as well that’s that’s actually very cool like when cro wasn’t even a thing you were already there making history and so I’m just curious how did you get into cro like um 25 years ago it’s interesting like how was AB testing 25 years ago well um people have been doing experimentation long before they were

(02:47) doing digital experimentation um and 25 years ago uh we were still doing uh offline experimentation 25 years ago when I got into the realm uh in Academia we were doing um experimentation and then I started doing email testing um uh there was a a company that I worked with that doesn’t exist anymore called

(03:11) Circuit City um and we were doing email testing back then oh W um because that that world existed um and um we had to do everything in Excel spreadsheets which is a lot of fun so subject line testing and stuff like that um and we were also doing offline testing to understand the impact of our display

(03:31) advertising um to inore um um footsteps so measuring those types of things back then that’s gotcha so in terms of like um was there any were there any courses back then to go into because like you know right now I’m thinking about advertising as a degree you know or uh like arts and something similar Bachelor

(03:59) of Arts something but back then are there any types of courses where you can learn how to do ab testing um well you could learn quantitative and qualitative stats and research um uh statistics in any degree that’s actually what I taught when I was in uh when I was teaching University which is how when I transitioned to what

(04:30) I call the real world um I was kind of eager to do that type of work um and you know actually my first job in the real world was in inventory forecasting which was incredibly not exciting but that’s okay um so I was real excited when I switched over to Circuit City and actually got to do some of the actual

(04:54) email testing but um that those courses exist and actually Dr Ellie fight um teaches folks those exact courses today at Drex University so if you all want to learn if if any of you are are listening and you want the opportunity to learn what we do today in this field you can go take Dr Ellie fight’s courses at Drexler University

(05:21) that’s good that’s that’s a really good one so you you know like if I’m like a you add your marketing person I’m like I want to do cro who’s the best person to talk about it or to listen to that’s it would it is it like an online course or in person or anything that’s you know self-paced and you know if you know if

(05:42) you’re talking to someone who is new to zero and want to get into this field right which type of learning style would you recommend this is Romo Santiago from experiment Nation every every week we share interviews with and Conference sessions by our favorite conversion rate optimizers from around the world so if

(06:03) you like this video smash that like button and consider subscribing it helped us a bunch now back to the episode um I I personally like the in-person courses but there are a lot of amazing online courses so uh Ruben dbor has a lot of online courses udic courses um that are amazing you can go to conversion ideas.com and see a lot of

(06:25) courses that Ruben offer off um con um online dialogue has a whole bunch of courses that you can take those are in person they’re amazing um sneak peek uh Ruben’s coming to the us to bring some of his courses as well um we’ll be co- teing them if anybody’s interested let me know um hit me up and um hi Reuben

(06:47) I’m a c yes me too I’m a super fan which is why I asked him to do this with me so if you all are interested let us know we’re trying to pick the first city um and we’ll be doing this cohort-based so we’re going to do it at least twice um so if y’all are interested let us know we’re going to pick the city and figure

(07:08) out where we’re going to do it um so you can go to online dialogue’s website see what’s going to be in that content um and we’re bringing that to two US cities probably in the next 12 months so oh that content is amazing and I’m not just pitching it because I’m a part of it it’s amazing content which is why um

(07:28) we’re bringing it to the US because it’s it’s a difficult time zone um to do it online and it’s much better content in person um of course everybody knows about cxls training it’s also excellent um but one of the things that I like about cohort-based training versus online training is you actually get to

(07:50) talk to your instructors and have access to those instructors and by the way cxl is trying to figure out how to do that so the the then um Ken Williams Doug Hall um Jill quick and myself are also starting some cohort-based training and the difference between um the training that you do that’s like online and the

(08:17) cohort-based training is the cohort Based training you get the content like like in person or online together with the the the teacher and then you have like the people that were in the class with you and you have a the uh access to the teacher over a period of time in some sort of slack group or teams group or something for some period

(08:47) of time for continual Q&A and the people that were in the class also have access to each other for some period of time to activate whatever you whatever the course material was and that is really really powerful because until you’ve tried to put what you learned into action you don’t like we all know until you try to

(09:12) actually do it you don’t really activate what you’ve done um and so that’s really really important to me as somebody who you know got started as an instructor as a professor like I know you have haven’t really learned it until you try to do it I agree so I that’s really important to me it’s like a mega ideation classroom

(09:35) and you can present like different things and then people bouncing off each other that’s actually quite really good when did you start becoming an instructor like I’m just curious was it like I I went like I went out of high school into college into grad school into being a professor um so I’ve always

(09:57) been a teacher um first and foremost because one of the things that I I learned right away is I don’t know something until I’ve really tried to teach a concept um and it’s kind of like when you’re trying to learn something you have to try to teach it first and if you can’t really explain it you don’t

(10:18) really understand it I agree yeah uh and that’s when you find out that you don’t understand it when you try to explain it to somebody else in fact if you try if you know that you need to understand something try to write a blog post about it and that’s when you figure out oh I don’t understand that concept and then

(10:39) you go and you interview all these people that do understand the concept until you understand it then you go write the blog post and by the time you you’ve talked to all the people who do understand it you will understand that concept I agree yeah back and that’s the best way to learn that’s actually by the

(10:58) way another reason I created the TLC surround yourself the smartest people in the room you get you get people to ask really great questions I agree that’s why I always say there’s no dumb questions there aren’t like it’s amazing you have all these brilliant people in the room to learn from and you just

(11:17) continue learning because everybody has different ideas different perspectives and I’ve learned so much from everyone it’s ah it’s my favorite place in the yeah it’s kind of like sometimes you you’d be surprised that the some people think out of the box when it’s a fresh eye kind of thing so it’s like oh I haven’t really thought

(11:42) about that before unless someone you know has presented the idea and uh that’s very interesting now my question is like okay because you’re an instructor yourself or a teacher yourself um um yeah so it’s kind of like we’re talking to a group of people who wants to learn cro you know if you’re someone who is in

(12:06) that position how would you evaluate like you saw a course right how can like how would you evaluate the The credibility the the quality of that specific course or instructor they usually have those you know like things that learn but how would I know that that’s something that is worth the money to pay

(12:34) for um I would look at the reviews um and the the testimonials um absolutely um there there’s going to be a wide variety of costs um associated with them I would absolutely consider um the you know your your a aable time the quality of the individuals that are you know giving the um training um that’s

(13:04) something to consider um the convenience you know is it in your city do you have to travel to it um I I would look at all of that stuff but also is it a match like is this something that’s where you need to be is it way above your head way below um I would look at all of that stuff like you know there are um there is

(13:33) content that is exciting because the person that is giving the content is um a name that you really really recognize and you get really excited about it but the truth is the um um you don’t need all of that because you already have 40% of it you already know and um you know the other part of it is like too technical or uh whatever and

(14:07) then there’s another situation where you have to travel to it and you’re never never going to get budget for it so like take everything into consideration and find the right fits that’s right yeah but also there’s other content that’s free oh wow that’s perfect there’s a lot of free content on YouTube really solid I mean very blunt

(14:32) Ruben has made a ton of amazing content available amazing either free or really reduced content available I’m going through his a table course right now which is awesome if you all have not taken Ruben’s air table content on Udi you need to go look at his air table content so good UD plan at the moment and you can just

(14:59) access Ruben’s course anytime like I found his j4 course when you know j4 was just starting like very very helpful and it kind of like shaped how I started reporting on things and yeah that’s why I’m like Ruben is he’s so easy to listen to very understand like it’s like he’s so good at explaining things and hiben

(15:23) again I’m a fan and yes yes so um any specific topics or any modules that you know should a person look for in a cro curriculum like where do you think a person should start well I’m I’m a big fan of ux um psychology um like yeah a lot of people don’t don’t think about that but the the psychology of um like uh user the user

(15:53) Journey so that that piece of it um of course related to analytics so you know GA Adobe whatever um definitely um the the statistics of um you know uh how you an uh uh calculate sample sizes those things are important um and often missed um and um it’s it’s sort of the foundational stuff and a lot of people

(16:33) will um dive in with just like usability and they’ll miss the core statistics so that’s that’s a must um and um um I mean if you get those three um you’re probably ahead of pretty much so everybody else um the technical stuff is important but those three are probably the most important I agree as far as I’m

(17:03) concerned I think when I was starting zero as well as like uh the sample size was a a big you know blunder on my end a few times like I would look at the pages and it doesn’t have enough visits and it you know the statistics doesn’t matter because it’s it’s the sample size too low um and so I agree with you like it’s

(17:26) just one of those blunders when I was starting with zero that I remembered when you were talking about sample size and it’s very true um now this is a big question probably because a lot of people may not have experience with marketing but wants to go to cro like if you’re a developer can you be a cro

(17:46) Specialist or you know are there any um recommended background knowledge or prerequisite or any background in terms of the type type of career that you have for taking cro courses or anyone can do it anyone can do it that’s what’s so beautiful um uh I was on the uh jury for the women in experimentation conference

(18:10) um that is about to come June 6 um register and go attend this beautiful online conference um some of the great stories yes yes uh the the convert in an experimentation conference June 6 you can go to LinkedIn to see the to register to attend it’s free for everybody um please attend the uh amazing conference um but one of the

(18:38) things that some of the stories um that were submitted were people from outside the industry who became um experimenters and the stories that they told uh which were amazing Stories like people who came from project management people who came from law wow yeah right you know why not um and but that was the point is like their

(19:05) perspectives were really interesting stories um and and those stories are fascinating because what happens is you know the the the the lawyer working on the the legal team with the experimentation team became fascinated with experimentation and was like I want to learn more about that and then ends up taking some

(19:31) courses and then becoming a cro specialist right wow and and D the project manager working with that team be is like I want to take some courses and becomes you know and so and those stories are pretty common um same thing happens where the experimentation specialist working with the product team decides I want to move to become the

(19:53) product owner and and so there’s a lot of movement that can happen um and those stories are interesting and there’s no reason that you can’t move as in your career as you decide where you want to be um you just need to be have your your mind open and and pay attention be flexible because most people think I want an upward trajectory

(20:23) but you should be flexible that growth isn’t always up sometimes it’s a cross um and you don’t have to be t-shaped you can be pie-shaped or comb shap to learn uh what interests you and fascinates you I agree experimentation has always been fascinating when you’re talking about psychology as well because

(20:45) it’s like what are what’s what’s the person thinking when he was looking at these Pages what is he like looking for and it’s kind of like you’re this CRI Crim I wouldn’t say criminal now like like psychoanalysis you know just wanting to understand the behavior of a human um is quite interesting so I get

(21:04) where the lawyer is coming from um okay well um you were talking about uh people coming in and having like this experimentation community and learning from each other right um yeah are there any other interactive elements or opportunities for practicing Hands-On practice included in cro courses that

(21:31) you can um think about or recommend you mean like besides conferences yes besides conferences like something similar to what you’ve mentioned um where all the people so it’s much better to have that um group together right that you were mentioning earlier like in person um where you can bounce ideas off from each other ask

(21:58) questions like cohort kind of thing but other aside from that do you can you recommend any other places where or any other courses that recommends the same thing um I mean um there’s a lot of events um lots of conferences lots of um um like measure camps and um like experimentation Elite is coming and then

(22:33) the TLC mentorship Summit is there um everybody get your seats the first 90 seats are um they’re free to anybody coming to that has a ticket to experimentation Elite it’s a big deal um but you have to come to experimentation Elite in Birmingham uh England so please come um but um are you flying to England of that

(22:56) or yeah yeah of course um hosting the the uh the event it’s a big deal um and our our mentors are um you know juring the mentorship summit so it’s it’s a very big deal um but um I just those those do what the mentoring thing is probably one of those that you can also get right just to have like someone to help

(23:29) you out when you’re exactly that’s that’s why I mention it like mentorship in general is like core which is why the TLC decided to be a part of that we’re trying to encourage more like that’s the other piece of the TLC and you know experiment Nation you know the crap Community women experimentation is it’s

(23:52) all about you know teaching and learning but it’s also about mentorship that’s why we you know we bring in as many you know experimentation industry leaders and folks new in the industry because it’s about teaching and learning AKA mentorship so that was like the impetus of the mentorship Summit is to bring in

(24:11) Industry leaders as mentors and folks a little bit newer in their career but the folks newer in their career are actually on the stage that’s amazing and the mentors jury um and so we kind of flipped the script now but um mentorship is a really fantastic way to learn um whether in person or virtual and like one of the biggest lessons that

(24:41) I learned in my career um I actually learned it from Gary Angel um is never ever ever be afraid to ask a question uh and ask advice of your mentor and one of the questions to ask is will you be my mentor um which you’ll be stunned how rare it is for somebody to say no because um and and often they may say I

(25:08) don’t have time but let me suggest three other people that you should ask um but I’ve actually never had a mentor say no I currently have three mentors wow uh they’re like an advisor panel um and they’re amazing um and um I I value them deeply um and my time with them is like absolutely precious and so I I highly highly

(25:38) recommend everyone to have you know mentors plural and have them as you know to play different roles y uh have have somebody that focuses on different aspects of you’re looking for in your career um and challenge you don’t don’t just get somebody who makes you feel good about yourself somebody who for what you’re trying to grow in

(26:10) your career really really important for your personal growth do you now question do you offer mentorship if in case like someone sees this and says like oh I want Kelly to be my mentor would you be open to it yes and it’s funny that you mentioned that because the people that know me are all laughing because I have a lot of

(26:33) mentors yes okay amazing so you you said that you have three mentors right can you tell me like what their roles are like what’s the first mentor and what is he mentoring me for y yes so I have a mentor that whose focus is helping me on operational stuff I have a mentor that’s helping me on um growing my business so

(26:59) that’s like the marketing side and I have a mentor that is helping me stay grounded in what what my um I guess the easiest way to explain it is my my beliefs so if I want to grow my business um sometimes it’s easy to be to get greedy but I have a core belief system so those two things can butt up against

(27:25) each other so the person who is keeping me grounded in my core beliefs has to challenge the person who’s helping me grow the business so the three of us meet at the same time to ensure that it’s kind of like checks and balances yes yes wow that would have been a very interesting meeting I would want to like see how

(27:51) that goes but yeah it was very interesting it’s great great they they love each other and we’re all um we’re it’s fantastic that’s amazing that’s amazing that’s what you need um so um are there do you need a certification when you want to go into the cro industry and if you do feel like you know there are some

(28:21) employers like you have to be cxl certified because I’ve seen those um what are the yeah what most there’s there I mean cxl certification is a thing and it’s it’s certainly a a a lovely thing to have um but no employers um a lot of employers wouldn’t know what it is but those who do are very impressed by it um

(28:48) by no means um I if I saw cxl certification I would know what it means and I would be like that person did a lot of work um absolutely um but the employers who don’t know what it means wouldn’t know what it means um it is a lot of um honor for the people who do yep um having said that it’s um it is not most

(29:18) certifications are not necessary um so you do not need a certification um and you know any any classes that you can have any work that you can do um what their best thing that you can have is something like you know Google analytics Adobe analytics but the best thing you can have is the name of testing platforms

(29:43) experience with testing platforms true um and multiple testing platforms shows that you’ve had experience with multiple testing platforms which means that you have more than one test um experience and um that tells people that you worked probably with more than one program um and um that shows adaptability and experience across more

(30:14) than one program which shows people that um if you’ve had experience across more than one test program then you probably have uh more experience than somebody who’s only ever worked in one program that’s amazing that’s a that’s really good um I would say advice to people who are thinking about going into

(30:40) experimentation it’s more about the platforms there’s a lot of platforms out there um now paid because you know well there’s a lot of paid platforms and there is differences in it there there’s there yeah but you can get certified in a couple of platforms but honestly certification doesn’t mean anything for platforms really either to

(31:05) be fair true true so long to work to to do the work that’s the most important thing yep now here’s uh another question if in case a person wants to go into cro how long does it typically take to complete a c course so if do you have a um current how about you Kelly because I know you’re instructor do you have a

(31:31) specific course that you’re working on right now or you know like that’s available for viewers to see and if so how long does it take for a a person to complete that cro C course and what’s you know level of commitment required um there’s there’s a ton of different workshops and courses available like I said you can go to

(31:57) online dialogue’s website to read about the course that Ruben and I are doing um it’s um it’s essentially um two half days with um um Inc course activities um the way it’s performed in the Netherlands it’s a half day with a week in between and a half day but the one we’re going to bring to the us we’re

(32:19) going to bring the activities inside the course wow um and we’re trying to decide if we’re going to do that in the same day so that it’s you know a little bit easier um or um not not the content easier but so that you don’t have homework so you can do it with us while you’re there um and you don’t have to

(32:43) travel home in between um or if we’re going to do it two full days uh to make it so that you can do the in-class activities where that’s TBD um but um um um so it’s it’s two full days um or two half days TBD that piece um but it is a ton of content and you can like I said you can see that on online dialogue’s website um the workshops I

(33:11) teach right now are um like full day workshops halfday workshops depending on the content um and they’re currently targeted for companies that want to bring me out I can do it for um cohorts as well I just need at least 10 people so because I have to travel for them um and I don’t do them online because

(33:39) they’re inperson focused um but you can see those on my website at for digital. org um then there are um courses that you can see and attend at um uh like I said on uh Ruben’s website that are conversion ideas.com that are amazing um and I recommend all of those they’re fantastic and of course cxls are great that’s

(34:09) amazing um so this uh thing that you’re doing with Ruben do you have like a date set already or is it still to be discussed we so you all can be the first to let us know you can reach out um and let us know cuz we are trying we are collecting um interest um to pick the first city so we need the first city um

(34:33) and we need at least 20 people to sign up so I need the first 20 people to tell us what city before we decide the date and um where we’re going so it’s kind of a race to the first 20 the first 20 people to sign up and uh then we will pick the city and the date oh my my my my that’s a good opportunity you wouldn’t have Kelly and

(35:04) Ruben in you know one room ever again I’m just kidding probably there will be more in the future lots of opportunities but lots of opportunities but this is good um well would there be any like now that we’ve talked about if I completed C course what would be a next step for me as a new newly cro certified or someone

(35:28) who’s completed a course what would you your um advice be for that um person um once you’ve completed training um you can of course put out your consultant hat which is amazing you can on LinkedIn you can say open to work and you can start listing the things that you’re available for Contracting work um

(35:53) dive. team is a amazing consulting firm that will hire you uh it’s Ken Williams um and he will make you um uh put you on the books anytime he has like contract work um so go check him out um he’s he is looking for people with at least 10 years of experience so you need 10 years of experience um but um so I’m assuming

(36:20) that like you have other work in your career before you took on experimentation work but um there then there are other plenty of agencies that are looking for anybody with new experience so consider them um lots of them but if your company is looking to add your work then you just let them know okay I’ve got this experience I’d

(36:46) like to start doing that if you don’t want if you just want to do it on the side then you can start Contracting that way by letting LinkedIn know here’s my skills and my availability open to work um and and that’s that’s how you start your little business it’s amazing um and then if you don’t want to do that go to

(37:08) thank you Kevin and Tom experimentation jobs.com it is the best resource and it lists all the available jobs that have the skills that you just earned that’s amazing who was it again who who created a um C jobs.com or what’s it what’s a experimentation jobs.com experimentation jobs.com so if in case you’re a newly

(37:35) certified C specialist and you’re looking for a role that’s a place to go so yeah that kind of like concludes what we’re talking about today Kellan you know like there’s a lot of things that um you know we’ve learned today with uh your time so we appre appreciate your time with us at the experiment Nation

(37:58) today yep it was awesome thanks awesome so yeah we’re going to say goodbye to our viewers now and hope that you learned uh more things again like the um Kelly do you want to go and say the things uh the websites where they can go to to attend uh yours and Ruben’s uh conference or the first 20 like a lot of

(38:23) people may be interested to sign up and just don’t know sure so if you want to reach out to me to find out about um my and Ruben’s uh online dialogue session uh just uh reach out to me at Kelly forward digital. org that’s Kelly forward digital. org if you want to read about the um session and see if you’re

(38:53) interested you can go to online dial og. NL that’s online dialog. NL and read about the um conversion uh um online the conversion training it’s pretty obvious what it is um and then if you want to learn about the test and learn Community to join it just go to testand learn. community amazing thanks so much Kelly

(39:18) well you guys have an awesome day thanks Kelly this is Romo Santiago from experiment Nation every week we share interviews with and Conference sessions by our favorite conversion rate optimizers from around the world so if you like this video smash that like button and consider subscribing it helps us a bunch

(39:38) now back to the episode

If you liked this post, sign up for Experiment Nation’s newsletter to receive more great interviews like this, memes, editorials, and conference sessions in your inbox: https://bit.ly/3HOKCTK

---