Build a Learning Library featuring Nick Murphy

AI-Generated Summary

Discover the power of experimenting to learn and win! ? Nick Murphy shares his expertise on building learning libraries. Here are 5 key takeaways: – Learning libraries capture the “why” behind successful and unsuccessful tests. – Organize your learning library with consistent and digestible fields. – Use “broad” and “specific” learning columns for a layered understanding. – Encourage cross-team collaboration by sharing insights company-wide. – Learning libraries foster a culture of experimentation and growth.

Audio

AI-Generated Transcript

(00:00) but you don’t want to create this thing that is overwhelming you don’t want to create something with every little aspect of um the test you don’t want to create too many columns where people are like feeling like it’s overwhelming to look at look at it hi my name is Nick fips and I’m head of cro at idhl a digital agency based in

(00:18) the UK I’m your host for this episode of The Experiment Nation Podcast on today’s episode we have Nick Murphy who shares his thoughts tips and advice on how and why you should build a learning library so sit back and enjoy the show hey everybody and welcome to another episode of experiment Nation I’m

(00:40) your host today Nick fips head of cro at idhl based in the UK and today we are going to be talking about the importance of learning and building learning libraries with our special guest another Nick Nick Murphy Senior research and experimentation lead at further hey Nick hey Nick yeah it’s fun to talk to

(01:03) another Nick um do you ever go by Nicholas are you a Nicholas only to my nan yeah sometimes I’m like approaching a new group group of people I’m like am I gonna go by Nicholas now it sounds so much smarter like formal you know I’m educated I’m Nicholas yeah um my gym teacher used to call me Nick at night or

(01:28) are you familiar with that show like Nickelodeon at night when we were kids they oh yeah yeah yeah like Are You Afraid of the Dark was one of the big shows um but yeah so I am currently at further I’ve been working with about 30 different clients in the past two and a half years I have a digital analytics

(01:48) background um I’ve set up analytics before done a lot of analytics analyses but I found out about experimentation as analyst kind kind of that becomes part of our job as the role has kind of changed in the past um and experimentation is just so much more fun than analytics so I’ve really really liked doing it and um yeah working with

(02:09) so many different clients I think I’ve seen a lot of different ways that they have kind of approach like the value of what experimentation is and kind of the value that people get out is usually like Revenue gained Revenue saved those are calculable right easy to quantify well somewhat easy and then there’s

(02:30) learnings and I feel like learnings are kind of just this nebulous thing and people don’t talk about it as much Simon jardan did a previous episode here and really talked about learnings which I thought was nice but I think I just want to talk today like more about learnings and kind of how we can uh help the

(02:49) industry approach learnings in maybe a little bit of a different way love it sounds good yeah I think you you said before we got started that you know learning is an artart as opposed to a science so I’m excited to unpack that a little more and I think let’s um let me just give you the opportunity then to

(03:08) kind of take a step back and just explain to the guys that are listening you know what what is it that you actually mean by learning um what do you see as the kind of the the problem that exists out there and what are you trying with this talk to kind of educate people about yeah so learning there’s there’s

(03:26) kind of few problems I see in the industry right now and um one of the main ones is that you know learning is intangible so it’s definitely more of an art than a science so you can kind of you set up your hypothesis statement in the start and you end up with your analysis but what happens in between is

(03:44) kind of like linking the treatment and the output of the test and you kind of just use your prior knowledge to understand what actually changed um so that’s kind of one of the things going on here another problem I see is that um the industry is really focused on quick W quick wins right now a lot of people

(04:03) don’t post about this on LinkedIn but when you’re in the conversations that happen you know organically this is what people are talking about they’re like we want the loow hanging fruit the quick wins we want a 10 yeah um and you don’t really hear like hey I really want to learn from experimentation I want to understand my

(04:23) audience why they buy why they don’t buy it’s like we want to we want to make number go up at any expense like that’s what we’re focused on it’s kind of um it it’s really something that that we push from our side as C isn’t it you know we it’s one of the intangible benefits of of what we do and that’s what we’re

(04:44) telling our clients you know like oh you know just because that test lost here’s some learnings and it’s it’s coming from our side as opposed to being really driven by or understood by clients or or kind of In-House teams yeah yeah so we really owe it to our clients and if you’re inous to the companies you’re

(05:02) working with to be honest and really do the work upfront to create the learnings you want I think that cro we do have this position where if we launch this test that is less than good and we get results that are either negative or inconclusive we can just say you know it’s okay because we test to learn like

(05:22) I am all about that like learning but when you’re using learning as like a coping mechanism when I started to have a problem um youve really got to set yourself up for the the best chance at learning um and there’s a few ways we can do that too cool so we mentioned it in in the intro but um I guess today we’re talking

(05:43) about this learning library right so a repository of of learn learn learnings and insights um what exactly do you mean by the term learning library and kind of uh just un unpack that that kind of naming convention a little this is Romo Santiago from experiment Nation every week we share interviews with and

(06:06) Conference sessions by our favorite conversion rate optimizers from around the world so if you like this video smash that like button and consider subscribing it helps us a bunch yeah yeah learning library is an is one of my favorite things going on right now that I’m working on and it’s it’s a tough one

(06:21) um I think a lot of people have seen Ruben dbor has a big class about how to develop a learning library and air table um I think my value that I hope we can talk about and we will talk about today is more of like how you actually use that and what you can get out of it how you customize it Ruben does an excellent

(06:39) job like setting up like how you use it um but yeah what I mean by that so most people are familiar with the concept of a hypothesis Library so it’s this document where you’re working with your stakeholders you are putting everything in that if then because format or something similar and you want to have as many hypotheses

(07:00) as you can and you want to like link where they are on the website um the kpi you’re attacking um that sort of thing and you want to prioritize everyone has their prioritization framework their uh different rice pie all those fun things but after that happens I think and I think some other people think as well is

(07:21) that when a test is completed or prioritized in the hypothesis Library it should end up in the learning library and my take on the learning library is it really should be something that can be shared companywide marketing should understand it sales should understand it um of course us we and CR should

(07:41) understand it um any aspect of the business should be able to open up the learning library understand the progress you’ve made what you’ve learned about with your dis different customer segments different techniques that they should try techniques that they should probably avoid um so really the learning library is

(08:00) this it’s providing you light kind of like when we start out in testing or companies that don’t test first of all they’re operating using gut feelings what they think is going to work they’re in like kind of like a dark room right they can’t see the customer they’re like I don’t know we’re just going to do

(08:16) stuff that seems good and then when you start doing testing and research you get kind of a little bit of a flashlight you can see a little bit in the room what’s going on and then the learning library really helps you kind of Orient all these different flashlights in the room so you can see like the different

(08:32) avenues that are ways forward um there’s different ways to create a learning library um but I think that overall the learning library is extremely essential and if you’re not creating it you’re kind of doing a disservice to yourself as you’re not able to tap into this sort of second brain that you’ve created

(08:53) you’re just relying on your memory to think about the research you’ve done in the past the research others have done in the past you’re relying on remembering what the interesting segments were in hundreds of tests and it just it puts you in a spot that’s not as effective as it could be you’re not seeing these different insights that you

(09:12) can with the learning library yeah and you you you touched on a on a really nice point there which was about the kind of access and understanding that the rest of the business has to this document because I think in my experience you know we create these spreadsheets like documents in notion where wherever they’re held we

(09:34) create these things that we as a cro team may understand um and you know we’re able to share them on screen and talk talk through them with a with a client or or with a colleague but if you were to just send that to somebody there’s no way that they’re going to be able to kind of decipher that um so I know I know you’re

(09:53) going to kind of get into the nuts and bolts around what’s included in this Library how it’s kind of laid out a little bit later but I just think it’s really important to understand that this thing that you’re talking about this kind of living breathing library is is important for for not only us and and and what we do

(10:12) day today in experimentation but actually could lead to a new marketing campaign could lead to you know an internal business change like there’s value to be had that spreads within the the company right exactly and we’re trying to create this repository too for all the different company like you mentioned everyone of the different

(10:32) company different areas of the company um but you don’t want to create this thing that is overwhelming you don’t want to create something with every little aspect of um the test you don’t want to create too many columns where people are like feeling like it’s overwhelming to look at look at it um you don’t want to just create a a link

(10:51) to a bunch of analyses like this is what I see with clients that are testing and they’re they’re starting to get it but then you see their repository of what they’ve run and it’s just this spreadsheet and it says like test name they’ve got variant a and variant B’s conversion rate uh the winner they’ve

(11:08) got the stakeholder Who’s involved the DAT it was run and then there’s a link to the analysis slides and it’s like it and it looks beautiful you’re like wow look at the work we’ve done there’s so much quantity here um and it feels like you’ve done this thing and you’ve actually just created something that

(11:26) looks nice that no one is ever going to use so I kind of take the approach of like thinking about the retrieval method what is the way people are going to open this document and try and take insights out of it and I think it should basically be this document where it’s at a high level where you can look at it

(11:46) you don’t have to click on anything else um and it’s kind of there to like just you can see it there’s no jargon plain language being used and you really take like the I think it’s Mary condo what’s her name the uh clean out your closet is mar condo yeah yeah yeah so it’s like that like if you don’t love a column get

(12:05) rid of it because like if you don’t love it like chances are other stakeholders are probably not going to see a benefit of it um will they should you include uh like stuff that you you aren’t interested no the stuff that I think should be in added and I’m still working on this too um what I want to be in the

(12:24) learning library but right now I have is test name simple enough should be something that anyone can understand you know and we get this wrong sometimes too though I get this wrong I was just fixing this yesterday the test name I’ll put uh you know um funnel maybe the test name is like Globe V2 uh remove carrier

(12:45) and it’s like What on earth does that me so it’s kind of like trying to make the the test name something that descri like describe the change you can maybe you have your test name then you have the description of what was actually changed make it short and brief people are not going to read your you know long

(13:04) description of what it is um then maybe you add in um the kpi that you’re trying to increase your primary kpi um what the result was uh very in in just a percent format um then you put the simple result of what what was changed so was it a save was it a win was it inconclusive um the page it was on the the learn and

(13:29) this is really the part where I think is the art part of things is you have the learning that’s broad so you start out with this learning that is a broad application of what you just um tested so an example of this could be um if you’ve run a test where you’re reducing the amount of options available uh and you find out that

(13:54) reducing the number of options available decrease conversion rates your learning that’s broad could be users prefer many options like maybe it’s a more analytical decision they want more things to see and then the the learning that’s specific is the next column so you’ve got the broad learning where it

(14:15) is a little more of a speculation um and and these you should speculate a little bit because we’re not trying to be 100% sure of why a test had the effect that it did you’re never going to know you’re just you’re just making your best guess based off the research you did up front based off the parameters of the test and based off

(14:35) the output of the test that can all affect the strength of the learning you’re getting but with this broad learning you’re being a little bit speculative why do you think it worked at a high level if you’re sharing this with an executive and wanting to see like 10 words what is it and then the specific learning so going back to our

(14:54) example maybe it’s you you add just a little bit more color users when when users were saw when users saw 10 options versus four options they had a decrease in or they had an increase in conversion because reducing the amount of options available was not as effective so you’re just adding a little bit more

(15:16) information and then you go even deeper where you actually L uh link the analysis slides you used and actually what goes in analysis slides is a whole different topic but again using this approach of what metrics did you plan to see changed from the get-go not just going on a fishing trip with every exper

(15:36) or every metric in the book and seeing anything that changed um that sort of thing and just to quickly wrap that up is the other columns would be year that it was launched quarter it was launched um one that I’m using right now is whether the change was in the funnel or not in the funnel so you’re able to see you know

(15:55) was the change made on a landing page a homepage or is it in this like funnel area and you kind of start to get an idea of like where where a test working so I’ll stop there and curious if you have any thoughts yeah yeah I think I mean it’s it’s it’s really nice to kind of think about those those fields that

(16:15) that sit in there and I know you you did a LinkedIn post earlier this year I think didn’t you explaining some of that so I’m sure if people want to kind of see that they can find you on LinkedIn but um I think the the bit I want to kind of focus a little more on is th those two columns at the end obviously that’s that’s the that’s

(16:33) the kind of main Crux of what this document is there to to um is there for so broad learning specific learning are you are you trying to then take those two columns and start maybe theming ideas together is there something that you’ll do with this next or is it a case of this repository is going to keep growing and you know maybe

(16:59) I’m looking at something that’s in the funnel as you say and I just filter by test that we’re in the funnel and then I start looking through my learnings like how how in a practical sense are you using this and how how do you expect others to yeah no there’s so many ways to use it that’s why it’s once you get

(17:16) this stuff in this documentation is a pain in the butt like there’s no way around it like you run a test you get the dopamine hit you’re like I’m done with this crap like let’s keep doing something else yeah but once you put it all on this rigid structure um it starts to unlock all these different ways to use it that I’ve

(17:32) and a few that I’ve thought have been really helpful are if I’m running a test now on the product maybe like the PDP page or in the cart I can like weed out just down to those results and I can see the learnings up front I can see what’s worked on those pages what hasn’t worked at a really uh high level and what I

(17:54) like to do as well is when I’m doing my test analyses I add in all the previous learnings we’ve had and then I add in the new learning we just created underneath that and then it starts to create this really like a different picture where your test analyses really start to feel like you’re developing that POV that point of

(18:14) view on the customer um so it really just it gives you easy access to other learnings that are in the same area it also helps you with ideation because you and I both know working with clients you’ll be having brainstorming meetings and people will say you know maybe we should try X idea and you go actually we

(18:32) tried X idea in 2022 five different times and we didn’t see anything there can you like help me understand what you’re thinking a little bit more there um it gives you like a really quick way to recall uh what the learning was too because people will I I’ve tried to run uh rerun test before and people people

(18:52) will say you know we ran that test before and we didn’t see the results that we think warned it and you actually pull that back up and you say well actually what we saw was uh X Y and Z so that’s super helpful um another really interesting way to use this and this is when you start adding in more columns to

(19:09) the learning library um you start to see stuff like you know are we winning more often and this is the whole thing I I really like this idea of like testing to learn and then you learn to win so learning comes first but then you learn to like win with your test but with this documentation you start to understand

(19:29) okay what what methods of research are driving winning tests what types of stakeholders Drive winning tests and um again like what areas of the uh website are we seeing that um we win most often with and we’ve seen a 3X increase in win rates in the funnel with one of the clients we work with so we now know like

(19:52) hey I know like Tim like you’re super excited about this test on the homepage but we’ve tried all these different applications to try and address um like fears uncertainties and doubts and that sort of thing but we have not seen anything happen there I think we should be spending more time in the funnel and

(20:11) continue to kind of drive and chart our course there nice yeah it’s really interesting I I guess just to just to go back a little bit you you touched on earlier about um documentation uh you know the Nemesis of of experimenters but um a necessary evil right um do you see this document as another thing we’ve now got to keep

(20:39) updating or do you think that it can maybe replace some of the other documents that we’re using yeah that’s such a good question I was I was sweating over my learning library yesterday um and I spent about two hours in it trying to update everything it is difficult but what it ends up showing is it shows so much of the progress

(21:00) you’re making it gives you so many different views uh to look at and again I think the hard part Nick is like we see that the industry mostly wants quick wins and revenue generated so you’re kind of creating this thing knowing it’s the the greens or the vegetables that the client needs your stakeholders need but no one’s going to

(21:24) say like hey Nick can you please update the learning Library I’m dying to see what is what’s in it so it’s kind of you’re building this off the back of knowing it’s going to help you build that culture of experimentation it is but but to back to your question it is difficult so the air table version is nice because it’s free

(21:41) uh you can watch Ruben’s course for free as well but then after a test you have to update all these fields and then if you forget after a test then you’re updating two tests you forget again then you’re like man what was the learning on that isn’t this the whole reason we’re creating this but yeah it does

(21:57) become I hate to call it another thing but it’s it’s almost required it’s like when a police officer pulls someone over like they got to write their their documentation of what happened they don’t like doing that but they’ve got to so we the job right yeah yeah and I suppose like the the bigger this thing

(22:14) becomes and and the more learnings that are in there the more useful it is outside of and maybe one day you get to a point where people are asking hey can you update the learning library because that they’re they’re starting to to see it paying off and they’re learning from the tests you’re running applying those

(22:31) to other areas of the business as well as experimentation so you should eventually further down the line it will start kind of paying back too right yeah um and who so with that in mind it feels like ownership of that document belongs to the cro team the program manager you know who who do you think owns that and um have you got any

(22:53) tips on kind of maintaining it yeah so the way that we’re using it right now is the cro uh has the the task of updating it um they are in charge of updating it and that should happen I’m working on checklist now to kind of include that as like the final step of a test analyses like you do your analysis look at all

(23:23) the segments input it in the learning library um that should be on your list but the way that we currently use it to kind of build usage of it is we’ll share it in client meetings um like you were saying earlier the benefits you see is like we’ve seen a 1.5x increase in test iteration so that’s a really cool step

(23:42) to kind of build uh the desire for people to want to see this but we send it out in our status email so we’ll say like thanks all for like talking today here’s a link to the hypothesis Library the learning library and the the test intake form so if you have any like ideas throw them in there um but we

(23:59) create a view that’s paired down for the client so it’s not as many rows but they’re able to use it and and we send it out to a large list of people and I know that’s not the best way probably to build uh usage of it but it creates this opportunity where people can open it it’s easy for them to share like the

(24:20) progress they’ve made doing this extra task that sometimes is just another thing on a company’s list um so it does do a good job of starting to build awareness of your program cool I I guess you kind of touched on it a little bit there but you know let’s say we’ve all gone away created our learning libraries uh we’re

(24:42) all praising you for putting us on this path um and as a I guess from our point of view VI you know we’re working on behalf of of clients but this could work internally like with with uh CR teams speaking to other stakeholders but how do you encourage the kind of two-way interaction that this thing probably needs in order to kind of build

(25:06) traction and ultimately you want people coming to this yeah kind of off their own back right you want them ask like we’ve said you ask me about the learning library you know you want you want to encourage that collaboration across the board with it like how have you found you mentioned a couple of things there

(25:25) you know bringing it up in in in meetings sending it as as follow-ups have you got any other kind of examples where you managed to get people excited about it it’s getting there it’s definitely one of those things that’s it’s not the easiest sell right like we know it’s we know it’s what they need we know it’s

(25:44) generating value through the stats we’ve talked about in this but it’s still just another thing for people to look at and I’m trying to get it to a place where it’s not just something that we use for better analyses better iterations that it’s something that is spread across the business we’ve seen a couple times where

(26:03) um one of the guys that we work with is in marketing and he’s taken the learnings we’ve gotten from Tess and put them into email campaigns which has been interesting so like based off what we’ve learned about trust pilot was a big thing we did so everyone’s seen that it’s the star rating that’s on a lot of

(26:22) people’s homepage we ran that and didn’t see any any lift like we ran five tests didn’t see anything and having those learnings available he was able to see like hey maybe we shouldn’t include this in email maybe we should think about like other ways to leverage email um but as far as like getting people to

(26:40) actively engage with it it’s tough it’s tough to get a lot of Engagement with C I think that’s why we see so many talks about how to build a culture of experimentation it’s because it’s not easy and this is like even this is like hard mode almost like to get people to do this part of it like getting people

(26:56) run a test is hard to begin with um but I think a lot of this the the responsibility is on you like we we need to leverage this learning library we need to put it in our weekly slides we need to include it when we intake test like hey reminder your intake or the test you submitted last week here’s five

(27:17) learnings that are related to it so you kind of you have to do the work for them and I don’t know if that’s because we’re in we’re in we’re both working for agencies or consultancies like that we by Nature a lot of people we work with they want to be served which fine by me um but maybe in house it’s easier to

(27:35) kind of like Get By into this but yeah it’s been it’s been difficult um it might be good just just to kind of segue on and just maybe touch on some of these other kind of tools and sites that are that are popping up and some have been around for a while you know you’ve got all of these different types of

(27:54) repositories as far as I can see a lot of them are based on that other dirty word best practice and I guess the what I wanted to to touch on next was where they’re looking at maybe Industries as a whole and saying you know you can segment down your mobile designs by manufacturing or by uh e-commerce and

(28:17) then maybe have a look through what other people are testing that’s you know it’s good it’s it’s it’s a good thing to kind of get you get the ball rolling get some maybe some idea another business ter um you know get some ideation sessions moving that kind of thing but with this learning library is there an

(28:38) opportunity here to and maybe more for the consultancy side and and agency side of things is there an opportunity for kind of macro learnings across multiple different accounts and different clients and different um different Industries yeah I mean are there’s a lot there so hopefully I don’t forget it all

(29:00) but there are there’s a lot of cool tools out there those are all great for helping you come up with ideas that are more there’s a good chance or there there’s been shown there’s a good chance that that will work in certain industries um another thing that people do to try and give you ideas that will

(29:19) be a good idea for your industry or just in general you’ll see those LinkedIn posts where people are like comment please send me the PDF below and I’ll send you all the ideas and it’s like okay but yeah those are those are really interesting areas for ideas for starting out your research um good way to see

(29:38) like what people are doing and like things you could apply to problems you’re seeing you want to make sure they’re kind of like synonymous this the problem and the treatment you’re finding are aligned but kind of what you’re asking to Nick and is more of when you start to take these learning libraries

(29:58) and make them even broader um what kind of can you get out of that and like what does it look like um we’ve taken the time to make a repository of all our clients in all of their tests and all of their learnings and we start to see this really interesting view of what works and what doesn’t work in our company and

(30:21) that can be really interesting learning too it’s like if we’re if we’re seeing um at our company that our our lowest win rate is heat map and session replay then maybe that tells us like we need to work on take courses in like trying to use those better um it’s really interesting to start to see stuff

(30:45) like that and maybe we see that you know the type of clients that see the best improvements are the ones that are uh engaging with us on a weekly basis that sort of thing um but there was just so much in your question I thought was really good uh is was there anything else that uh I kind of missed there no I

(31:03) think um yeah the the main point there is is I guess around macro learnings and and how that applies and I I think you know for a for a consultancy or for an agency learning what you’re maybe not doing well is is a massive um a massive win and you know you start training teams up in those ways so that’s that’s

(31:26) really interesting um I think in terms of other resources that are out there is this the kind of thing really where you know you’ve you’ve given us each field that needs to exist in this in this spreadsheet air table whatever it is is there anything that we can any tools out there that you prefer to use

(31:49) or is your stuff all in an Excel spreadsheet um everything was in an Excel spreadsheet before started to get annoyed with how learnings were being used but yeah everything right now our most effective uh uses have been in air table um so go to air table you the course is on you to me uh with Ruben D’Or he’ll

(32:10) get you all set up with how to set up this amazing air table but the the thing to think about while you’re taking this course is like how am I actually going to retrieve this information you want to customize it to your needs if you don’t want to use Tech that’s new to you you still can use Excel if you want to or

(32:27) Google Sheets just make sure that the columns you’re using or columns that you love that are actually digestable do the work of trying to make that speculative broader learning do the work of putting that specific learning in there as well link to slides that are not a hot mess like slides that have metrics that are

(32:47) like cleaned up and the insights are uh easy to understand um but that would be kind of my my top U resources for this this specific um use case really a lot of this is just kind of on you to take this approach and start to try and use it a little bit better and try and try and like fight back a little bit against

(33:11) that quick win feeling and try and think a little bit deeper like it is hard and I think that’s the reason this all is coming up again is like it’s hard to think about learnings it’s difficult it’s not going to be something that you just see the percent lift it’s you’re going to have to figure out the tool

(33:28) that works for you which there were a few we just mentioned and then really do the work of getting this to like work for you and is there like a is there a particular point and in time where you think Now’s the Time to start building this or is it literally from day one from test one get it documented and just

(33:46) start doing it and adjust from there yeah I mean I’d love to say that this is something you should start up right away um it’s just I think it’s the way that I do things best and that’s just my experience is when I really feel the pressure for like hey this system is not working anymore then you get that fire

(34:06) in you and you’re like I need to create this thing that’s going to help me do my job better and then you have motivation to do it it’s so easy to say just yeah just start doing it from the start when you have no motivation and you don’t really know what you’re doing you’re just starting out a program but I mean I

(34:20) would say like once you’re running and volume is like not very good to even talk about but like if you’re running more than three four tests a month like you should start putting them somewhere um but yeah if you’re just running one test a month or something maybe you don’t need something like this but I

(34:39) think that once you scale up and mature it’s going to be something that’s really useful and you really do need somewhere to show like your progress and what you’ve actually done or someone from executive leadership is going to be like hey what are you paying these people to do and then you’re gonna have to go

(34:55) through a million power points and try and find out what the heck you’ve done but if you do this up front you’ve got the work done for you already so that’s one easy motivation point for doing it from the start yeah yeah for sure so you know we we’ve touched on loads there I think there’s there’s lots for people to be

(35:12) doing and and what I imagine is that people have maybe versions of this in their you know that they’re updating daily maybe it’s more of a road map or a test repository or something but I think some of the the KE points that you’ve raised today are going to maybe encourage a few people to go away and

(35:30) and assess the documents they’re keeping me included um I might give it to someone in my team rather than me doing it but um uh because it sounds hard but sounds beneficial um so I guess it’d be nice to maybe just touch on how how this learning library fits in with I guess the way of doing business that is that is cro you know

(35:55) we’re we’re trying to encou as an as an industry and as a group as a collective of of cro we’re kind of tired of being treated like a like a line item right you know we’re we’re we’re trying to tell people and educate people about cro how it actually is a a culture shift within a business it’s a different way

(36:15) of of doing business it’s about experimenting learning growing how do you think that this learning library could maybe fit in into that ecosystem and maybe even help us to to tell that story yeah exactly Nick so pretty much um you touched on it like our industry right now they’re focused on most a lot of companies are focused

(36:37) on quick wins driving Revenue um we’re kind of like pigeon hold in a spot where we’re like the number go up wizard like we’ve got to do this thing quickly um but we all I think most people that really know this know that we’re trying to make c a whole way whole new way of doing business we want it to be the

(36:54) foundational piece um where people are just obsessed with making better decisions and AB testing and all the different types of testing are a big part of that um and this is whole like Brewing thing of experimentation Le growth not product L growth not sales L growth we want to have experimentation lead growth which is amazing but the way

(37:14) that we do this is by no longer being this percent uper down machine we make our learnings understandable to all groups of people if only people that know what statistical confidence is and power is can understand the learnings we’re creating then we’re siloing ourselves not doing ourselves any justice so by creating this learning

(37:38) library however you decide to create it you are doing the work of making the learnings um broad so anyone can understand them making them specific so they can get a little more details and making your decks and your analyses um to the point where people can really see the value being created amazing so we have tried to use our

(38:03) words to to help people visualize what this thing looks like but conscious that you know we haven’t brought anything up on screen uh which is kind of on purpose you know we wanted to have a chat today not just uh share another presentation um but you mentioned to me before you’ve you have written a presentation uh called test to learn

(38:24) learn to win which we’re going to try and put in the in the description so there should be a link to that in there and that includes not only kind of ways of doing cro but also some examples of what this learning library might look like right just to give people a few prompts to get going yeah yeah no I think we’ll link that

(38:43) deck and it’ll give you it’ll really give you a good idea of how you can show the value of experimentation to and then it’ll show you a pretty pragmatic approach to how to create this thing um that we call the learning library can also um link to maybe a few photos of just like what my learning library

(39:01) looks like make an album or something where you guys can see but also don’t hesitate to reach out to me on LinkedIn if you’re interested in this thing and you want to talk a little bit more about it I love to like post about this kind of stuff uh it drives me crazy when I see that learnings aren’t being shouted

(39:16) from the rooftops and also when learnings are being used in a dubious way kind of drives me crazy too so if you want to just chat about how to actually do this thing I’d be more than happy to him awesome well uh it’s been great to have you and uh I was trying to remember your nickname then Nick it sounds too creepy I can’t use it

(39:38) no it’s great to have you on um I think there’s there’s some even just from this podcast alone there’s some great learnings so hopefully you’ll add this to your learning library um yeah it’s been great to have you and uh thanks very much guys yeah no thank thank you Nick thanks for having me on the experimentation podcast it’s been my

(39:58) pleasure this is Romo Santiago from experiment Nation every week we share interviews with and Conference sessions by our favorite conversion rate optimizers from around the world so if you like this video smash that like button and consider subscribing

If you liked this post, sign up for Experiment Nation’s newsletter to receive more great interviews like this, memes, editorials, and conference sessions in your inbox: https://bit.ly/3HOKCTK

---