The Challenges of Building a Culture of Experimentation: Insights from Industry Leader Nima Yassini

AI-Generated Summary

Join Nima Yassini as he dives into experimentation culture, the journey of building a brand UX and digital marketing agency focused on conversion rate optimization, the integration of digital marketing and experimentation culture, the struggles and benefits of adopting a culture of experimentation, and the need for leadership and reinvestment in the experimentation industry.

Audio

AI-Generated Transcript

(00:00) well how we’ve lost 10 one we only won one out of 10 that sounds like a stupid like if I went to casino and I bet 10 times and I only won once like if I’m a rational human being that’s pretty bad odds I’m not doing that again whereas if I don’t know how much of the stuff I’m pushing out into the world productionizing is working

(00:18) well ignorance is bliss hi my name is Romo Santiago and I’m the founder of experiment Nation on today’s episode we have Nima yini and he shares his views on the challenges of building a culture of experimentation in agencies as well as with clients he also speaks of the importance of technology investment and change management in our

(00:40) industry we hope you enjoy the episode hi this is K Gala welcome to another episode of experiment nation and today’s episode our M excited have Nema Nema yini with us uh Nema was the CEO and founder of of of New Republic a brand ux and digital marketing agency focused on delivering conversion rate optimization

(00:57) strategy that help clients understand their ideal customer and build website experiences that convert n I’m super exced to have you to thanks for having me on and thanks for that wonderful intro might use it on my LinkedIn amazing Nemo uh you know one of the things I know know researching you and reading about you is the fact that

(01:17) you know you’ve combined two things you’ve combined digital marketing and you’ve combined sort of experimentation conversion rate optimization in Australia can you talk about the Journey of building that agent kind of took these two things together and combined it under sort of user experience um so

(01:32) my my background I started it’s going to show my age I started in the.com rise in Australia so we’re building websites on CD ROMs I’m sure a lot of people listening probably go what is a CD ROM anyway look it up on the on Google um and you know that I kind of evolved in that journey and ended up selling

(01:51) working in this company that sold banner ads on websites and then fast forward and then 2007 I got really interested in um user experience design and that that was really fascinating to me I really love the idea of understanding why people behave the way they do how how Journeys could be designed better

(02:12) because you know in the old days Engineers would build websites and and you know trying to convince people that no no no you got to think from a users perspective and Engineers think from a coding perspective and the internet wasn’t as friendly as it is today and so that that really fascinated me and so I

(02:27) got into that space and then you fast forward I went to London came back got introduced to experimentation came back and started an agency and I started an agency as a creative agency and it was not only until my partner um came in and we kind of refounded the agency did we start you know we went back to our

(02:47) heightland moved away from creative Services went back to our heightland of users experience design and then she and I started thinking about how do you differentiate in ux and I can remember we’re working with a big bank and the client said I want to put the logo on the right hand side so you know logos

(03:08) traditionally top leftand Corner they used to take you to the home but anyway all of a sudden and we’re like why and it’s just like oh I just I just like it and you know that one conversation was this kind of precipice where I was like I am so fed up with this industry because it doesn’t matter how much experience you have if the

(03:29) client wants something that’s it you’re doing it regardless of it’s good practice or bad practice and so I just started getting you know I started getting quite passionate about the subject and all of a sudden I started throwing my toys out of the pr and then my partner who’s far more rational than I am um came back and said

(03:48) find another way to manage these conversations and so it was at the same time that you know Big Data was coming on board and talk about data lakes and I kind of knelt back into experimentation based on the work I was doing in London with HSBC and then I thought hey why not merge these two worlds together and and

(04:10) what was really unique for us was a lot of the people doing you know conversion rate optimization or conver or experimentation in those times with more media Leed companies that were looking to you know reduce the cost per lead um convert more on the landing page and we were like this could be quite different

(04:29) because we’re looking at websites more holistically from a journey perspective big complex Journeys how do we optimize those Journeys and so that became our kind of integration point and to be honest it was it was really around not wanting to fight with clients but having a way of nullifying um subjectivity of design

(04:48) using data to have that conversation so allowed us to be competitive and be evolving our product offering but at the same time helped us evolve in into something that becomes less about the subjectivity of what a client wants to what is the data and the customer telling us bringing the customer truly

(05:05) into the design process and that was that was the journey that without knowing it that one decision that integration Point changed everything changed the way we looked at user experience design changed the way we were as a business changed our proposition it changed changed that changed our team structures it changed

(05:24) absolutely everything in our business for the better I mean we ended up you know went from working with companies you probably wouldn’t know of to the biggest brands locally in globally so it was a it was a a change we didn’t expect but when we did embrace it completely changed the way we looked at the world

(05:44) and approached our work and changed me as a designer and um as a ux strategist as well in the way I thought about my work this is Romo Santiago from experiment Nation every week we share interviews with and Conference sessions by our favorite conversion rate optimizers from around the world so if you like this video smash that like

(06:02) button and consider subscribing it helps us a bunch now back to the episode amazing Nemo and just one thing there I want to kind of I want to dig into and it was it was sort of the notion you said but we have this client who kind of said who said they wanted to put the logo in the top right corner so

(06:16) I what I imagine is a lot of other agencies if they were just focused on creating a website they would just say yeah whatever the clients want put them up there but my sense is the fact that because you’re focused on experimentation and cro is this is is is it is it kind of notion where you’re pushing back on data because you have

(06:31) sort of targets because you have sort of because it’s not just the visual it’s actually experimentation is actually moving the needle so when you problem conversion rate optimization moves the needle experimentation doesn’t it’s not mil to and and this is the we talk about this now our category loves the nval gra

(06:49) we love talking about the subjectivity of words right but fundamentally conversion rate optimization moves needles it is it’s measure is a conversion outcome so you can very directly go it’s sold more it didn’t whereas experimentation is a learning platform it’s really trying to understand what is a better way to do

(07:07) something so for example if if I’m trying to circumvent putting engineering hours into a piece of work moving the needle might not be it’s proving that what I’ve got now is actually better than what I could be doing and so not wasting all that operational hours which what is an a that is that is exactly the

(07:28) definition of an a test is what I’m doing now is better than what it could be right and so there’s an operational efficiency if you’re looking for um you know we had a retailer who was trying to reduce the number of return rates because it was costing them a bundle during Co and so that was more of a cost

(07:41) savings rather than a Roi you know increasing so it’s much broader and I and I feel like the world is going to move into those two you’re either looking to move a revenue needle or you’re looking for oper op AAL and insights and so we’re using the fundamental layer of AB testing but we’re looking at the problem in

(08:04) different ways and measuring in different ways if that makes sense okay and see and and and if you think of the two things are they do you imagine the two different departments looking at those because I imagine for like a conversion optimization is probably marketing and sales or e-commerce but let’s say operational efficiencies might

(08:19) be more the this isation right a whole category talks about because it is it’s it’s a it’s a tool that everyone should have in their suite and should be able to use not one department or one person it’s a tool that and you know you you look at all these organizations that talk about culture and you know you go to these

(08:39) centers of excellence structures where everyone’s testing and you got a group who’s just managing what everyone’s doing but ideally um yeah it should be something that everyone does it’s not a department or a person and it should these are all tools you know you use different tools at different times to

(08:56) create different outcomes they’re just tools yeah so so so maybe on that point Nema about of like like the the culture of experimentation right and and here you’ve talked about how like you know in terms of like agencies agencies are kind of like almost like a third party they’re kind of doing that work on their

(09:13) side but can you talk about maybe their journey of sort of you know there’s an agency that may have its own sort of culture of experimentation versus a company that who’s H like who’s hired how did that culture ofation go for this is something I’ve been you know since I’ve sold the business and now left the

(09:30) business I sold to it’s something I kind of I can’t stop thinking about is you know this idea of a culture of experimentation is really around making yourself as the service provider whether you’re a consultancy or an agency whatever you want to call yourself technically you’re trying to make yourself redundant because you’re trying

(09:49) to infuse that culture into your client’s business and ideally what you’re doing is helping them bring it inhouse technically you’re doing yourself out of a job right and that doesn’t really mesh well with an agency model or a consultancy model where you’re selling resource time and fundamentally the product of an agency

(10:10) or consultancy is skilled labor you know very high skilled specialized capable labor so China sell in a culture experimentation just it doesn’t it doesn’t bode well with the future of trying to grow your revenues as an agency owner or a consultancy owner so I think I think the the idea around how building a culture of

(10:33) experimentation you can’t get agencies to do it and on the back of that clients on their own can’t do it because they’ve got a way of working they’ve got a baou and to and I mean I went through our business went through a massive change program to to adopt a culture of experimentation little did we know we

(10:52) were going to do it but we ended up moving down that pathway you know like you know the way you cook a lobster you don’t put into hot boiling water you put it into cold water and same thing right we didn’t realize we just slowly evolved that way um so a lot of times when you go to clients they try but they fail

(11:10) because they don’t really realize the different points that they need to go to the key metrics within each one of those points you know I talk about the Three Peaks of experimentation you’ve got a political operational and democratization Peaks and you got to go through each Peak and each Peak has a

(11:28) different metric that you’re trying to hit and if you don’t understand what Peak you’re in you end up doing the wrong things expecting a different outcome and so a lot of clients don’t know this stuff because they’ve haven’t done this over and over and over again to learn how to do it and so they hire

(11:41) the agencies and the agencies want to sell people and so those two things just don’t seem to work together and so building a I’ve spent a long time thinking how do you actually build a culture of experimentation and I read this phenomenal book called Atomic habits and I realized something what we did at agencies was we we would or

(12:02) consultancies we would sell you people what we didn’t do is formulate the habits that the organization needs to adopt a culture of experimentation and so I’ve become really passionate around what are the habits you need for experimentation what are the things that you need to adopt to create a habit of

(12:20) experimentation and that’s what experimentation is we talk about culture culture is a series of events and activities that you do ritualistically over time and have a shared belief system between a group of people and that’s a culture and so those things are fundamentally grounded into what are the habits and beliefs that you have that

(12:41) cause that culture to occur agencies traditionally don’t do this type of work they do productionizing adopt a culture of experiment truly adopt a culture of experimentation because it doesn’t benefit them if they’re an agency and so and the C and the clients don’t really know what to do because a lot of them

(13:05) are self-learning and so it’s a real dichotomy around how do you embed a culture of experimentation and and a lot of it comes out of habit formation and I think the people who are really enabled to do stuff like this are organizations who focus on change programs or who have a human capital capability who

(13:24) understand how you change a group of people but a lot of those people right now don’t understand experimentation so those two worlds need to come together for that to occur but the challenge that we face is that you know I think experimentation with optimizely being bought out um I think the the category

(13:42) is at risk of becoming a cottage industry because the people who talk about experimentation are practitioners like us would just talk around each other and because we tell Al you know the algorithm on LinkedIn what we like our LinkedIn look like like experimentation but I’m not sure the wider business Community really

(14:02) understands what the hell we’re talking about and we all think it’s important because we all see each other’s feeds like I see I see shiver’s feed I see your feed I see rl’s feed like I’ve got all these people who I you know admire and respect and love their opinions and like I mean shiver is is awesome he the

(14:18) guy puts up so many funny memes that I’m like I just love this guy he brings humor to the category right every one of but fundamentally we’re all talking to each other there’s no one growing the category there’s no Technologies investing in growing the category and so I I the idea of a culture of experimentation I’m slowly starting to

(14:37) figure out how we can do it but I don’t think there’s anyone building the category and educating the sea Suite to allow for this to start to occur and we’re at risk of becoming a cottage industry where we all just talk to each other and think we’re all really important I’m included in that group that I’m talking

(14:55) about yeah I mean this is where you know what what I noticed when I was working in different marketing departments was you’d have the you know it’s it’s it’s always a simple of high school analogy you know you’ve got the jocks and you got the nerdy we don’t have and I always we don’t have that you know like the

(15:12) experimenters always feel like the nerdy kids are pulling up their like statistical significance and their P values time yeah yeah you know versus versus where those executive decisions are made where those powerful decisions are made by people who can who can talk who can you know can you know really make things simple and not get

(15:33) lost down into that massive data whole the love experiment and also the Silver Bullet everyone thinks people who bring on experimentation usually think it’s a silver bullet like oh we’ll do a test and we’ll get a billion dollars doesn’t doesn’t really happen that way you lose more tests than you win you know but you

(15:50) learn and and I think that’s really important you learn more but without a technology partner you know what Sou Force an adobe does to the market is it Mass funds um focus around subject matters which technically they are investing in to R&D and build you know they are r& Ding and building into personalization

(16:14) there are like the number of technologies that talk about artificial intelligence all of a sudden the sea sued are all like how do we get artificial intelligence into our business because Salesforce has has a huge sales team huge huge marketing budget same with Adobe and they flood the market with this conversation and

(16:32) then you know see and like they’re not illogical difficult Concepts to to absorb like a CEO will like yeah how do I find efficiencies through artificial in intelligence how can I reduce my Workforce or how can I find more profit and so that’s their job you know Coos C CEOs that’s their job without someone

(16:50) funding the market and pushing it Forward you’re never going to grow and the current Technologies in this bace they all they’re all fighting against each other to win small contracts and no one’s really investing in building the category like there’s no we don’t have an association for experimentation and

(17:09) growth specialist who then takes the voice of this conglomerate to technology companies to Industry and so at the risk of um you know I’m kind of incriminating myself but at the risk of it being a passion project this thing is very close to becoming a cottage industry and and in terms of c one thing

(17:34) that’s kind of bouncing on my head right now is is it a question of the Personalities in the space or in the sense that are they is some you know the kind of person who would be doing experimentation are they the kind of person who just end up in a cottage industry to be to begin with no matter what thei you know you’re talking about

(17:52) I come from ux background you come from marketing background um sorry I’m paraphrasing if I got that wrong let me know um you know everyone comes at it from different angles you know people come from a data angle people I do agree we’re all nerds but anyone in the digital category is a nerd like in my

(18:09) point of view you got to be a nerd to be interested in the evolution of technology and you just got to have that kind of inquisitiveness or you know and and they’re complex stuff so you got to kind of be able to break things down to understand it in your own context so I I don’t I don’t necessarily think it’s the

(18:25) people I just don’t think that the conversation has been delivered in a consistent manner you know using marketing at the end of the day advertising and marketing drives awareness if I want you if I want remember the the early days of optimizely they were everywhere like literally everywhere I used to go to

(18:46) Opticon and it was huge you know I mean it wasn’t close to what Adobe was but there’s nothing similar and the events around the world you know you got conversions CR doing the growth summer got cxl in Austin you got all these really fantastic but there’s no Technologies and I’ve noticed that our category Grows by a technology if a

(19:06) technology funds something the conversations Elevate you know people start talking about it and then the service providers follow through to deliver services against it without a technology leading the conversation this the category has no leadership it doesn’t have a a beacon to follow and the service providers to

(19:26) follow through through and so what ends up happening is someone in the organization gets interested and gets a little bit of budget to trial it to see if it works and then they like it and then they make a business case to bring it inhouse right which means hiring people and then as we all know they all

(19:43) get stifled because they can’t get enough engineering away from baau work into experimentation and and those people who take it up they usually like you said they’re a marketing manager Ecom manager data and analytics optimization specialist and they don’t have enough seniority and they’ve got to ask someone else for budget and that

(19:59) other person because they don’t understand what’s going on and they’ve got no one in their e educating them they end up going well how we’ve lost 10 one we only won one out of 10 that sounds like a stupid like if I went to casino and I bet 10 times and I only won once like if I’m a rational human being

(20:18) that’s pretty bad odds I’m not doing that again whereas if I don’t know how much of the stuff I’m pushing out into the world productionizing is working well ignorance is bliss let’s just keep doing that because I I put more stuff out something’s working right we’re growing something’s working let’s keep

(20:33) doing that so that’s that’s the that’s the trouble I I think we need we need a technology we need optimizely and you know I don’t want to talk badly to the guys but they’re selling a much more holistic solution and so they should they’re still leading the experimentation conversation is in there but you know Adobe doesn’t talk about

(20:50) experimentation at all it talks about personalization and they’ve embedded their experimentation into their personalization package right you AB test within a personalization experience but they don’t even talk about it so if no one’s talking about it the technology side and the category behaves in a

(21:06) similar pattern which is Tech talks about it CEO CEO seite listens to it talks to their people the people go out and find a technology to support it and then the market grows and service Prov if that doesn’t if that routine that is shows to be the pattern of the category that pattern is not happening how does

(21:25) this category grow so I don’t have the answer is it that we need a body is it I don’t know but I do know that there’s some incredibly intelligent people doing some incredibly great work but without that rising up to the SE Suite I just I worry that it does it won’t grow anymore you know and and and dear P

(21:46) something I’ve noticed as well is me you know we spoke about it as well is someone telling me how they were telling their mom they worked as a cro and the mom kept thinking they work as a chief Revenue officer and what I’m trying to get to is that even in companies you might be in a place that has an

(22:02) experimentation culture but it’s not like looking for like a digital marketer role we could just you know you know Google that word and a ton of and a ton of jobs show up it’s still very Niche and so what what I’m wondering is well you mentioned how you know this this of tech element you know you know Tech

(22:18) talks about it CEOs kind of get into it there’s of an element of having technology behind it but it seems for me as well it’s almost a human resources element to it as well because you don’t have it’s a it’s um it’s not a singular discipline it’s a multiple like I don’t know any experimentation program that’s run by

(22:36) one person I just I mean if if it’s a decent program it’s got a team of people you know you need Engineers you need data people you need designers you need and that’s if you’re just doing front if you’re doing backend stuff you need more engineers and so forth right so it’s a team of people and then if you think

(22:50) about it and this is why why most organizations don’t land a culture of experimentation because the kpis that they give these people to what an experimentation program does they’re very at odds with each other most kpis are based on success metrics rather than failure metrics and experimentation is about a failure metric because failure

(23:07) teaches you what to do and so the way the organization is set up doesn’t allow for it I mean like I said to you we completely have to change our business like we have to change the way we thought about team incentive programs and staff incentives and kpi metrics and moving away from kpis into more agile

(23:25) structures and moving into more quarterly based like it was a complete change program it’s multi-dimensional and so you can’t you can’t just hire a guy or a girl and go you are a cro specialist because they’re not they need a bunch of other people to fulfill on the work and I actually don’t think it’s

(23:44) a person’s job I think it’s part of everyone’s job to be experimenting I don’t think it’s one per one person might lead the process but fundamentally everyone before they make a decision should go well let’s go test that hypothesis or let’s take that qual research that we had out of talking to you know I don’t

(24:04) know 100 people and what’s its impact if I put it at a Quantum on my website or my app what would be the financial impact on me because yeah 100 people said it but it might only have like 2,000 people apply or respond to out of a million is it really worth my engineering Department’s time to do that

(24:22) and it might be depending on the value but I I I just don’t think it’s a person I think it’s a it’s a change I I honestly the more time I spend around this it’s a cultural change program it is a multiple faced cultural change program if you’re embedding experimentation in a business if you’re looking to increase leads then yeah it’s

(24:42) a conversion rate optimization program you can probably get one person in the Ecom team or the media team to drive that stuff you can keep it very unilateral just like seo seo has C it has Dev it has has a whole bunch of people yes one person becomes the SEO specialist because they got to learn the Google algorithm but experimentation

(25:04) doesn’t work like that it is a program of work that everyone should be doing so NE as you’re coming close to the end of episode one of the question I had was let’s say let’s say someone is either working experiment experimentation right now they’re thinking about it are they better off kind of working on their own agency

(25:22) maybe working as solar preneur or are they better me looking for the few places understand experimentation or is a situation maybe even more complex than that um I mean it really depends if if you’re out there going what do I do with my career if that’s the question you’re asking um look I I’m I’m somewhat you

(25:41) know biased I love this category I love the subject like I I write about I I’ve stopped working you know I’ve sold the agency I’ve sold I’ve left the business and I’m still interested in the subject you know hence why I’m on this podcast I just love this category I find it Inc incredibly interesting but for someone

(26:04) starting their career I I think you need to have multiple um until there’s a technology investing in it you really need to have a multifaceted career capabilities because experimentation is a tool within a suite if you do wanted to get into this space I would say agencies are the best place to hone your

(26:21) craft and there are incredible leaders out there who can teach your craft but really I think where you’re going to see real impact is when you take it inhouse when you take it into a business and you’re able to manage the politics and bring the business on a journey I think that’s where you’ll see the power of

(26:39) what’s possible and if you ask me the people who run experimentation programs the change enablers if you do this properly your business quadruples like I know I we did it you know we changed that yes we’re agency and yes but we were able to you know deliver multiple millions of dollars to our incremental Revenue to our clients we

(27:04) were able to save them money we were able to tell them hey invest in developing this part of the product don’t invest in that part you know and statistically prove it like we were we were change agents across the business once we were given permission to have those conversations and I think if someone can learn the practice at its

(27:22) depth you know move Beyond masterer into Artistry and go client side I I really feel like they are change agents they’re not this Niche I drive conversions they are really thinking about how do I get people to you know use an experiment to get data to make better decisions in their day-to-day practice that’s a

(27:40) that’s a real liberating and interesting place to be but you need to learn your craft and I I feel agencies teach better than clients but clients enable you to do some really big changes because you have card launch where as agency you’re always asking for permission to does that answer the question

(27:59) amazing I think I’m either going to be the most hated practitioner in this category for calling at a cottage or it will be celebrated and talked about and some of the tech guys will come and say we’re going to invest in this category and build it up again because I I think the space for a tech player to reinvest

(28:16) um to do what you know pick up from the fantastic job that optimizely did and continue that conversation amazing emo and so uh before we wrap up is there anything that you wanted to highlight or anything that you want to kind of focus on oh please don’t hate me whoever’s listened to this focus on that

(28:36) and then the second thing is um if any you know out of the conversations we’ve had one thing I love about this category you can reach out to any practitioner and they will share that I I 100% get I can reach out to anyone and go hey I’d love to learn about this you know Ben from spir I know he puts out so much

(28:56) great content about how he runs a program I mean the guy is phenomenal in Sharing knowledge um so I I guess in that vein if some of these topics are interesting to people and they and there are agencies trying to build out of practice I would love if you could reach out to me I’m happy to share everything

(29:17) we know about how to build proper programs so that the agencies when they deliver to clients they don’t make basic mistakes and they start to deliver better work because I think if we can all do better quality work and clients get the results the category will continue to grow so I’m I’m offering any

(29:32) agencies out there that want and then any clients who are interested in adopting culture and they want to know what to do I’ve got a ton of material and information that I can share I’m Keen to help people be better in this category and help the category grow because I feel like we’re lacking leadership we don’t have any kind of

(29:49) beacons or people at the front saying we’re going to help grow the category and so I’m whatever I can do to contribute to that more than happy to amazing NE maybe with that as well could you could you share for example if anyone get in touch with you what what what what what’s the best LinkedIn is the best way you and I kind of hooked up

(30:08) through Linkedin and talked so um look me up so it’s ni m a uh yini y a s i n i so name yini I don’t know if you can do a caption on this incredible platform that you’ve created so happy to share R’s got my details cly you’ve got my details um any that whatever I can do to contribute to help people do better work

(30:29) deliver better results and continue to grow this category that’s what I’m interested in amazing NE thank you so much for being guest today man thanks for having me on the platform thanks for doing it late at night and also early in the morning for many say passionate people this is Romo Santiago from

(30:47) experiment Nation every week we share interviews with and Conference sessions by our favorite conversion rate optimizers from around the world so if you like this video smash that like button and consider subscribing it helps us a bunch

If you liked this post, sign up for Experiment Nation’s newsletter to receive more great interviews like this, memes, editorials, and conference sessions in your inbox: https://bit.ly/3HOKCTK

---