Building a CRO Culture & Ideation Strategies with Lucia van den Brink

AI-Generated Summary

Ideation strategies are essential to building an experimentation culture. Companies should encourage employees to generate new ideas and provide a platform for sharing them. This can be done through brainstorming sessions, hackathons, or online idea-sharing platforms. It’s important to involve stakeholders from different departments and levels of the organization to ensure diverse perspectives are considered. By implementing ideation strategies and fostering a culture of experimentation, companies can drive innovation and stay ahead of the curve. Building an experimentation culture is crucial for companies that want to innovate and stay ahead of the competition. It starts with creating an environment where experimentation is encouraged and failure is seen as an opportunity to learn and improve. Teams should be given the freedom to try new ideas and test them quickly using small-scale experiments. It’s important to collect and analyze data from these experiments to make informed decisions about what works and what doesn’t. Lucas Vos interviews Lucia van den Brink in this great AMA session.

Video

Audio

AI-Generated Transcript

Lucia van den Brink 0:00
But most case scenarios where testing client side, it is all like cookie based code gets injected, then it will impact your page speeds and I guess then it’s a trade off like, Hey, are we willing to trade a second at I don’t actually know how much it is, but

Lucia van den Brink 0:18
I’m sure we can look that up. But are we willing to trade a second for making better decisions and not shipping things blindly?

Lucas Vos 0:36
Welcome, welcome to the experiment nation podcast. I’m your host, I’m your guest host for today. And for those who don’t know me, I’m a lead zero, especially as RTL Netherlands, a broadcaster in the Netherlands. But enough about me. Let’s

Lucas Vos 0:53
talk about the most important person for today. Our guest is Lucia from Greek experimentation strategies as Spiro and do lots of things on site, talking several speaking gigs, and I think she can talk to you more about that in a few seconds. And we are going to talk about Yeah, setting up experimentation programs, stakeholder pressures, and many, many things more. So yeah, let’s get started. Michelle, welcome. Welcome. Good to have you here. It’s also a bit weird for us have we are both Dutch? Are we speaking in English to one another? So we have to shift a little bit. But yeah. It just to get started? You are working now as a consultant. And you have also worked in ours for DBG. Media. Yeah, what is the most different thing you have gotten in the past year to where you might have gotten used to?

Lucia van den Brink 1:57
So I guess switching from being in house to an agency is of course, yeah, it is very different doing experimentation. And maybe it’s, it’s good to give that I give a little bit of my background. So I’m really a crazy, crazy nerds, like I started building websites like really coding it. When I was 11 years old, and I had my own webshop, I had several websites, and I would just write a code myself. And at that time, there was no easy thing like Shopify or like WooCommerce. It was, it was actually quite hard. So I’m glad things have changed by now. But that’s where my love for websites and with the internet actually came from, and I guess I am yeah, I still have this love for the Internet and for the digital worlds. So and I guess it started when I was working for big companies like a telco company in the Netherlands. And later on, like the biggest news website in the Netherlands, which has like 8 million visitors. Very different from each other already there were both in house positions. But let’s say the first one was more ecommerce. And the second one was, how do you optimize a news website, right, so then it’s all about increasing the pages read by users, because in this case, they make money from the ads on the pages. So it’s very different. Also kind of the same. So there you already see, yeah, quite a difference between those two. And then, when going to work for an agency that, again, is very different. But what is really nice about it is that you have the opportunity to look into different organizations and how they do things. And it helps me a lot to zoom out and look to experimentation from like a higher level and see like, okay, there, they did it like this here, they did like that. And I think this is the way for example, or, you know, next time, I can recommend this or that. So I guess like having a lot of diverse experiences in organizations, but also, yeah, just being in house or agency or in one country or in multiple countries. It’s all beneficial to your experience. And I guess also the way you would be helping the websites perform.

Lucas Vos 4:22
All right, all right. So you have a lot a lot of different experience already. And also you said a there is a lot of difference already between a ecommerce kind of company and a let’s say journalists kind of company, a media company. And also now you’re at Spiro like surfacing a lot of different kind of organization in different countries. Are there any major different things between the different cultures that you have worked with, in terms of exploitation and zero?

Lucia van den Brink 4:59
Yeah, there are So, I guess, and that was also my next challenge. So I had experimented on the whole population of the Netherlands, I was quite sure about that at one point. And then I thought, okay, what can I do next, which was like, taking the step to do that, for the whole world. And then yes, you do find things that, for example, would always work on the Dutch, the Dutch markets, that that doesn’t work in other countries. And also the way you have different rules about things. So I remember I was optimizing for an Australian ecommerce website, also quite big at one and a half million visitors. But then they had special rules about how they could show the discounts and what they couldn’t do. And that was something I had no experience with. So I had to do a little bit of research and see like, Okay, how, how do Australian websites do this? And so I guess that’s something new to learn, like regulation stuff. And other other hand, you also have, like, what our main job is, is like influencing how visitors behave on your website, right, and possibly shaping or changing that. So in that sense, it’s also all about culture, and how people respond to the same websites is also different, based on which country people are in and what’s their background. So that’s super interesting to learn.

Lucas Vos 6:33
Yeah, I can imagine, do you have maybe a bit of a test set example of it so that people can relate to, like me? I’ve learned is sort of major difference between Yeah, I don’t know, an Australian something.

Lucia van den Brink 6:49
Well, I have noticed that, like, we Dutch people are very direct, which is an unique thing. And also kind of like, we don’t like hear key. So if you would be saying to me, like, Hey, I’m your boss, you have to do this. I’ll be like, I’m gonna Yeah, I don’t know, what’s this, but I’m not sure I’m gonna comply to this. But in other countries is not necessarily like that. An example of this, like on the website is that things to show safety or to say, like, hey, this thing, you often see a message like secure checkout, for example. I think those stuff, it doesn’t always work in the Netherlands, because people automatically think the opposite like a if they write it here, like I’m not trusting you, I only trust myself. So I don’t always Yeah, I haven’t seen much good results with that unless you do it more subtle. In the Netherlands.

Rommil Santiago 7:46
This is Rommil Santiago from Experiment Nation. Every week we share interviews with and conference sessions by our favorite conversion rate optimizers from around the world. So if you liked this video, smash that like button and consider subscribing, it helps us a bunch. Now back to the episode.

Lucia van den Brink 7:59
But then for other countries and for people, I guess also, not all countries are as far with the internet and using web technologies, as in Europe, so that’s also different. So in other countries that can work, you know, just saying like, Hey, this is safe, you’re good, and people will just trust it. So yeah, that’s that. That is an example. But of course, there are many subtleties in these. And of course, it it won’t be 100%. True. But yeah, to give another example. I’m also optimizing for a webshop currently that has a website in the Netherlands, and has a website in Germany. The sights are both the same, but the conversion rates is very different for the German sides. So that’s really been a process of okay, what’s how does the German audience perceive this? And why do they not convert? So that’s really interesting to do research about that. And to kind of learn about, okay, they look different to a website, and they trust things differently. So I won’t go into more detail, but at least you have a few examples, I guess.

Lucas Vos 9:13
Yeah, I can imagine and I can relate to a bit of personal experiences as well. You can have it even within the Dutch language area with your floors and Belgium, which can have the people over there can react differently to a website because they have a bit of a few different standards for E commerce as well. So yeah, I can I can imagine that. Besides all those differences, you’ve seen quite a lot of companies from inside or at least from a close distance. Are there any main pain points you have seen like hey, there’s more or less similarity that’s within CRO and getting experimentation progress. audits are running, that I experienced the same kind of things.

Lucia van den Brink 10:06
Yeah, I think they’re definitely like the same kind of problems. But I would say like, the pain points are different everywhere. So I know that some experiment, experimenters look for, you know, the perfect setting, and it needs to be like this. And we need to have all this data. And it’s, I know that some people are very idealist, and then you know, want to want it to be perfect, but in reality, everywhere, where you will go, there will be something that is, you know, not so easy or difficult. And, in the end, I think, what what I’ve learned working for a lot of different companies is also in a way being flexible with what is there. And, of course, you have to make sure you can trust the data and all that stuff. And but sometimes you don’t have all the data, for example. But you can still learn from certain things or, you know, certain setups. So I’ve also learned to be, you know, not so strict with saying, No, it should be this, this and this with more like, Okay, you have this so we can work with that in this way.

Lucas Vos 11:14
Okay, maybe? Yeah. But to maybe roughly translated is like, Hey, see, really, what is there? What is there to use and use whatever is usable? Right?

Lucia van den Brink 11:31
Yeah. And don’t, don’t get stuck on what is in there, because, of course, you’re gonna want to fix that. But on the meantime, you also want to keep your process running of experimentation, whatever that is. And maybe that’s only research or maybe that’s only, you know, one test a month. But I think keeping that process running is actually like kind of determines the health of your program and of your experimentation culture as well. Because if you are doing nothing, and if you have nothing, then you know, there is no experimentation at all. So it’s also important to keep things going, even though it’s not perfect. And even though you want to go like in a very different way. I think the first step is to not freeze and work with what is there and what is what is available. But there is a lot of issues, for example, data collection, tooling, cookies, Cookie messages, stopping your tests. I mean, there there can be so many things that go wrong. Yeah, they’re just endless.

Lucas Vos 12:29
Maybe there was something as a very perfect testing environment.

Lucia van den Brink 12:35
No, probably only in our heads like, oh, yeah, it would be like, yes, it would be perfect.

Lucas Vos 12:42
Yeah, I can imagine that. Yeah, good to see see how your your than your messages like, Hey, keep keep ongoing, if you are ever in doubt of him. There are some quality issues. But you say, hey, let’s keep going keep going and use keep improving from that point. Right?

Lucia van den Brink 13:06
Yeah, because often there is also like you, you do kind of have to prove yourself within an organization. And it doesn’t matter if you’re in house or if you’re like coming in for an agency or isn’t consultants. There will be always, I think there are often doubts about experimentation. And it’s good to have those doubts as well. Like, I think that’s only healthy to doubt what you’re doing, right. But yeah, sometimes you will need to prove to other people who are not only doing experimentation, what it is and what the value is. And in order to do that, you need to have something to show right. And that doesn’t mean like go run a B test with random data or like double data on your variant. Of course, you want to maintain a certain certain standard. Yeah, in there. Sure. But But I do always try to look for okay, if this is not possible, is there something else I can do? Or is there another way or Yeah, another research? Sometimes it doesn’t need to be a be testing as well, right?

Lucas Vos 14:10
Yeah, of course, of course, there are a lot of automatic methods to validate this kind of assumptions, or do at least a bit of research and see, what are people actually doing on your website. The speaker will say to you, because he said about how you have to convince people that you’re there and that your work is valuable, etc. This is my translation of it. Mmm, good for the listeners to understand is we while announcing this, this recording, we asked a bit of questions about hey, if you have any questions for Lucia, to answer, please shut down and putting them in the comments in a LinkedIn post. Although we didn’t get loads of them, we got some some valuable ones and one was from Tim barrels and you asked how to convince them Calls are the usefulness of experimentation. And I think that’s relates to the thing that you’re talking about No, maybe yeah, you can can get more into detail about it, what are the main areas, the main points that you need to bring across?

Lucia van den Brink 15:15
Again, I think there’s not one way of doing this, probably there’s like tons of ways of doing this. But of course, what helps is if you can tie it to monetary value, like, and that’s the easiest in E commerce, right? You know, like, hey, there’s a conversion, that means so much money for you, and so much bent on experimentation, and you can enhance, calculate ROI, that kind of stuff, you can also estimate the value of winning test over months, or you know, the declining value, if the value will be less, for example. So that’s one way you have you have the money cards to play. I think there are more cards to play here. Because it’s also about how you make decisions in a way. So yeah, yeah. And it, what we’re going away from is we’re going away from that opinionated boss that tells you how to do stuff. And we want to go towards like, hey, we have this evidence. So we believe that this works. And here, look, we also tested it, so we have proven an indication that it actually works. So that is the I think a second thing, like making better decisions and making, you know, doing better stuff without just chipping it. So that’s also one car to play. And then there is also, I mean, there’s lots of ways to go about this. But the there is also a part of it, which is kind of democratization. Which means that maybe it ties together with that decision making process. But in fact, everyone in a company has a different background, right? Different values. Maybe one is a UX researcher, maybe one is a developer. And I found in my programs, and especially the bigger ones, the more people you will involve into experimentation and ideas, the more diverse growth opportunities you will also find. So for example, at nudata now, which is that big news websites I worked for. There, we also got test ideas from developers saying like, Hey, I’m not sure about if I should load these A B test. Or sorry, if I should load this page immediately, like only once, or like slowly in steps while the user scrolls. So that was a test from his idea and totally valid, it was a big winner, even when we learned what was actually better. But it’s just an example to illustrate, like, using all the people in your company can actually be beneficial to, you know, to your company, as well. So those are at least three guards that I would play at. But I’m sure there are more. And I’m sure there are people better in like the political game of it. Yeah.

Lucas Vos 18:18
Yeah, it’s all definitely the politics coming into blame. Used to take money. And there are a why about decision making, making better decisions, and democracy. And I wanted to dive into at least one of them and the money part because I think lots of experimenters, zero people are maybe struggling or having a challenge. So hey, how do I calculate estimated ROI of my experimentation program? How do I get valid numbers or at least something that I can work with and which is referred to as trophy for? Yeah, whoever it concerns? So yeah. Can you maybe help us a little bit like, hey, how to approach that? Or how have you done it in the past, which was maybe a success or something?

Lucia van den Brink 19:15
Yeah, this also depends on you know, what background you’re from, like I said, ecommerce is just a little bit more straightforward in that sense. Because you can just have, you can have your revenue. Or maybe What’s better is like the profit from that revenue, and just put that next to all the costs you’re making for experimentation. But that’s like the global argument for experimentation, money wise, right? But you can also do that smaller and Bert tests. And I know that there are calculators that’s, that helps you with that. You can put in like average order value, and then depending on your traffic and your uplift, it will say like, Hey, this test will give you an N then it gives you an estimate of dollars euros can come out of that. But the general idea is that also the effects of test will get less like after implementation. So usually how I do that is I take, I take the uplift, and I think every month I add that uplift. So, cumulative, and then subscript substract, like 25% of the uplift to, you know, see, actually, that affects the creasing there. But that might not that’s like the worst one of the worst case scenarios because it might even, you know, cause an uplift. And that’s something that we often don’t measure in experimentation we can, if you do server side testing, and you do things for longer periods, I guess, maybe that’s another topic. Because we were talking about, yeah, how do you? How do you convince that the money arguments, right? How do you win that over. And, for example, if you’re optimizing for a lead generation website, so for example, you’re you have a website, and you’re selling software. And you have like, the main goal is people to fill in a form and click the Get demo. And then later, a sales team will call them. And they also have a kind of a conversion rates, maybe they close like 20% of the deals. So then you also need to think like, hey, the value of the click of that button is not the actual value of our product, Nate, sorry, I said needed means knowing that, you know, know, the actual value of the click of the button is lower, because the team afterwards has only a 20% closing rate. So actually, you have to multiply, you know, the closing rate with the average lifetime value that you get from that, and then you have the actual value. So it also depends on what you’re working with. I mean, don’t get me started on the news website with ads. And it’s it will get more complicated. And I guess if people need help, just send me a message on LinkedIn to help out. Yeah,

Lucas Vos 22:16
Google go. So first of all, like, rule of thumb thing is like, every shell you open up live, you get I get like you calculated with average kind of traffic, and then you let it decline over time. That’s, that’s one of the things that you say. Yeah. And it can more be more complicated when it’s different than just straightforward ecommerce. Well, we can relate to that. Yeah, you have to take multiple things into account. And as always depends that’s typically see row answer, right.

Lucia van den Brink 22:49
Yeah, yeah, it is. It’s a great answer, I guess. Yeah.

Lucas Vos 22:53
Yeah. Yeah. So we can all correct. All kinds of question. And the answer is always It depends, like 42 or something.

Lucia van den Brink 23:03
Well, I liked, like, the world is not so black and white. Yeah. Yeah. Sorry.

Lucas Vos 23:10
Curious, and it makes it also more fun. And you get, it’s also the area where you can make a lot of differences in independence. And then what’s then and then you can get into a real conversation, like we have also now. Just also wondering, because before, yeah, while preparing this, we had also a bit already about talks about developers, you have most zero people and experimenters work with developers. And yeah, it can be sometimes really fun, but mostly is. And sometimes we have also a bit of a challenge, because there is also there are also people sometimes to convince because they have to do actual work for you. Or at least with you. So yeah, you had some experience I learned about developers not so fond of experimentation because of PageSpeed. So maybe you can elaborate a bit more on that. And also, yeah, take us along how you maybe tackle that issue?

Lucia van den Brink 24:16
Yeah, so they’re the arguments from development side here is valid, right? If you implement VW o or Google Optimize, it will slow down your site’s And worst of all, it will inject code into their perfect codes. And like, totally mess up the website that they have built. So there are multiple things going on here that developers don’t like.

Lucas Vos 24:37
Definitely.

Lucia van den Brink 24:39
Yeah, yeah. So So yeah, where to start? I guess the arguments for site speed is just I cannot do anything else than agree. Yes, it does impact site speeds if you do it client side. Let’s say that as well, because you can also do server side testing, which is different, but most case scenarios we’re testing client side, it is all like cookie based code gets injected, then it will impact your page beats. And I guess then it’s the trade off like, Hey, are we willing to trade a second at? I don’t actually know how much it is. But I’m sure we can look that up. But are we willing to trade at second for making better decisions and not shipping things blindly

Lucas Vos 25:23
stick to the development part of bit. My personal experience is also that there can be also complaints about security, and that you’re testing tools not secure. So yeah. Insecurity. For sure. It’s something that you want to you want to be don’t want to be insecure, or have a website, which is hacker vulnerable. Yeah, but just how would you approach that, if you got that kind of comment? Wouldn’t relate to.

Lucia van den Brink 26:00
It’s something I haven’t heard before. But I can imagine this to be an issue. And I guess the solution is also giving the power to the development and security team in a sense that, you know, we don’t have to use a testing tool, we can just build our own. And we can do it server side. And you can manage all that. So I think that will be the one of the solutions here. Because I’m not an expert in like security, around testing tools. But that would definitely be something I would suggest, because that would be even better for your experimentation program, potentially. So that’s, that’s one road to go. And I guess another road could be like investigating if that security is really a problem, yes or no? How did you go about it?

Lucas Vos 26:51
To summarize it? We had a discussion, I think, almost two years ago. And by the end of it, it was really the trade off? Hey, are we going to build our own testing tool? Yeah, but that’s product that you also need to maintain? And are you willing to, to support it, etc? And to invest time in it? Or should we continue maybe with this tool or another one? But are there any things we can do to really improve security? And, you know, when we got one of the most Yeah, strong critic guests onboard on this one, like he was willing to cooperate and to improve stuff. He was part of the project team, which eventually, yeah, redesigned the whole setup, like we really didn’t have another kind of setup, because we’re using suspect it’s a little bit different than using client side tools, like Fe WL. And it was a bit like service server configuration and lots of technical stuff. I don’t understand anymore. But yeah, at least the most critical developers were talking to the developers of the testing tool, and some other ones. And they got into a group and there had, yeah, they’re agreed on a certain kind of solution that was implemented and still running today. So yeah, and, and the discussion, what is more or less, has became silent about that. Although, yeah, critique is everywhere. Yeah, that’s, that’s, that’s our daily life. Right?

Lucia van den Brink 28:30
Yeah. Yeah. That’s also good solution. Like, okay, maybe this is going above our heads, like, we cannot understand the depth of all the security stuff. Let’s, you know, the DevOps and security team just work together. I think that’s a great solution.

Lucas Vos 28:45
Yeah, yeah. I think in the end, it was a great, great solution. The road to there was a bit bumpy, but yeah, okay, we survived. And we are still running. So that’s, that’s cool. Another thing, why the art of not forcing, I didn’t understand it before. And so maybe you can introduce it a bit and why it is relevant for our field.

Lucia van den Brink 29:09
Yeah, I guess this has to do with a little bit of letting go like, and now it sounds really maybe spiritual in a way. But for example, I have a client now, and they are quite small, but they want to go really fast. So they want to do like, they already want to do the user test of to a B test that we still need to run. So we could set that up, right, we could link one A B test to another AB does have the user testing in there, but it’s a lot of hassle. So it’s kind of like swimming upstream. Instead, we could also choose like, Hey, can also wait a month and then we do the user test and then it’s actually live and then we can you know, proceed to learn again, which is swimming more like you know, a little bit more with the flow and I guess it’s good sometimes to see Eat a difference in. And this is also part of decision making, like, are we really willing to swim upstream here? Or do we let it go for a bit and swim with the flow later. And this is just one example. Because there are many more like, for example, sometimes there is a really big rebuild of websites or web shops, for whatever reason that might be. And of course, it’s great to help out there with experimentation and test a new platform, if possible. But sometimes that also means like, developers are building the codes and they don’t have time for experimentation. And if you would be pushing on experimentation on that, you know, maybe it’s just a quarter that they’re doing that, but if you’re would be pushing on experimentation that quarter, you will be like, really going against the flow, I guess. So. It’s also good for our work to see sometimes the bigger the bigger picture of like, okay, now. And it, I have seen this in house. So I will be sitting there like, oh, okay, so that means I won’t be able to run experiments anymore. Which is really weird, right? Because you’re an experimenter and you just want to run you just want to go, you just want to keep that process healthy. But then it’s also like, okay, yeah, I can go against it. I can try to push, or I could let it go and do user research or something else. So this is yeah, I found a thing in Chinese called movie. I don’t know how to pronounce it. Sorry, if I’m pronouncing it wrong for other people. Listening from China, maybe I’m even interpret, interpreting it wrong. But I hope I explained it a little bit like, yeah, sometimes experimentation is just like swimming of the stream or with the stream. And there, you know, there are moments in if you need to run a test the decision making or you know, how to scale your program. And and sometimes it’s all about also paying attention to that, that flow and how that, you know, if that could be better, maybe in a few months, it would be easier if it would make more sense. Or that you’re going to spend all your energy swimming upstream. Does that make any sense?

Lucas Vos 32:20
Yeah, it does make any sense in a way that you’re what you already said, Okay, use the things that are already there. So there’s all the what’s already there. There’s also the direction to pay the whole company and at least your departments or business units is running. So that’s, that makes sense. Like a you have to use it and not try to really change the fundamentals of it. Yeah, although it can be pretty pretty hard when someone say, yeah, for the coming months, we’re not going to run an any experiments. Yeah, I think most of us could freak out, at some point, like, You’re killing my job or something. Yeah. And that’s, that’s something you you have to deal with that something’s Yeah, things outside of your influence can happen. So that’s Yeah, I think a good thing that we have to learn. And I know I’m not that good in that kind of thing. So it’s also very, sometimes a lesson for me, but yeah, in a certain way, it can help.

Lucia van den Brink 33:21
I think we as optimizers, we get very results driven, right. We want to have results, we want to have results for others. So it’s like something that becomes almost natural. But it’s also a question to ourselves, like, Hey, if you’re not delivering a B test right now, does that mean you are bad experimenters? Does that mean your culture of experimentation is not there? Or is it just temporary? And should you just let it go for a bit? So that’s also something that’s, yeah, it sucks. But sometimes it’s good to acknowledge.

Lucas Vos 33:51
Exactly, although I’m realizing you said also that you have to get philosophy that you have to get running. And if you’re not running, that’s it. That’s not. Yeah. Not compatible with each other at some point. So yeah, how do you relate those things to one another? Because yeah, if you are really on running, they have a bit of a standstill.

Lucia van den Brink 34:19
Yeah, true. So yeah, ideally, you always want to run something, right. But sometimes doing research is also running experimentation in a way. Not everything needs to be a B testing, of course. Yeah, but I agree. Like, of course, we like to see a healthy program with experimentation and what learnings. But sometimes, that’s just not the case. And then, you know, what, what are you going to do? How are you going to be flexible with it and how you still going to do something useful with your time?

Lucas Vos 34:54
Yeah, exactly. Exactly. Yeah. At some point that you can have that and we have some numbers here as well. And we try to use to move on to another maybe another part where the opportunities are more there. And, and also talk together, if people are busy with a rebuild or something. Most of the time, you can help them as well like, Hey, we’re just in doubt about these kind of elements. And then maybe we can be tested or help them with this kind of research to maybe pre validate something. Maybe also using tools like usability help or something. That’s also something we we try to do sometimes. So yeah. Before starting the recording, where we will already be talking, we also talked a bit about ideation, how to get the best ideas, and because I think, yeah, if you listeners will have these kinds of challenges, maybe like, hey, how to get maybe better ideas, or more ideas, or more ideas from maybe more stakeholders or something like, Hey, can you talk us through like, Hey, what are your thoughts about it? Yeah, so

Lucia van den Brink 36:10
I have this nice combination, that I also write literary novels, which is a very creative thing to do. But that really helped me develop my thoughts about ideation and how creativity works. So something we usually know in the cero, experimentation space is about system one and system system two in the brain, I think this is yeah, this is just a model from Daniel Kahneman. There are also people that explain it with the original brain and like kind of the new ring, whatever you call, it doesn’t matter. But it’s important to understand the interplay here, between those, well, between the two, because creativity is coming more from that system, one, that automatic brain that is very good at associating, it’s very good at, you know, coming up with ideas, the creativity part, it can solve two plus two for you in no time very quick. So then it’s actually the part that comes up with ideas and can associate also very well. But it doesn’t work well, when pushed or under under pressure, because that’s when usually the system two kicks in. So that’s your ratio. And your ratio can do you know, a lot of cool stuff. I mean, you can you can solve you can, maybe you can calculate ROI for your program, you might have to think you can judge whether or not your maybe your stakeholders gonna find your test ideas too stupid or not. You can judge with it so and do more complex stuff. But that part of your brain is sometimes in the way of the creative brain. So what I often see happening is that I see people having an idea, and then they tell the idea, and they say like, okay, my idea is to, I don’t know, put products on the homepage, if there aren’t any Indigo is like a comb focused. And then they say like, oh, no, that’s a stupid idea. Because sales doesn’t like that. And this doesn’t like that. And yeah, I don’t know how to do it development wise. But that means you’re already killing the hypothesis of like having products there would work or at least a prediction that you have for that. So in that sense, sometimes the ratio and having, you know, thinking rationally too fast, can really block you in ideas, because it might still be a valid idea, right? You just, sometimes we really queue ideas too fast. So what I always recommend is also kind of give you a ratio, its own space and time and come back later to do you know, create a hypothesis, do prioritization, all that stuff that we have, but also leave a space there to be creative and Associates maybe based on the data that you have, or maybe based on competitors, you know, whatever input that is, for you to come up with a creative solution. And then there is also another myth that you need to come up with one big idea. And that idea, you know, is going to fix everything. And often what that does is it puts a lot of pressure on you. So you immediately start rationalizing like, oh, is this the big idea or it’s not? And also, if you have only one idea, then man, it really has to be wrong. Great idea. And that’s just not something that we’re often capable at, I guess, there there is also research done. So to ideation, whether you should focus on quality or quantity. The answer is quantity. So if you have, let’s say, a bucket of 100 ideas, then your one great idea it’s just going to be in there, maybe there’s a load of crap in there doesn’t matter. You will later filled or that I was with your ratio, and with all your tools and prioritization, but just to kind of give that space for your creative brain is very important. And it’s something that sometimes it seems like we have lost this a little bit to just, you know, we recognize it when standing in the shower or when having a walk. So that is when you’re creative. Like that’s, that’s reptile brain is really working, but we you don’t need to stand in the shower, you can already facilitate that just by you know, not judging and not. Yeah, coming up with rationalizations. So that’s, that’s a little bit of ideation that I like to teach to people. Also, you know, there is lots of stuff to talk about, like, also, should you do brainstorming in a group yes or no, and how you should facilitate it. But I guess for now, like, this is kind of the base of it’s like, that creativity and the ratio or like, I don’t, I don’t know if you can say that, like water and fire to each other, like they, they don’t work together well. And the second thing is like having one big idea, it’s just really hard. It’s better if you come up with hundreds and then you know, like, your one big idea is going to be in there.

Lucas Vos 41:20
Yeah, exactly. That Yeah. Personally, I tend to focus also on getting more more and more of these, like the quantity of of quality, because yeah, you can trust all of that you have really good ones in there. What when you have lots of them? So yeah, I can imagine that. Are you talking about? We’re talking about touching a bit about brainstorming? And should you do it or not? What’s your answer to that? We have any recommendation? I mean, depends. How would you do that? How would you organize right?

Lucia van den Brink 41:56
Now, so based on that, knowledge of like creativity and ratio don’t go well together. That also means that in a group, we’re also constantly kind of judging or like thinking things about others. So that is something that is always at play in groups, and I think it’s called, you’re conforming to the group in a way. So if you put a group of people together, what they will come up with is one good idea. That is a good idea, according to the whole group, if you put them aside, so usually what I do is I give people like their own 10 minutes or 15 minutes to come up with ideas, and that can be in a brainstorming session. I mean, I could sit next to you and work on my own right. But then at least people you know, go into their own knowledge, their own background, and they say like yeah, for me, this would be this five things will be great ideas. And then for example, if a developer sitting next to me, then he would come up with five totally different ideas. But you know, it doesn’t mean they’re not valid or doesn’t mean they’re not good for example. So in that way, you can facilitate a little space and time for for each to have you know, to to just think to themselves what would be great ideas and then later you can come to the group and like maybe make little groups of like three people and they all present their ideas and pick out together the three best ones and that’s when you’re going to use ratio right that’s when you’re going to think like oh, this is impossible to develop or this this will work or this won’t work and then you know you’re you are already filtering together and using that judgments but I would say before that’s give everyone space and time to think and try to keep that judging part of the brain on on a distance.

Lucas Vos 43:52
Okay, yeah, so you are a bit of fan of brainstorming but keep the group pressure a bit out of his league as well that’s it will be your answer and also the creativity part sometimes we do hear brainstorm and we do kind of warm up sessions like okay get get rid of maybe frustrations from the past meeting or something out of the out of the room. Would you recommend that or wouldn’t you’re with him and so

Lucia van den Brink 44:32
yeah, that’s that’s can be a great idea. Like you’re coming into the room with us. Yeah, kind of a setting right and the things you have experienced and I’m not sure how that works, actually. So next time I see you I want to do that

Lucas Vos 44:49
bye bye. Know because maybe someone listens to everything what what kind of things yet think of creativity games like you have kind of a puzzle or a I have less finger five minutes like how many names of fruits can you come up with? Or how would you draw a bridge? How would you draw the person next to you or something like that really a difference? Tune out of what you have done before, really is really something different. And also maybe different from warmer just to get rid of a reset in your brain. That’s that’s the main goal. Yeah, that’s nice. Personally, I’m pretty much fan of it. Because it especially when you do a brainstorm more in the second half of the day of the working day that people Yeah. Yeah, have something with them, which the experience of the day already, and it can be positive or negative. But it can bias the whole thing altogether.

Lucia van den Brink 45:58
Yeah, because someone’s brain can be in the meeting before i They are that thoughts? I mean, yeah, I see.

Lucas Vos 46:05
Yeah. Maybe they had a negative message or negative surprise. And they they had their, their their negative feelings can come into the room, because we’re all humans, which so yeah, we can walk through the door, and it’s not really out of the window. In one second,

Lucia van den Brink 46:24
I think you should share nosa those exercises because like it, just put it, put it on LinkedIn or something?

Lucas Vos 46:31
Yeah, well, maybe good idea. Maybe good idea. So there are people you might might see something. Just one question, I think it’s also good to have because when you do a lot of A B testing, where you a lot of experiments, you, you build, you get a lot of learnings. And yeah, we have already talked about a bit of rationality, and also maybe the limited rationality that we have the weekends, we have not a unlimited kind of memory. So how to keep track of all the learnings and build onto that. So it was a question from variabel. Smart. How would you do that? How to keep track of all the learnings that you have and build on? On top of it?

Lucia van den Brink 47:22
Yeah, I guess the standard nowadays, just kind of using air table. But that sounds like a very easy answer. But I guess even if you use air table, that doesn’t mean you automatically are very able to find your test or Yeah, you do have to put some work into categorizing tests, maybe based on strategies teams, or maybe even a cycle psychological tactics that you use. So you know, if you use air table and you, you, you put your test reports in there. It’s good to be a little bit more detailed. It can be also just like a this this was a mobile or this this was a winner, yes or no. But these are all variables that you can link as information to your test, which can later help you to build like a dashboard of all the winners on mobile, for example, on the social proof strategy, or on fixing something that you might have found in research. You can also link it back through your research, of course, which is also great. But for some people airtable is too far away or seems too difficult. And you can always start with just a Google Sheets and just you know, list your test and see if it’s when yes or no. I mean, it can start as easy as that. If you’re just starting with doing zero experimentation, that’s also fine. But later on, you’re gonna want to have something that will help you find what you’re looking for or like dig in to your own memory, but then you know, the memory of all the tests you have done. Yeah. How do

Lucas Vos 49:02
we use now airtable. We started with that three years ago. And it also was based on a Google Sheets, which was not not that complete. And we step by step, build it, build it. And we got also to the point and maybe that’s also good to share. We felt that we sent people into the woods, like and they were really in danger of being lost in there. So how are we maybe to give a bit of context now we have done more than 300 AP tests in the past, I think three or four years. On up to five. The majority is in the last three years. So yeah, that’s that’s a big list. You have to wander through if you just have a list with no context. So now airtable has a new functionality and it’s called interfaces and now we have built that On top of it, like a bit of browsing thing, like search and browse for filters, and also a page where they can follow the current experiments around because we have now multiple teams, running experiments and yeah, people want to have keep track of, hey, what’s running right now? And I’m just curious. Yeah, so yeah, the first learning because we now doing it for two free months are pretty positive. We’ll have to learn also how to use to, for ourselves, but also for our stakeholders who want to keep track of all the things that are happening because yeah, you, you want don’t want to have to people are asking for the things they did there and are not able to find because there there have to just dig into all kinds of data. And yeah, that’s, that’s the thing that you don’t want to have, and also for yourself. It’s also a bit of a challenge that we had. Before. RTL, I worked at NSA, in one of the major newspapers in the Netherlands, part of media house and big newspaper company in Netherlands in Belgium. And we also did, and there was also a nice, we had a close, just WordPress kind of environment. Like everyone just creating lines, it was a blog posting thing. You you can create, like a blog post with a template. Very simple. And post over there. And it was easy, awesome. Also pretty easy to do to use. And you can categorize it and yeah, it was also nice. back then. So

Lucia van den Brink 51:47
yeah, smart. Yeah. And then you can just search in everything. You know, that’s written in there. Great. And yeah, no, that’s that also. Yeah, like air table wise, you can do so many things like it’s Spiro there are like automations in there that you can immediately create a new test from an idea somewhere, and it just automatically pops up a document or what else? What else do we do like send messages to slack like, hey, this, this test is ending, or we also build a dashboard for clients. So they can see like, this is my to do list as well. For example, the client has a to do list like they have the check stuff. So lots of cool stuff in there. That you can do, potentially for sure.

Lucas Vos 52:34
For the for the for the listeners out there of questioning, hey, do they have stocks in a table? At least the host doesn’t have. Okay. We’re not sponsored by Atal at least yet, man.

Lucia van den Brink 52:53
Sponsored by like enthusiasm most of the time.

Lucas Vos 52:57
Exactly. Exactly that. But now, to cook to conclude on the airtable pipe party. It’s I think it’s a really great and powerful tool if you set it up correctly. Yeah, and for me, for us, what works is also documenting screenshots, like okay, how do they look? Have I? What do we do or, because if you wander through again, and then see the ABX testing that you did three years ago, that’s consulted. Sometimes we also a bit of a spar to get new IDs already. So hey, back that we did that. And it was already maybe a winner. We I didn’t record it anymore. But hey, it’s maybe something that we can use. Because we tend to do something else which might interfere with that, that learning that dance. So yeah.

Lucia van den Brink 53:55
Yeah, it’s crazy. I mean, images, say more than 1000 words, right? Is what people say. It’s just like, easy. Don’t need to read and it’s perfect. Yeah,

Lucas Vos 54:03
exactly. Exactly. Yeah, I think maybe to conclude, for now, maybe you have something something untouched, you might want to share with the listeners, maybe some final words.

Lucia van den Brink 54:23
No, no final words like that to inspire you. And I guess this whole talk is already there. But what I would like to say is, I’m very active on LinkedIn these days. I don’t know what happened. I didn’t used to be. But if you ever have any question or just, you know, want someone to talk to about zero or experimentation, just send a message. It’s also what to do to a lot of people that I look up to, or at least you know, just ask a lot of questions on LinkedIn and I hope other people will also find me because I will be happy to help out. And I know a lot of people in the Xero community See are always willing to help out. So that’s really great about our Yeah, the community that is there. So hit me up on LinkedIn, I guess. And any questions. I’m also open to that.

Lucas Vos 55:11
All right, yeah. Thanks. Great. Final comments and I think, a great invitation for all. Yeah. To conclude, I want to thank all listeners who have listened to this podcast. I hope it was inspiring and helped you somehow. And, yeah, there was it so hopefully, some next time. Thank you.

If you liked this post, signup for Experiment Nation’s newsletter to receive more great interviews like this, memes, editorials, and conference sessions in your inbox: https://bit.ly/3HOKCTK

---