Product Experimentation: A chat with Moshe Mikanovsky about building competency and fluency with data (Part 2) Experiment Nation

Home / Product Experimentation / Product Experimentation: A chat with Moshe Mikanovsky about building competency and fluency with data (Part 2)

Podcast transcript

Rommil Santiago 0:01
From Experiment Nation, I’m Rommil, and this is Product Experimentation. Product Managers everywhere are leveraging the power of experimentation to solve customer problems, and build better products. Learn from seasoned PMs and find out how to up your product management game with the latest experimentation strategies.

Siddharth Taneja 0:31
Hi, welcome, everybody. I’m Sid and I’m joined by my co-host, Jaya.

Jaya Gupta 0:35
Hi, guys.

Siddharth Taneja 0:35
This is our first session of PM chat, where we invite experienced product managers to share their experience about building products. Every week, we will cover a broad theme. And this week’s theme is impact of data while building and managing products. For those of you who have been following us on our previous sessions. The last session we had was also about the impact of data and how can different teams bring in data as one of the key drivers while building different products. Over to Moshe, now. Just a little bit about Moshe from what I have spoken with the great man is that he’s a product veteran with over 30 years of experience building and helping companies build lean software products, but I’m going to leave the introduction to Moshe, himself.

Moshe Mikanovsky 1:20
Thank you so much Sid and Jaya. And thank you for having me today, on this podcast. I’m quite excited to be here with you and share my experience as a product manager. So yeah, as you said, said, I’ve been building software products for 30 years, which shows my age, I guess, and, and 20 of those, I started my product journey from as a software developer, so the for the 1st 20 years, I was doing engineering, so I was in the engineering and doing many different positions. And even back then I didn’t have really product managers and many of those positions, I was like a lone developer working directly with my clients and my users to understand what they need. And that really gave me a lot of this hands on experience with directly understanding what the problems that they have, and then try to create solutions for them. In the last 10 years, I’m been focusing just on product management, because that’s really where I love. And I talked about it. So and that’s also how we met.

Jaya Gupta 2:30
I can kind of imagine a child project prodigy here. So don’t worry.

Siddharth Taneja 2:37
Yeah, well, that’s really an interesting journey. Moshe, I know, we spoke about products about our experience, you know, throughout the whole lifecycle, but it’s, it’s amazing to see that, you know, you were there when product management was not even a field. Right? Yes. And you’ve seen it evolve, which is pretty amazing. Um, if you compare the products that were being built about, say, a decade ago, right, and compare them with what the way it’s being built now. Could you draw some parallels in terms of what the listeners could take that?

Moshe Mikanovsky 3:12
Oh, that’s, that’s very interesting. The, I’m not sure about the parallels more as in who drove those products? Was it really driven by true design and empathy to the users? Or was it driven by the engineers that build them? Right? Because we didn’t have that skill set. But we still have to build something. And also back then in and I’m thinking also, you know, 30 years ago, and 20 years ago, and 10 years ago, which were each one of them was taken by itself and was different from what it is today. The the tools were not there. So we didn’t have like the internet, right. And then we got the internet, but it was in the very early stages that you can’t really do much with it. So you need to head like a flash. To get out. It was really useful on the internet, or you embed an entire one of my first internet experiences, I embedded an entire desktop app inside the browser. And that was it for our life. Oh, we’re on the internet. So so the tools were not there. And and the knowledge of how to do it. And then also the the discipline like the UX UI design discipline. And I’m not even talking about product management this, right. So this evolved a lot in in the past decades. And in the last 10 years, I would say the understanding of what product management is all about is evolving. I’m still getting questions about what is product management role about? Do we even need you here and which you know, I don’t take it personally I because I understand that it is something that sometimes it’s hard to understand, right, we’re at work like Marty Cagan’s books, Inspired and Empowered, perhaps we have much better understanding of what the need is for and what great companies are doing out there, and what what is really needed there. And the same thing also with many other books and other frameworks that have been developed. And I think one of the reasons there are so many of them these days is because there was really a vacuum that there was nothing there and everyone needed to create something to be able to get the work done. So that’s why we see like, all these books coming out now and all these frameworks.

Jaya Gupta 5:37
Yeah, Moshe, I think, I totally agree. Even if just remembering 10 years back with my experience, it’s always been a thing to understand product management. It’s just, I still remember the first week when I joined the product team. My manager had actually asked me if I was interested in creating a product manager guide. While I because nobody was interested, nobody wanted to do it. And I’m like, Oh, are you giving me junk homework? And now I realized that probably could have been very helpful. What we ended up having is like, some checklists to make sure you do these things so that you had a sound product launch. But oh, my God not having internet. Like That reminds me of all the conference calls, and and meetings face to face that would drain your energy out. And sure, I don’t think much has changed from the meetings and stuff. But I think people appreciate that. You need some time to look at insights, digest and strategize and make some sense out of that to create a roadmap and execution plan.

Moshe Mikanovsky 6:42
Yeah, there’s definitely a lot there that needs to be learned and still, I mean, even even with all the experience that I have, I still feel that there is a lot for me to learn. And that’s why I love these discussions, because I always learn from them something new. Yeah. And, and it’s up to us really to do that, right. In most cases, no one will just feed it to us

Siddharth Taneja 7:03
That that is such an awesome sentiment, to be honest with, you know, the every conversation, you’re right, like, like such is such a great learning platform. Right. And to the listeners just to steer us in the direction of the theme on I know, Moshe, I’ve been following your posts a lot on LinkedIn. And I’ve been seeing that you are indulging in a lot of these certifications around these data analytical tools. Right? If I’m wrong, the last one that you had was Mixpanel. Right. And Jay and I were talking about this right? How data can be influenced to make more informed decisions to building the right product? Yes. Do you mind sharing your inspirations to you know, aligning yourself with a couple of these platforms and how your experience has been?

Moshe Mikanovsky 7:48
Yeah, for sure. So my journey with that started really from my engineering background, because I always were very close to the database and what the data said and what information I have been there. So before some of the these platform existed, I was always looking into the database querying, you know, how many users I got, and how many transactions they created and interactions and stuff like that. But it was always very limited, or and then, of course, Google Analytics came to be and we implemented that in several instances, but it still didn’t give me the right information about how people are using the system. Correct? Because it was quite, I think, until these days, I’m not I’m not really using it this day, so much because the other systems exist. But I think it’s better for a marketing website, where you want to see how people came to your site and how they went through and how you, you will convert them. Right? in applications where the applications are very, sometimes very complicated. And I’m, I mostly deal with b2b, b2c products that are quite complex. And there is a lot of interactions and a lot of different things going on. It’s very hard to understand from Google Analytics, you know, what’s going on, right? And, and how the users are using the system. Right? And especially when I want to build, I want to build products that customers love. So I want to really understand their sentiments towards the product, not only in what they say, but in what they what they do is really the data. So about probably seven or eight years ago, maybe more seven years ago, I was at a conference and Pendo was given the presentation over there there were quite early in my days, maybe it was actually more like six years ago because I don’t even think they existed seven years ago so it’s probably I’m just kind of like time just flies you know, so now I’m not even sure when it happened. But what what when when I saw what they’re doing I was like, this was like a ha moment and a wow moment for me because They were able to identify features. So it wasn’t just URLs and pages like Google Analytics. But it was actual feature. And they were able to do to show me, this is your feature in your app. And here, you can go and set it up on your system with WYSIWYG way to do that, and the idea really appealed to me. And I looked, you know, more into it. And it took me about a year to convince the company where I worked for to, to get it because, you know, some of the systems are not cheap. And I looked also into other systems like mixpanel. And I looked into heap and amplitude and whatever, at the end, we chose Pendo. And actually, I was very happy with that decision. And I brought it into some other systems. But what it allows us to do was to have really an almost like a magnifying glass into our blind spots, to see how users are actually using the system. And from that, we were able to set up different things. So for example, when we wanted to define a new feature, and we wanted to know if that feature is going to work or not, we would set up the success criteria right up front. And, and then based on the performance of how the, you know, what we saw in Pendo, the people are using it, we would like, Oh, we were successful or not successful. And now we have to do some work to to make a change. Right. So that’s just one example of that.

Siddharth Taneja 11:36
Yeah, that’s awesome. Man. I really like the analogy that you used. The data is really like a magnifying glass into what your users really believe. Right? One of the things that I felt in a product that I was building was, we did not have any of these tools at our disposal, we had no Google Analytics, or any other tool that could help us analyze the different user insights. So and we were also tight on resources and budget. And since this was a digital product, me and my team, we went ahead and literally created something from scratch. And I can see a lot of battles with your experience Moshe, because when we were building out this digital web product, we could actually align and monitor every UI element that we want to do on that interface. And the power that it delivered was insane, right? We could successfully identify which micro interactions were failing. We couldn’t identify at what stage was the drop out of the bounce rate the most for the client, right? Yes. And I feel that was such a great way to know more about the users themselves. What about you, Jaya, how is your experience been while you were building the digital products? Yeah, I

Jaya Gupta 12:59
I find I first gravitated towards or minus the whole mobile app device battles iOS and Android, let’s put that aside, I’m not going to turn this into a more conversation. Once you get past the distribution of users, I found myself gravitating towards the flow charts. So that I can understand where our users going from the homepage, in a previous experience that we created, because there was a new flow within a flow. So let’s say you’re, you’re completing a task. And within that task is a sub task that you could choose to create or reuse something that’s part of that task. As a result of offering that and building that sub task, it represented itself as a drop off to the entire task. And when we rebuilt that experience, we built it in a way that it was integrated so that when you returned, as a result of completing that subtasks, you returned back to that main task. So then we could say it’s actually not a drop off, we’ve actually promoted or continued to have a completion rate of acts as opposed to like, you know, 80% of users dropping off the entire flow. So that was just an example of looking at a single flow out of the entire flow chart, but it’s very telling to where users spend a lot of their time and what’s what’s useful to them.

Siddharth Taneja 14:32
Yeah, that’s, that’s a really interesting point. For all the listeners out there, are there any recommendations on how they can start their journey into exploring data as one of the tools that they can use in the product management? And also, what’s the best way to identify which of these bazillion amount of platforms really helps them to solve their product needs?

Moshe Mikanovsky 14:57
Yeah, that’s a very good question. So I’m also getting this question in some other perspective, because some product managers don’t come with technical background like engineering background. So I’m being asked like what they have to learn in order to do that. And it’s always good to have a basic understanding of what they use, how they structure and how you can get data. So a lot of things that product managers can do is like, you know, just learn SQL language. So you can actually, you know, get data from the database, because that’s the first place where you will want to look into the into data, sometimes if you don’t have anything available for you, right? Then there are tools that are just can create, you know, different types of reports like tableau, and things like that. And there are more tools available out there, right. And you will need some type of, you know, integration between your different databases to that tool. And usually he will have some data person in your organization like a bi team or something like that, that creates a data warehouse and tooling for accessing the data and be able to get sense into that data. On top of that, there are tools like I mentioned earlier, that are more directly integrated with your application, and give you the understanding of how users are using the application. It’s a good question, you know which one to choose, and actually, maybe a bit of a plug in into a project I’m working on, actually going to start soon a new podcast with a friend of mine. He’s a data analyst, and he’s moving on with product management. And this podcast is going to be we call it product for product. So it’s going to be about all the different products that are available for product managers, because there’s so many products out there. Yeah, for sure. And, and our first theme is, is product analytics, actually. So

Jaya Gupta 17:05
Oh I’m definitely go listen to this one. Yeah, I am one of those product managers, that doesn’t come from a data engineering background. So like all the advice that you’re giving is very useful. I did learn SQL. Unfortunately, based on the companies I’ve been in, they don’t give me access, because they don’t trust me.

Moshe Mikanovsky 17:25
There is a view on the access that you can get. So databases are very dynamic that way that you don’t need to get, I also, as a product manager, I never want to have edit abilities into the database, because I don’t want to mess up anything. And I know the power of it. Because I’ve been an engineer for so many years, I’ve learned and worked with databases for many years. So I prefer to just have a view only. And then with a view only, you still need to know the structure of the database, and every application might have a different structure. So maybe that is some of that thing, that they’re not trusting some people to understand the database. But this is really up to them to teach you. So if if it’s not documented, or if they don’t follow up any specific, you know, data structure, optimization or whatever, you know, if their database is a mess, then yes, it’s going to be very hard to understand. But he said if database is optimized, and it’s written well, and the naming convention and everything, then it shouldn’t be that hard to understand.

Jaya Gupta 18:31
Right? Well, she, you’re gonna help me in that. So people who work with me watch out, I’m coming.

Siddharth Taneja 18:38
Jaya is gonna be a different person altogether. You can mess with her anymore. Yeah, or no more select star.

Jaya Gupta 18:46
Yeah, Moshe and Sid are gonna be my coaches.

Moshe Mikanovsky 18:51
Yeah, it’s it’s definitely you know, it’s powerful. It’s very powerful to do. But it all depends on what data is in the database. And so things like, you know, what I mentioned before that Pendo gives you or mixpanel or tools like that, usually developers or even us, as product managers, we don’t create stories, or we don’t create these functionality in the system, because it’s quite expensive to build. And why invent the wheel? If there are applications out there that already do that? Right. So we, and that’s the next layer of understanding which product, you know, is good for you. I’m a bit biased, because I do like Pendo and I’ve been using it. And I implemented it in three companies already. And the reason is because it’s a moralistic platform, right? That has no just the data analytics is has other features for product managers. So you can create enough messaging for the users, you can create NPS and then get the data related to the NPS in the system. And even you can associate different sentiment of the NPS to your specific features. So it combines these two things together, right? You have also additional features like feedback collections for prioritization and stuff like that. And I do like platforms that give you a lot of things in one platform rather than trying to stitch up a solution for many different smaller small systems.

Siddharth Taneja 20:24
Definitely, yeah. Oh, that’s, that’s a really good insight on, I am definitely going to try Pendo, probably in the next week, just to get our hands on. And I will circle back to you Moshe about the tool if I get lost, because there’s one more thing that happens, right? To all the listeners, different tools have different UI interfaces. And there is always a learning curve involved when learning a new tool, right. So while selecting a tool, it’s always a great practice to look at a couple of videos online, look at your network, to see if anyone can share the challenges and advantages of each of these tools. And, you know, finally, since you are responsible for building a product, I always go that extra stretch to try out a trial version of plugins, some sample data, right, and try and see if that tool really helps not just you, but your team. And a tool like maybe the one that most you spoke about also helps the user so it’s a really good practice to try and get hands on this was something that came up in the conversation more sure was the kind of different kind of data or the different kind of data sets available for a product manager. And as an associated question to that would be different challenges a product manager might have while working on these data sets. So can you share some experience on the different challenges that you think a product manager just working on a data set might have?

Moshe Mikanovsky 21:52
Yeah, I think the main challenge is really sifting through the data, because too much data could be a problem as well. So you want to understand, what is the goals of really getting the data? What are we trying to get out of this. And you know, we have those leading indicators versus lagging lagging indicators. Exactly. And a lot of us are very, leaning towards the lagging ones, because there are quite easy to get right where the leading ones are much harder to get to. But they are much more important than the lagging ones. So the login ones could be like how many users I have, and you know how many new registrations I have and stuff like that. But it doesn’t really mean that they’re using the system or they’re happy with the system, right? Where the leading ones would be things like they, they actually went through a specific transaction or a specific activity in the system that let them make, basically, they saw the aha moment you let them through to an aha moment. And because of that they converted and they will become a much more happier user. So you’ll really need to define those data points that you’re looking for. And you can order as much as you can ignore all the rest. And that that is also one of the issues with that I see with reports and dashboards, right, those reports. And dashboards are also like, whether it is data points that you do for yourself, or data points that you create for your users and your product. Right? They are solution for a problem. But many times the if we don’t define what the problem we’re trying to solve, which has given a generic report or generic report system, right, and then tell the user Oh, you can do all of these neat things over here, but they have no idea how to do them, because it’s so complex, right? And at the end of the day, they don’t really get the value from it. Or if you dive down into what exactly is it that they need and why they need it and what the problem they’re trying to solve. Right. A report or dashboard might not even be the the right solution for that.

Siddharth Taneja 24:05
Right. Right.

Moshe Mikanovsky 24:06
So these are really the main things that I see about the data that to look for, right? And of course, the second thing is is how true the data is. Right? So is the data is correct? And are there issues with the data. And for that you really need to have good data people. That’s why bi sometimes so important to have in an organization because getting to have data that is clean, doesn’t have noise, and and is brought properly from different places where where it comes from, is also not an easy task.

Siddharth Taneja 24:45
Right? Right.

Jaya Gupta 24:47
We talked about a lot of great stuff and I’ve been profusely taking notes. Mostly I’m going to be reaching out to you after selfishly but just in a quick sum, I think what I’ve heard number one, we’ve been graced with your presence today, we’ve got so much experience. And you’ve talked about, I’m going to put it in maybe not the order that we talked about. But what I got out of it was the importance of being comfortable with discussions to understand data and really introducing yourself to the data experts, right. Business intelligent experts, because they know the tools, they know, the capabilities, and they can help you shape what your ask is, in a way that can help satisfy your objectives. Yes, I think Sid what you kind of promoted also is, while you’re going through this as a product management, don’t feel afraid to try some of these platforms hands on, you know, get free exposure so that you can have the guts and something I pulled in to ask for view access, get used to SQL and some basic things that kind of promote you from Excel into something a little bit more perhaps advanced, though Excel is a great start. Don’t get me wrong, I that’s where I started. But of course, there’s much more advanced tools, so so why not put some sense to it? And Moshe you actually had a couple of promo with one of your podcasts, you wanted to just mention anything about it first episodes?

Moshe Mikanovsky 26:23
Sure, yes. So we are going through, you know, the technical things of setting it up and all of that. And we just recorded like a quick promo. And we uploaded it the other day, to, you know, the different platforms. And it’s called Product for Product. And so sometimes if you search product for product, it doesn’t find it I saw. So it really depends how you do the search. But you can either search for my name Moshe Mikanovsky in there, or Matt Green, he is my co host. And the idea of there is going to be to look into different themes of products, or product managers. And the first theme, we’re looking at his product analytics. So we were doing one episode that we were talking about it What is it? That theme is all about what we expect him for products like that. And then because we don’t really know all the products, we are inviting guests that are product managers and are using this product so they can tell us from their experience. And then after we are we have a few episodes talking to guests about different products. We have another episode of talking between us what we learned, etc.

Siddharth Taneja 27:35
Right.

Jaya Gupta 27:36
Well, there you have it, that’s product for product, and you definitely will have a couple of listeners from this show to join.

Siddharth Taneja 27:42
Definitely, I think we had a really good conversation and Product for Product is something that I’m looking out for. And guys, I would strongly recommending reaching out to Moshe. And because he’s a great mentor. He’s done the things that he can share with you. Thank you. This is a wrap for our session today. Thank you Moshe for taking out some time today, Jaya as well. Thanks again for bringing us all together here to the listeners. Thanks for joining in. And stay tuned for our next episodes. And you have a great weekend ahead.

Jaya Gupta 28:13
Bye guys.

Rommil Santiago 28:16
This episode was edited by Jin Yang. She is the co founder of Mindset Masters. Mindset Masters works with professionals in all industries to encourage high school and university students to develop a growth mindset. If you are interested in speaking in one of their online workshops, please reach out at any time at contact.mindsetmasters@gmail.com.

Transcribed by https://otter.ai

Connect with Experimenters from around the world

We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.