Product Experimentation: Building a competency and fluency with data (Part 1) Experiment Nation

Home / Product Experimentation / Product Experimentation: Building a competency and fluency with data (Part 1)

Podcast transcript

Rommil Santiago 0:01
From Experiment Nation, I’m Rommil, and this is Product Experimentation. Product Managers everywhere are leveraging the power of experimentation to solve customer problems, and build better products. Learn from seasoned PMs and find out how to up your product management game with the latest experimentation strategies.

Jaya Gupta 0:29
Welcome to another session of Product Experimentation with Experiment Nation. I’m your host, Jaya, and today with me is joining

Siddharth Taneja 0:38
Siddharth. Hey, everyone!

Jaya Gupta 0:42
Thanks Sid. And today we’re going to be talking about a little bit about skill sets, and what product managers might think of data when they think about their roles and responsibilities as a product manager. Today, we’re going to actually start off a conversation with a question about what’s been your experience with data? What does that look like, and when you’re starting as a product manager, and as you grow through your experience. So maybe Siddharth I’m going to ask you you, when you first started as a data Product Manager, what was your first exposures to data? What did that mean to you?

Siddharth Taneja 1:17
Yeah, um, so I was a data science professional, and turned into product management on, I came from a unique perspective where data was at the center. So everything else revolved around it. But when I started building products for clients, I quickly realized that it’s the other way around, where clients are the center, and everything else revolves around it. So my experience while working with data, was I used data as a means to an end, right? It was like one of those really crazy weapons that you could use to derive clear and more concise value for a client. I believe, I use a lot of different techniques, right from the discovery phase, all the way up till the end of the product lifecycle, which was literally part where I tried to leverage data insights metrics to make informed decisions. How was your experience, Jaya?

Jaya Gupta 2:24
For me, when I joined product management, I got into an ops role. So it was handling product, product outages, service outages, communications and things like that I didn’t really get into the launch phase until a little bit later, just so that I could get the lay of the land of the products that our team was managing. And so with Ops, you got to see the behavior of not only the products and, and the stability and health of it, but also how customers responded. So I wouldn’t say that I had hands on statistics or metrics, but those outages or those service disruptions became the data and statistics that I was exposed to first. Yeah, so So I started to realize that, you know, when you had to give management updates about when the incident started, when it finished, the response times the volumes the impact, you started to realize how important your product actually really was, in your customers’ lives. And then thereafter, when I when I was trusted to, you know, start a product launch or or get behind a product launch. I was personally just genuinely curious about who am I building? Whenever the Ask was for, it wasn’t really blue sky at that time I was given the product to build. There wasn’t really much of a focus on vision or product strategy. It was there’s a need. Go ahead and build this.

Got it? Yes. One of the most toughest scenarios, right?

Exactly. And so that ended up being relying on market research and partnering with my marketing teams about the demographics of the individuals that make up our customer segment. So those are the two exposures. You know, growing up

Siddharth Taneja 4:15
Yeah, that’s, that’s really cool. And I think what’s interesting is, are your first exposure, which was product outages, right. It’s it’s really interesting how you stumble into actually like, you know, a situation which forces you to explore with data on. I’ve had so many colleagues who, you know, who lead their product management with design with business, right. But ultimately, at some point, they had to leverage whatever data hands on, right on. Did you have any Did you have any situation where you know, you could not really rely on say the analytics team or the engineering team to get you insights from data and instinct you have to go really hands on.

Jaya Gupta 5:08
I wouldn’t say that I couldn’t rely on the analytics team ever, particularly with the analytics team, just because when you’d ask a question, they’d be able to at least tell you this is what we’re equipped with. And if they don’t have it, I was fortunate enough to have partners that looked forward to other alternatives, like how else could we get what you’re looking for? And even before that, understanding the what I’m interested in so that they could help me formulate the questions that I’m looking for, right? It’s quite often that as product managers you think you know what you’re looking for. But when you actually talk to a data scientist or your data analyst, they actually take a couple 10 steps back to understand what’s motivating your questions. What are you really after? What’s your objective? And then you discover, oh, actually, you helped me find out it wasn’t A that I was looking for it was B. From the data engineering side, I think, just because of current implementations, there’d be inabilities to get data. But I think with an appreciation of what you had you used, you took it at face value. So for example, if you don’t have digital analytics implemented on day one of your launch, please make sure that that’s not the case. For those of you who are listening, it’s very important that you enable yourself with the analytics. If you don’t have that, then as you can you use the logs, or when I mean logs, there’s there’s user logs, there’s activity logs that generally you implement, from a technical standpoint, as a best practice to identify and help with troubleshooting also, to check the health of your, your system and services. Right? So what can you use out of that? It’s never a shot case, where if you’re not, if you can’t get something very, very specific, what can you get around it that builds up the support to your hypothesis? Yeah. So this one’s interesting. We’ve talked about our first exposures to data and what, how that’s helped us. I wanted to steer us into the conversation around skill sets, and how do we actually build a fluency with data? Some of you, listeners, you may have heard about this product competency toolkit that was created by Ravi Mehta, who is a former C, Chief Product Officer of Tinder, Facebook, very named companies. He actually created this last year, while I guess he was busy with COVID, with everybody else. But it’s been actually a very, very meaningful reference for me. And I hope that it’s going to be for for listeners out there that are interested. So this product competency toolkit, basically, it’s broken down into, let’s just say four categories, and then split out into competencies for each. I’m not going to go through all of it. But one of the the four main categories is customer insights. And within customer insights, the competencies that he highlights one is fluency with data. Or data, if you say, one with voice of the customer, and one with user experience design. Okay, so today, we’re going to talk about fluency with data. What does that mean? When we talk about fluency? I think what I read and understood from this competency guide is it’s beyond reporting data that you see in the system, or that’s collected from users. It’s making some sense out of it and helping others understand what the data is telling you. If you put points together, right, right. So in that case, how would How might you take a look at your data fluency? How, how might you shape that in your world?

Siddharth Taneja 9:14
RIght. That’s a really tough question. And that’s a really tough scenario, right? I have worked with teams where we were building products with no existing architecture to house the different data points coming from the fraud of usage and the user experience side. When I was working with this set of teams, one of the key challenges was to time um time and actually figure out the exact situation when we would create an architecture for example, during the first iterations, or the MVP phase of our product, we did not really care about you know, setting up on some data architecture, which could capture the usage data, right. But as we moved on from one iteration to the other, we realize that to scale the product and improve and iterate on the product, we do need data, we do need insights from how our users are interacting with the product. And that is when we brought in the influence of data to our decisions.

Jaya Gupta 10:23
Interesting. Okay, so for you, if I’m hearing right, it wasn’t, it definitely wasn’t step one, to make sense out of the data that you’re exposed. First, you’re kind of creating awareness of the data. And then, as a next phase, you’re using it in a way that you can grow your, your knowledge of the product.

Siddharth Taneja 10:46
Yeah, yeah, absolutely. And I work on on the other side of the spectrum, I’ve worked with mature products, right, which were already live, they were customer facing here. Data Analytics, and leveraging the insights that you can get from a user were pivotal for the success and the next stage of the product. In this scenario, there were multiple tools that I used with the teams that I was working with a couple of them were Google Analytics, mixpanel amplitude, to actually understand the user journey through the product, and make meaningful iterations using data and not just by our assumptions.

Jaya Gupta 11:27
Nice. And for those of you who are listening, we are not sponsored, just letting you know. But yeah, I’ve also had exposure kind of using some of these platforms. And when you implement digital analytics for the first time, you know, that itself is probably a little bit of an learning experience, where you’ve tagged things where it’s providing value. It’s funny that when I take a look at the first dashboard that’s created, what I thought I was interested in, sometimes doesn’t end up being the focus of my attention when I look at the dashboards, and then I have to fess up and say, Sorry, I actually would find this more meaningful and valuable. And then, you know, you go back and forth, but this is the thing, right? I, for folks who know me more closely, I’ve, I’ve mentioned, the Dell maturity model, which kind of takes a look at data awareness, data proficiency as stages until you get to a point where, like, in nowadays, you can say you’re artificially intelligent, but that’s so much of a far thought that, you know, you got to start with the first basic, so I appreciate what you’re saying about putting in the, the site around developing a data platform that can give you much more value. Right. I wanted to, I wanted to then talk about when when we look at building the skill set, what made the difference be between knowing about data, and then stringing data together. And so I’m going to start off with an example that that might help. And then I want you to share your thoughts and maybe some experience around that. The first time, I felt really proud about a product delivery, related to a payments experience that allowed corporate organizations to exchange funds between their companies across Canada and us. Okay, it was huge. Especially because you’re going across the border, there’s a lot of complexity with the product.

Siddharth Taneja 13:42
Definitely a lot of complications, a lot of governance issues and whatnot.

Jaya Gupta 13:46
Exactly. And so, me and my partner, this was a collaborative effort. We were so proud that at the time it was, you know, when we had to actually announce our launch and go live, particularly with our sales group, because it was in corporate groups, we don’t necessarily have broad broadcast customer communications live. So we didn’t, we didn’t really prepare. Like, because I’m not going to go in and explain the product that helps you transact between company that’s pretty self explanatory. Right? Right. And we created into an experience that was pretty much as simple as transferring, right? So what I did in the last 15 minutes, because I was tracking every single transaction coming in, I summed up the total amount of all transactions. And basically, the only line that I dropped was that we’ve facilitated in 700 in seven months, about 500 global corporations to transact up to a trillion dollars between Canada and the US.

Siddharth Taneja 14:58
That’s massive. Wow.

Jaya Gupta 15:00
And that’s the impact that I made. It gave people a sense of how we’re helping the economy, right about merger acquisition type activity, because if you’re transacting between your company, your own company, there’s a purpose for that, right. And so that’s probably a pivotal point where I changed my behavior of looking at numbers and data points and making something out of that.

Siddharth Taneja 15:24
Oh, that’s really cool. So if I got that, right, like, when you went in for this engagement, you weren’t really sure about, you know, what the stakeholders would need from the product or the data that you can bring in, right. But as, as the engagement matures, you’ve decided to bring in more variables that could help make better decisions, right?

Jaya Gupta 15:49
Yeah, this way, exactly. This way, when you’re giving that type of information, it was a bit of a mixed bag. In terms of audience, there were sales individuals, they’re also very, very intelligent when they carry compensations and understand the impact that they’re making. They’re also senior managers, who are thinking about, I mean, when we think about payments, that’s a pretty big, monumental experience that is integral to banks, when any financial service provider raise. Yeah. Especially in the business context. It made them think about where we could take that experience and how our footprint was influencing global clients of ours.

Siddharth Taneja 16:35
Right. Right. That’s, that’s really interesting, right? I feel there are, there are a few different ways in which teams actually maneuver we’re using data. And I can imagine, right, like, we’re getting as a PM, it’s important to interact with the different data professionals. And it’s always helpful to be a little hands on with data. For all the listeners out there, who are, we’re trying to bring in data as one of the core skill set while building products. There, there are a lot of courses on Udemy, and Coursera, which are very tailored towards leveraging data for product managers, right? I strongly recommend going through one of those courses, and getting more comfortable. Because if you can speak the data language, you can definitely create that impact not just externally for your your users, but also internally with your teams.

Jaya Gupta 17:44
Exactly. And that’s really well summarized, thanks for sharing that there is a lot of reading material. When we talk about fluency with data. If I point back to the the competency toolkit that we just introduced, again, created by Ravi Mehta that you really start to realize, once you’ve gotten through the basics of understanding the reporting side of it, it’s really getting passionate with, why are people doing what they’re doing? And what is that telling us where are the opportunities that we can, that we can dive into. So that was our, that was our chat. And I think we talked about some really cool stuff. Today, it was all about understanding how we can build our competency with fluency with data and get better at talking about data and what’s happening with our systems and how it’s helping customers. Tune in to another session, where we’ll be talking about how you might be able to apply this knowledge. So it’s a sequel. And we’re we’d be excited to share with you our secret guest. Any comments Sid?

Siddharth Taneja 18:59
No, I think I’m gonna leave it for the listeners to catch up. We have a great guest lined up who would be able to share his experience throughout his journey while building products. So stay tuned. And thank you for thank you for joining me today.

Jaya Gupta 19:14
Awesome. Thanks, guys.

Siddharth Taneja 19:15
Bye, bye.

Rommil Santiago 19:19
This episode was edited by Hilda Bastidas. Helga is part of the Buffete de Nutricionistas team, a digital space to learn about nutrition in a friendly and simple way. It has never been so easy to clear up doubts about food, visit their Instagram at https://www.instagram.com/buffetedenutricionistas/

Transcribed by https://otter.ai

Connect with Experimenters from around the world

We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.