Hi Luis, thanks so much for taking the time to chat with us! How have you been?
I’m grateful, and happy to be talking to you. I love sharing knowledge with the communities, because that’s how I also keep learning. Will try my best to answer your questions 🙂
Let’s start off with a bit about yourself. Could you please share with our audience what is it that you do and a bit about your career journey up to this point?
Currently I’m leading the Experimentation area at Farfetch. But let me give you a quick background story: Started as an software engineer, building solutions for many clients (the whole spectrum, from native apps up to governamental intranets), but soon realized that I needed to be closer to the user, understand how they were interacting with the solutions we created and why. This wasn’t the status quo for the typical consultancy agencies that tend to project deliver and move around and the end. This led me to become Usability Director, working as the interface between engineering/design, business/clients and final customers.
After that, the crazy world of the startups pulled me in, and embraced a new challenge to start creating a company builder, that allowed me to start to apply many of the experimentation concepts (at the time I didn’t called it like that) to the buildup of ideas that would become real products and then eventually companies. This mix between engineering, customer focus, and business drive, was what made me realize that product development was the sweet spot I wanted to pursue moving forward.
After I joined a couple of startups I helped grow as Product Lead and that’s how I actually joined Farfetch after the acquisition of one of them (Gleam – a mobile fashion discovery app) that was strongly data-driven.
At Farfetch helped to build the Data Products area (e.g. Recommendations, Conversational Commerce, Computer Vision initiatives) and after that we saw the potential of creating a Test & Learn center of excellence – a fancy name for an horizontal team, that would enable all the other product (and not only) teams to follow the Experimentation mindset, cultivating practices and building tooling.
As someone with a good deal of experience in building Experimentation culture, at a high-level, could you tell us how you can identify whether an organization has a strong Experimentation Culture?
When a company is able to challenge their own processes, test, learn (& share) and iterate on them following the same scientific methods that we can apply to any product test, this means that they are really embracing experimentation and that it has become part of their culture.
Experimentation is much more than a tool to validate business impact, and only mature companies with this strong culture are able to go beyond the challenge of applying it to their product and engineering teams.
Of course in the middle of it, you have super strong product teams that keep applying these methods and practices, and an immediate way to identify that, even from the outside, is the continuous capacity to iterate their products and adapt and learn from their customers, focusing on their success at first. Being customer obsessed is the beginning for any good product-led company, and per consequence a great indicator of their experimentation maturity level.
Are there critical things that leaders can do to help ensure their companies embrace Experimentation?
Believe in the process. Accept that they (and most of us) are wrong in our assumptions, but that we can learn from them. Have clear strategic orientations, using customer success leading indicators (some call them a North Star) that will help guide all the product teams. Accept the risk of testing, but also don’t be afraid because, with the right approach and practices, experimentation has risk mitigation and loss prevention as part of its DNA. Hire true Product Managers (not just people focused on the delivery of projects) and enable them to do their job, and give them the tools they need to (user research capabilities, good product design peers, analytical frameworks and product analysts and testing tools).
Are there any pitfalls that leaders should avoid? I.e. are there any Experimentation Culture killers that must be addressed at any company?
Biases are the main pitfalls at any place. Safety biases, experience biases, distance biases and many more … All of them are part of human nature, but we need to know how to accept them, identify them and use them in our favour to mitigate their impacts.
When the company leaders start dictating roadmaps instead of listening to their customers and allow the product teams to explore opportunity spaces and learn with their users – this couldn’t work.
When company leaders are unable to provide focus and clear leading indicators around their company/product strategy – this couldn’t work.
When company leaders incentivize their teams to keep delivering at all costs, without measuring the impact of their solutions, only focusing on the milestone delivery and not focusing on having the right solution for their customers – this couldn’t work.
When company leaders don’t encourage and demand their product managers to listen to their customers/users, not blindly doing what they say, but embedding them as part of the learning process – this couldn’t work.
As a former founder, when should new companies start considering running experiments? Are there any signs that it’s time?
Moment 0 🙂 Experimentation is not just about testing product experiments. A big myth is that experimentation = AB tests, because in reality that is just one (an important one) of the many experimentation methods.
Founders can and should apply many of these methods (User research, user interviews, design prototypes, fake-door tests, real AB tests etc), starting by prioritizing their key questions, then identifying the best testing methods to answer each one of them (faster with the right confidence level for that moment) and keep iterating through that with all the learnings and new questions that will come up.
You can apply this when you are even testing a product market-fit, or even a concept of a business model. We did that when we were actually building and launching (and failing) many startups. Only some of them ended up being true products and real companies of success, but all the ones that failed were learning opportunities that helped to build the best next one that would solve an opportunity.
As a product leader, in your opinion, how would you introduce a new product manager to Experimentation?
A true Product Manager will be the one that is already customer focused, with a strong analytical and qualitative enquiring need. After that is just a matter of providing the right tools, work with him/her to know how to take the best advantage of all them, and making sure that they embrace the practices (like some of ours: standard hypothesis format and documentation – LEAR, hypothesis Peer-Review sessions, regular training sessions, Test & Learn company-wide sharing sessions).
Finally, it’s time for the Lightning Round!
Are you a Bayesian or a Frequentist?
Both. One size doesn’t fit all 🙂 But we currently have been evolving our own Sequential model that has a base on the Bayesian.
If you couldn’t work in Experimentation, what would you do?
Would quit calling myself as a Product Manager at that place 🙂
Describe Luis in 5 words or less.
Passionate. Educator. Customer-obsessed. Foodie. AFOL (just for the geeks to decipher ehehe) 😉
How has inspired you the most in this field so far?
So many … John Cutler, Lukas Vermeer, Stefan Thomke, Ron Kohavi …
Who should Experiment Nation chat with next?
John or Lukas 🙂
We’ll see what we can do! Thanks so much for chatting with me today!
A real pleasure, and when you come to Lisbon, say hi 😀
Connect with Experimenters from around the world
We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.
Related posts:
- About our new Shorts series: CRO TMI - November 8, 2024
- Navigating App Optimization with Ekaterina (Shpadareva) Gamsriegler - October 18, 2024
- Building Your CRO Brand with Tracy Laranjo - October 11, 2024