Vipul Bansal: Experimentation is the right way to be wrong Experiment Nation

A Conversion Conversation with VWO’s Vipul Bansal

Experimenters are very passionate people. We have to be. We understand the power of Experimentation but generally face many uphill battles in getting others to embrace it. We push through technical challenges, cultural issues, HiPPOs, etc. every day?—?yet stay passionate through it all because we believe in the Practice so much. With that said, I haven’t met many Experimenters as passionate as Vipul. It was a pleasure to chat with him recently and hear his thoughts on a broad range of topics such as being customer-focused, building tools, and how Experimentation is really the best way to be wrong.

Rommil: Hi Vipul, how are you? Thanks for taking the time to chat!

Vipul: Thanks for having me, Rommil. Excited for this!

Can you share with our audience a little about yourself and what you do?

I handle co-marketing for VWO. My primary responsibility is to find a solution to the education problem. Online businesses still do not invest in the concept of (continuous) experimentation because they do not know how to handle failures, how to set up teams and processes, how to manage ideas, etc. People are always on a lookout for inspiration from brands for whom experimentation is part of their company culture.

As a co-marketer, I try to bridge this knowledge gap by identifying bottlenecks that keep online businesses from embracing experimentation and resolving them by inviting Masters to share their story. In view of this, we organized our very first annual event?—?ConvEx where we invited the most admired experts in experimentation, from Microsoft, Booking.com, Trainline, HubSpot, Encyclopedia Britannica, Avast, CXL, and more. I’m currently working on our LIVE webinar series?—?Masters of Conversion (the name says it all 🙂 )

I’ve been involved with VWO for close to 4 years now and previously I was working for their sister product PushCrew, a web push notification tool, which was imbibed into VWO last year as VWO Engage. There I had the pleasure of wearing several hats including, analytics, lead nurturing, content distribution, webinars, and more! Been a joyride 🙂

With so many Experimentation platforms out there?—?how is VWO different from all the others and why do companies choose VWO over other services?

Love that question!

Short answer?—?VWO is a lot more than just an A/B testing tool.

Long answer –

Being understandable is a virtue. Isn’t it? People trust you and prefer to spend time with you if you understand them, their problems. It makes the relationship worth establishing and pursuing.

The same applies to the software world too. A buyer steps out in the market because they just encountered a problem. They are out not only to buy a solution but to build a relationship. And if your software provides a solution that is built on that understanding it instantly builds credibility.

That’s what makes VWO unique in its offering. We understood that the primary challenge businesses are facing isn’t how to run ab tests. There are tonnes of articles on it. Online businesses are struggling with setting up and following a structured approach to experimentation.

A structured approach involves identifying the exact leakage point in your conversion funnel, that point where user experience is breaking. This is followed by collecting data on what exactly is causing the experience to break. Maybe there’s too much or too little information for the buyer to make a decision.

Once you have all the quantitative and qualitative insights, you have enough ground to create a hypothesis. A hypothesis is simply a summary of why you think a variation will produce an expected/desired impact.

After all this is done, now is the time to test! You can set up a simple a/b/n test, multi-variate test, or split URL test.

Now that you have a test result, you have factual data (and not opinions) to make a decision about improving user experience.

This is our competitive advantage. While other testing tool providers are just that?—?a testing tool, VWO is an experience optimization platform that enables you to get a better understanding of your users.

“If you have the resources you can build an in-house platform but achieving efficiency will soon become a challenge.”

There’s often a debate between buying a 3rd-party tool vs. building a platform in-house. What’s your advice?

That’s a great question too! Part of it is covered in my previous answer.

I’d say ‘let the experts handle it!’

If you have the resources you can build an in-house platform but achieving efficiency will soon become a challenge.

You can write code that enables you to create variations of a webpage you want to test. But believe me, you don’t want to just ‘test’, you want to ‘understand’. There’s a difference and explaining that difference is a challenge for me as a marketer.

I also want to take this opportunity to explain one more thing.

The importance of the role of data in decision making cannot be debated. How else will anyone know what next steps to take? Forget about running an existing business(es) it’s impossible to even start one if you don’t have an appropriate amount of data on your market.

The insight here is that data is the most important hook.

Now imagine how important business decisions will be impacted if some data points are lost. This is most likely to happen for disconnected tools where you are using one tool for gathering user insights, another for testing, and yet another one for keeping track of what you’re testing.

This is where VWO stands out as well (not being biased at all). All the data is stored, analyzed, and tested on from under one roof.

In your opinion, what is the culture of experimentation? And what are some of the reasons companies haven’t adopted it yet?

I hope I could have answered it simply but this is something that needs explaining.

So, there’s something called the Red Queen effect. It simply means that to stay relevant you have to work harder.

“It is not the strongest of the species that survives, nor the most intelligent, but the one most responsive to change.”

— Charles Darwin

People’s lifestyle is evolving. Their expectations are dynamic. If you, as a business, do not invest time and resources in aligning with these expectations, someone else will.

Experimentation is a means to that alignment.

Only the CEO of the company doesn’t need to run fast. The entire organization needs to maintain the same pace.

This is the understanding that everyone in the organization should have in order to build a culture of experimentation.

Paras Chopra, the founder and Chairman of VWO, gave an amazing presentation on this. You can view it here.

And this is something even Lukas Vermeer talked about.

Now, the only reason I see companies not adopting this culture is because of the lack of patience. Sure enough, changing mindset takes time and making everyone in your org to think in the same direction is a tough task. Paras has mentioned really cool points in his presentation at ConvEx of how willing businesses can start building that culture. Do have a look.

Also, consider reading the document shared by Ronny Kohavi listing some reasons.

With companies running more tests every year, can you share how they keep track of experiments and share learnings?

That’s a good trend we’re seeing as well. Experimenting consistently is a sign that businesses are increasingly investing in understanding customers.

A widely used method by businesses in terms of keeping track of experiments is via Google Sheets or MS Excel. All the observations and hypotheses are recorded in rows shared with the people involved in the project or across the organization.

I recently read an article about Duolingo’s testing process. They train their employees in running tests. I think that’s a great way to not only improve customer experience but also build a culture of experimentation.

Duolingo also organizes meetings where employees can share learnings from their tests.

VWO has a kanban board for an organized and easily understandable way of keeping track of experiments in the pipeline.

One thing that should be taken care of while running multiple experiments is the audience segment being tested should be a part of only one experiment at a given time. Showing multiple variations to the same audience contaminates the data and any insights derived are not reliable.

I recently loved something you said on LinkedIn. You said, “#Experimentation is the right way to be wrong.” What does that mean to you?

Haha!

I keep thinking about stuff. It’s my favourite pass-time activity 🙂

I was led to this conclusion through several ideas.

Firstly, about the fear of being proven wrong. I’ve been on sales calls and people are always concerned about being right. They’re clearly avoiding the thought of being ‘wrong’.

Secondly, avoiding conflicts within the team. Maintaining positive relationships is important. When people have ideas they are excited about it. Telling them the contrary would create conflict and damage their relationship.

Thirdly, businesses still want to improve customer experience.

Most businesses choose the easy way?—?opinions. The HiPPO wins. I won’t delve deeper as this topic has already been talked about a lot.

Experimentation, I feel, is the only customer-driven method to understand what experience they really desire while avoiding any conflicts and fears. It improves team collaboration and gives the confidence to be wrong.

So, yeah, Experimentation is the right way to be wrong.

Could you share with us a bit about the Masters of Conversion series?

So glad you asked.

Masters of Conversion is a go-to source for all information around experience optimization.

We feel passionate experimentation professionals have already consumed enough theoretical knowledge and are looking for more practical understanding. So we invited the Masters from the most aspiring brands of the world who have been there and done that, to share their learnings with an equally passionate audience.

The sessions are presented through a LIVE webinar that the audience can attend from the comfort of their place. It gives them a chance to ask their questions to the Masters and get answers immediately. It’s also an opportunity for the Masters to interact with the people who admire their work.

The foundation has been laid with six sessions already live that received a great response from the experimentation community. More are scheduled for the coming months. I’m excited!

“Experimentation, I feel, is the only customer-driven method to understand what experience they really desire while avoiding any conflicts and fears. It improves team collaboration and gives the confidence to be wrong.”

Finally, it’s time for the Lightning Round!

What books or articles do you read?

I love reading. Although I’m guilty of not picking up enough books last year as I used to but haven’t stopped reading.

Books that have opened my mind and get a better understanding of the world:
1. Sapiens by Yuval Noah Harari
2. Art of Choosing by Sheena Iyengar
3. When coffee and kale compete by Alan Klement
4. Predictable irrational by Dan Ariely

and many more.

For articles, I frequently visit HBR, Farnam Street, Stratecherry, AL-Daily, and many others.

If you couldn’t work in Experimentation?—?what would you do?

I’m a marketer by heart and soul. Serving the experimentation community has been a great learning experience. If not a marketer, I would be a chef. I can cook decent Indian food.

Describe Vipul in 3 words.

Haha!

Vipul is a word of Sanskrit origin that means ‘a lot of’. So, I’m a lot of happiness, a lot of curiosity, and a lot of perseverance.

That’s pretty awesome?—?and fitting! With that, Vipul, thank you for joining the conversation!

Connect with Experimenters from around the world

We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.