How Spiralyze’s Sophie Buonassisi demonstrates the value of Experimentation to clients Experiment Nation

A Conversation with Spiralyze’s Sophie Buonassisi about Experimentation

Clients are always hungry for ROI on their Experimentation efforts. I recently spoke to Sophie about 2 ways the value of Experimentation can be demonstrated for clients, how to build a Culture of Experimentation at companies, and how Experimentation is important to Growth.

Rommil: Hi Sophie, how are you? I hope you’ve been doing well.

Sophie: Hey Rommil! I’m doing well thanks. Feeling super fortunate to be in my hometown of Vancouver, Canada. I was in Italy when COVID hit there and it’s great to have close proximity to family now, thankful everyone is healthy. How are you?

I’m well?—?thanks for asking! So Sophie, let’s start with a bit about your career journey thus far and what you do today?

I have an educational background in criminology and worked for the Canadian federal government on regional and national projects.

From there I transitioned into sales and ended up being exposed to revenue-driving marketing, which I fell in love with.

My direct optimization experience began when I joined Spiralyze, a full-service conversion rate optimization agency. At Spiralyze, I’ve been able to work alongside a fantastic group of people passionate about experimentation and across widespread clients.

Currently, I’m the VP of Optimization, which includes directly working with companies and marketers who want to improve their conversion rates to understand how to strategically do so and what putting a program together entails.

Awesome. Could you tell us more about Spiralyze and how they’re different from other agencies?

Sure, Spiralyze provides full-service conversion rate optimization services, helping companies convert existing traffic into more leads/sales.

A huge differentiating factor from other agencies is our flexibility. This comes in the form of our team, engagements as a whole, and pricing model.

We drop what we call a “pod” in place, which is an entire conversion team consisting of a project manager, conversion specialist, data scientist, researcher, designer, copywriter, developer, and quality assurance functions.

This pod enables us to be full service, including everything from analytics to research to ideation to design to development to testing to implementation to program management.

We become an extension of our clients’ existing teams, increasing bandwidth, expertise, and testing velocity. This full functionality enables us to be super agile and deliver results quickly.

In terms of pricing, we have a fixed monthly fee model or an entirely performance model. The latter is unique and includes absolutely everything upfront with no fee?—?any fee is tied directly to incremental conversion gains.

We also never tie clients into any retainer models?—?we create complete accountability and operate on a monthly basis regardless of a fixed fee or performance model, or if for a new client or a client we’ve been working with for years.

Clients are always interested in ROI?—?how do you go about demonstrating the value of Experimentation?

When thinking about value, a good way of demonstrating it is twofold?—?through direct revenue results and also strategic value.

First, tracking absolutely everything from tests?—?documentation is key. The impact of experiments can then be tracked down the funnel and translated to revenue and growth.

Breaking testing results down by quarter and having an annual summary report can be an effective way of recording this information in a way that’s easily presentable and trackable over time. This could include a summary of every test run in the form of a spreadsheet that captures factors like test name, control conversion rate, variation conversion rate, statistical significance, improvement, and impact factors. Then, a visual summary of the tests run for each page.

Second, experimentation creates a long term growth strategy and uncovers lots of valuable strategic learnings and information. Customer insights from research and testing results can be leveraged across product teams, sales teams, and so forth.

Similar to how tests would be individually analyzed and reviewed after completion, insights derived from research and tests can. They can also then be packaged into quarterly and annual reports, so this information can more easily circulate around the company and provide as much value as possible.

Having worked with a wide variety of clients?—?how transferable have learnings clients been to others?

There are tons of transferable learnings! For example, we do lots of work with SaaS companies and there are many conversion commonalities across our clients.

That being said, conversion behaviour can also vary dramatically, which is why testing is so important!

At Spiralyze we have a database of every test that has ever been run and associated data points mapped. We are then able to leverage this data across all clients, to see conversion behaviour in different sub-categories and for different labelled elements, and understand the likelihood of transferability.

What role does Experimentation play in Growth?

Experimentation and Growth are a match made in heaven. Experimentation enables businesses to test different concepts and determine which drive more growth. Rather than jumping into implementing assumptive decisions geared for growth, any desired change can be tested.

Experimentation is a constant driver of growth, as improvements can be iterated and improved. Together, they create a data-driven continuous loop. Experiment, Learn, Grow, Repeat.

Changing gears a bit. I’d love to hear your opinion on what a nightmare client is?

I would define a nightmare client as closed-minded. Experimentation requires just that?—?experimenting. Companies may want to be more data-driven and ramp up experimentation or start testing to begin with, but when it comes down to it an open-mind is needed first and foremost to be able to experiment with making changes.

I have been fortunate to not work with any nightmare clients! (*knocks on wood*)

How do you define and build an Experiment culture?

That’s a really interesting question. Many companies I’ve recently spoken with have also asked this?—?specifically how to build.

A definition…hmm… a culture that embodies curiosity and humility, and embraces cross-functionality and data.

To build an experimentation culture, I would say that experimentation needs to not be a segment of the business but ingrained in the foundation of the company. It needs to be in the company blood (criminology side of me coming out, ha!), circulating through the entire body of the company.

There’s different techniques of building and maintaining this and I would say transparency is a key part. For example, every week an email or Slack message could be sent out company-wide summarizing what tests were run, what were the results, and what were the learnings.

This helps everyone understand the value of experimentation, leverage the learnings, and be excited about testing.

Finally, it’s time for the Lightning Round!

Frequentist or Bayesian?

Hot debate topic. I lean towards Bayesian but don’t really feel that either is inherently better than the other.

What is your favourite Experimentation tool?

They all have their benefits and they all have their limitations?—?it really depends on what’s a best fit for the business goals. Up there for favourites are Google Optimize and AB Tasty though.

How useful has your degree in criminology been in your current role?

Directly technically useful?—?not very! Generally useful?—?absolutely!

I think everything in life provides learning and development opportunities, even if involving more greatly transferable skills rather than technical skills. I will say though that statistics, which was part of the criminology degree, has definitely been helpful on the technical side of things.

Describe Sophie in 5 words or less.

Dynamic, Passionate. Fuelled by Curiosity.

Thanks, Sophie for joining the conversation!

Thanks for having me and for all of your work bringing experimentation information and community together!

Connect with Experimenters from around the world

We’ll highlight our latest members throughout our site, shout them out on LinkedIn, and for those who are interested, include them in an upcoming profile feature on our site.