Building a culture of experimentation

Most tech companies love the idea of rapid-fire experimentation. We’ve all read Sean Ellis’ Hacking Growth, and we’ve collectively been inspired by the stories of unicorns like Airbnb, DropBox, and Pinterest. But a lot of great product, marketing, and growth talents are working in environments that operate quite differently.

Because experimentation isn’t for the faint at heart.

It means doing bold, sometimes risky things to maximize learning. It means some successes but also lots of failures. As per the definition, experimentation clashes with incremental growth. If you want KPIs to go up at all times, then experimentation isn’t for you.

This also means that fostering an experimentation culture tends to be a lot harder for bootstrapped companies — who need revenue to keep the lights on — than for VC-backed companies who can afford to take the risk.

But if you’re looking for a fundamental breakthrough, and have an aching to create something drastically better, then this one’s for you.

The theory behind experimentation culture

It’s easy to echo the theory from the books:

• No matter how much research we do upfront, most of our features will fail to deliver value — According to a McKinsey study, >50 percent of new product launches meet business goals. According to Pendo’s 2019 adoption report, 80 percent of new features are never or hardly used.
• What makes unicorns unicorns is that they’re awesome at quick experimentation and learning — All great “growth hacks” came through relentless experimentation, which was only made possible by embracing failure. It was never the first idea that was a success, it was the 100th idea that came after 99 fails. Airbnb famously runs about 700 experiments PER WEEK!
• Without proper ex-post analysis, you learn nothing — Checking your results against the success metrics you set upfront is non-negotiable.

The realities of experimentation culture

But most of us are familiar with a different picture.

The first revolves around leadership buy-in. “Sure experimentation is fine, as long as every experiment is a success” simply won’t do.

There’s also usually a struggle and a lack of (organization-wide) clarity on the target audience — i.e., it’s not specific or narrow enough. If you’re not clear on who it’s for, you’re going to build a mediocre product that’s kind of okay for anyone. Bloat is given, leading to the next point.

How about the actual capacity of the tech team? 80 percent of the tech team’s time goes into maintaining the product, leaving a meager 20 percent for experimentation. These organizations are not gonna build a rocket ship — there might be small, incremental progress, but no breakthrough inflection points. For early-stage startups, those numbers should be flipped around.

Remember what I said about metrics and data? The reality is, a lot of the time there are no company-level goals or OKRs, and there’s a lack of clarity on which metrics matter. Sometimes product or growth teams aren’t able to measure success. Product analytics is missing entirely or the team needs to harass the data team for answers.

Departmental silos are another common problem, which is an issue for one million reasons. If marketing and/or sales are selling to a different audience than for whom the product and dev teams are building for, experiments are doomed to fail.

---

Over 200k developers and product managers use LogRocket to create better digital experiences

Learn more →

---

Finally, there’s sometimes a lack of expertise. This includes the organization simply not knowing how to run experiments, which levers to pull, how to identify the metrics that matter, how to set the right success metrics, etc.

Making the shift

If any of those above realities sound familiar to you, you might wonder how it’s possible to build a culture of experimentation with all of those roadblocks. Well, it’s possible:

1. Get expansive buy-in

No matter how talented you are, you’ll never change your company culture on your own. You’re asking your organization to start taking greater risks for greater returns and to prioritize learning over shipping, and you’ll need resources, for example, marketing, design, engineering, and product.

Without initial buy-in from key leadership figures, you won’t get anywhere.

After kicking off, you’ll hopefully be able to deliver some quick wins to win over the hearts and minds of the non-believers. This is what I mean by “expansive” buy-in. The more you prove, the more people you’ll be able to bring on board.

2. Become data-informed

It’s difficult to argue against gut feelings or work in a “loudest voice wins” environment if you don’t have any data to back up your argument.

It’s impossible to know where to start working and which lever to prioritize without data.

4 steps to access the right data

1. Create internal alignment on which metrics matter the most (be selective here!) and which questions we want to answer with data
2. If needed, get the right data in a quick and dirty way. You want to show your value as soon as possible. If there’s no data analytics tool in place yet, setting this up will take time.
   1. Example: You’re working for a B2B SaaS company, and one of the key questions is “Which type of customer is getting the most value out of our product and bringing the most revenue?” Start with an export of all paying customers plus one rudimentary product usage metric (e.g., login) plus revenue (e.g., monthly subscription fee). If needed, enrich that data with public company data (industry, size, or segment). Start digging
3. Pick a (product) analytics tool, such as LogRocket, that anyone who needs to work with the data can access and use. Ideally, product, growth, and marketing teams can self-serve instead of running to a data-team service center for every single request
4. Create data awareness and data appetite. A good way to get everyone to infuse data into their thinking is by feeding important information into a dedicated Slack channel. This can be things like a new signup, new aha moment reached, or new upgrade to paid

3. Align goals (and incentivization)

This goes hand-in-hand with #1: buy-in. Learning needs to be prioritized over small, incremental gains, and goals and incentivization need to reflect that.

• Goals — Without clarity and buy-in to company-level and team-level goals, your teams will drift aimlessly and likely have unpleasant run-ins along the way. Whether you use OKRs or a different goal-setting method, the key is to keep things simple, explicit and focused on the customer
• Incentivization — People will do what they’re incentivized to do. If marketing is rewarded for MQLs, you’re at risk of getting long lists of emails of people who will most likely never convert into paying customers. If sales are rewarded for new sign-ups, they’re incentivized to promise the moon to make the sale.

Better options to align goals

There are better, more strategic ways to align your goals and create incentives, including:

1. Using the same team-OKRs and incentivization structures for both sales and marketing, with an emphasis on retention (and expansion)
2. Infusing all team bonuses with 50–100 percent of achieving company-level OKRs (with an emphasis on retention and expansion). Possibly even extend the same incentivization strategies (commissions and bonuses) to product, tech, and CS. They are just as much responsible for customer success as marketing and sales

4. Communicate failures and successes

It’s imperative to communicate openly about both failures and successes. Make sure that everyone can stay informed of what is happening and can ask questions or voice concerns. A simple step is to create a dedicated Slack channel with regular updates on what experiments you’re running and how things are going, always pointing back to the data supporting your hypotheses and findings. Share your learnings with anyone who will listen.

Invite others into the process. Communicating with people is one thing, but co-creation is the next level. When the time is right, open up your idea pipeline to other departments (and possibly even to customers and partners).

5. Go for a quick win (and celebrate)

You’ve got a lot to prove, so better get on that right away. Identify your low-hanging fruits and go for them right out of the gate.

Simple ways to get started:

• Qualitative — Eat your dog food! Go through the onboarding journey to identify unnecessary friction points. Be as critical as possible. You can also do this as a team — Slack calls this their “complaint storm”
• Quantitative — Dig into the data to see which part of your customer journey is causing the most drop-off, so you can identify what to focus on for an almost certainly measurable impact

As Maja Voje says, “Even a half-win is fine at first. Anything that gives you and the team a strong signal that you’re on the right path is a cause for celebration.”

Increase the risk (and corresponding potential gain) as you progress. Strive for quick wins to win over the hearts and minds of more and more team members, and go for the bigger swings later on.

Final words

You can’t take a sledgehammer to an organization and recreate it in the shape you like.

Pace yourself, get buy-in, and rebuild together, brick by brick.

Featured image source: IconScout

LogRocket generates product insights that lead to meaningful action

LogRocket identifies friction points in the user experience so you can make informed decisions about product and design changes that must happen to hit your goals.

With LogRocket, you can understand the scope of the issues affecting your product and prioritize the changes that need to be made. LogRocket simplifies workflows by allowing Engineering, Product, UX, and Design teams to work from the same data as you, eliminating any confusion about what needs to be done.

Get your teams on the same page — try LogRocket today.