Understanding causation vs. correlation in product management

There are as many product manager hats as there are product phases, and there are as many puzzles as there are product challenges. One main puzzle is understanding causation vs. correlation.

Grasping these two concepts is crucial for project managers to understand how certain features, stakeholders, or processes play a role in the success of the overall product.

Today, PMs have multiple tools at their disposal to gather data. While this abundance of information is welcome, PMs should remain as skeptical and cautious as any detective worth its salt. “Caution” here implies gathering criteria that can be applied when your instincts suggest certain insights might be too good to be true (or too bad to be believed!)

Here, you will learn all about causation, correlation, and how they are connected. You will also understand their differences, the benefits you might gain from accurate causation, and the way the approach saves time and resources for PMs.

Defining causation and correlation

In brief, causation implies a particular relationship where one specific thing or process results in another specific thing or process. Think how the elevator comes down when you press the right button, or how you can burn your tongue if you are impatient and drink your coffee too early! Here, cause and effect are easy to obtain, as in 2+2=4.

Certainly, the goal of modern research (since the inception of the scientific method) is to establish slightly more complex causations. For example, how does heat evaporate water? Why do snow avalanches occur? How do seeds turn into fully formed crops?

Different professionals specialize in developing a set of more or less fixed approaches and tools that can be replicated and tested repeatedly by different teams. If enough repetitions return the same results, then one can confirm that the proposed explanation is accurate, unless a more solid reason emerges, of course!

In comparison to causation, correlation is a simpler affair. Correlations refer to two processes that, at least on the surface, are happening at the same time. Again, water heated strongly enough will evaporate. Both the heating and the vapor occur simultaneously. However, and this is key, we are not concerned with explaining; the observed relationship is all that we have.

In many ways, this is how humans developed prescientific knowledge. Early farmers, for example, would observe that seeds planted and watered in a certain way would grow to become crops.

This is, of course, the first way that causation and correlation are connected: they are both used as explanatory devices (though they hold different weights, as you will see below).

Furthermore, correlations are often the starting point for exploring possible causations. It’s normal to assume some sort of relationship when you see events repeatedly happen together.

That said, the following point is the crux of the matter: not every correlation will equal causation.

Head’s up product managers, correlation does not equal causation

Yes, simple causal relationships are easy enough to spot when you are a product manager, and yes, they often appear as correlations. For instance, if the app for your product introduces responsive design, your intake of mobile and tablet users should increase.

However, many other connections are harder to explain, and some apparently tightly-knit developments can be totally misguided.

Let’s return to general examples. Climate change is a confirmed challenge that took decades to establish in the public consciousness. Thousands of scientists in dozens of countries have been working together to identify and isolate the most important causes.

One of the most observed parallels is the rise in CO2 levels in the atmosphere, and the increase in global average temperatures, with corresponding effects on weather systems. So far so good — an initially interesting correlation has been confirmed repeatedly, both in labs and tests as far away as space.

There is, however, another correlation that many scientists are not really interested in pursuing. And this is not because they are afraid that it will challenge climate change science, it’s actually quite the opposite.

This is the inverse relationship between the number of pirates operating across the world and the rise in temperature levels. You read that right. As pirates have disappeared from the scene, global temperatures have risen.

Should we bring back pirates to solve this global challenge?

The obvious answer is no.

What we see above is a baseless or doubtful correlation. These are prevalent in business, but they are harder to disprove when making quick-fire product decisions. Here are some examples:

• The Californian sun is particularly beneficial for digital companies (the modern tech sector might have emerged in Silicon Valley, but it has spread around the world)
• The more superheroes a movie has, the more successful it will be (there are many profitable superhero movies, but some have been massive flops)
• The number of users and profitability go hand-in-hand (Twitter, with millions of users, has barely turned a profit; whereas smaller social sites have)
• As more people are connected to the Internet, there’s an increased expectation that content and information should be free (in fact, there are more paid digital subscribers today than at any point in the history of the Internet)

Please note: it might be that these observed relationships turn out to be true, even if some counterexamples are listed. However, there is a high probability that they are not sound enough to provide business guidance.

If you lack an established approach to determining causation, you will never know the real reasons why so many firms in California are successful; superhero movies are so popular; monetization and popularity do not always go together, and people are increasingly spending more money on online services.

Next, we will look at how you can leverage causation at different product phases.

Identifying positive causations in product management

Firstly, note that the term “positive” here does not imply beneficial or good. We’re referring to an observed relationship that’s been confirmed to operate causally after isolating all possible alternatives.

Now, we’ll go over the process of identifying positive correlations in product management throughout a product’s life cycle:

1. Researching the product
2. The planning phase
3. Implementing product testing
4. Monitoring the product’s development and success

Researching the product

At this stage, there are often no actual products or features ready for testing, but you might be exploring a particular market segment or specific audience.

Your desk and field research should aim to confirm or disconfirm observed correlations, such as the connection between your competitor’s physical store strategy and digital traffic. Does storefront presence really contribute to online orders? Or is this an expense that your more nimble project could avoid to scale up quickly?

The planning phase

You should make an effort to embed a causal perspective across your product teams when in the planning phase. If you manage a professional operation, tech and non-tech people will have become used to attaching evidence to requests and appraisals.

Without proper instruction, they may be merely employing improperly assessed correlations. Make sure to explain the difference across your teams to avoid false starts.

Implementing product testing

Here is where testing becomes viable. You are likely familiar with A/B testing. When in doubt, most development, marketing, and other solutions offer you the chance of trying out two or more product varieties to understand which one is more successful with your target audience.

You can do this more or less blindly, without paying attention to causal implications, but then you are missing out on additional value. If you can identify why exactly one version of a product or feature performed better, the insight is even more important.

Monitoring the product’s development and success

Evidence-based, permanent development is crucial for managing digital products. But your feedback will be even more powerful if you pay attention to causal factors.

Why is it that my users disliked the latest release: technical issues, or bad design? Why was that feature so difficult to implement: lack of team cohesion, resources, or management? How can we explain our rapid increase in light of our stagnating user growth? Why should the sunset period for this legacy feature begin now, and not further into the future?

These are some of the intriguing questions you should seek answers to. Forget trial and error, embrace your detective hat and the scientific method that helps you identify causation.

Avoid wasting time and resources on baseless correlations

You might think that embedding yet another perspective on your already convoluted development process is too costly. However, causation can save a lot of time and money.

For example, vanity metrics are a plague of digital and other industries. That is, feeling proud about a particular piece of data that actually has little or no connection to your business performance. This happens a lot when new platforms gloat about their rapid growth and popularity, without really considering whether there is a sustainable strategy to retain users or to turn engagement towards profitable services.

Baseless correlations are similar, as they can make you waste time and resources. But the simple explanatory appeal that some of these unexplored observations have can make them even more dangerous.

It is almost like being in the casino, betting on a number and a color for some random reason and making some winnings for a while. However, as you know, the house always wins. In that case, you better have a good causal explanation of why you failed to ensure that you can pick yourself up and try again.

Conclusion

In this article, we talked about the importance of understanding causation vs. correlation in product management. It’s easy to fall into the trap of assuming a correlation is causation, so may this guide be a reminder to always do your due diligence.

Featured image source: IconScout