Survivorship bias is a cognitive bias that occurs when only successful or surviving elements are considered in situations. This can lead to skewed perspectives and flawed decision-making processes.
Understanding and overcoming survivorship bias is crucial for making informed choices and developing a more realistic view of situations. In this article, you’ll learn what survivorship bias is, how you can identify it, and how to avoid it within your team.
Survivorship bias is rooted in the tendency to focus on the survivors or successes while neglecting the failures or those who did not make it.
This bias often emerges when analyzing a sample that is not representative of the entire population, leading to distorted conclusions and misguided decision-making.
One classic example of survivorship bias is evident in historical aircraft analysis during World War II. Analysts studied planes that returned from missions, examining bullet hole patterns and reinforcing the areas that showed the most damage:
However, this approach ignored the planes that were shot down and did not return. The focus on survivors led to reinforcing non-vital areas, as those were the only sections analyzed. Such an oversight can have disastrous consequences if not corrected.
In the world of product management, survivorship bias can occur in a number of situations. Some of the most common include:
Identifying survivorship bias requires a critical approach to data analysis and decision-making. You can take the following steps to support you in doing this.
When analyzing data, the first place to start is to question the data set itself. Does it include both successful and unsuccessful cases? Does it only focus on a subset of outcomes? What might be missing from the analysis?
Investigate how the data or examples were selected for analysis. If there was a specific criterion for inclusion, it may introduce bias. Ensure that the selection process is transparent, fair, and representative of the entire user population.
Evaluate whether the data takes into account changes over time and historical context. Survivorship bias can occur when the analysis overlooks how conditions may have evolved and impacted outcomes.
To avoid survivorship bias you need to make a conscious effort to consider both successful and unsuccessful cases, ensuring a more comprehensive and realistic understanding of a given situation. All members of the team have a role to play in identifying and addressing any potential bias.
Here are several strategies to help avoid survivorship bias.
One of the most effective ways to avoid survivorship bias is to diversify the sources of information used in decision-making. Instead of relying solely on success stories or readily available data, actively seek out a broader range of cases. This can include looking at both successful and unsuccessful outcomes, as well as considering a variety of perspectives.
As a product manager, try to:
As you’ve seen, actively including instances of failure in your analysis is key to ensuring that you don’t encounter survivorship bias. If you are studying a particular product or feature, make sure to look at both successes and failures. Consider return rates alongside sale conversions and account for the scale of site errors alongside site uptime.
When dealing with data analysis, you can use random sampling techniques to ensure a more representative selection. This involves randomly selecting cases from the entire population, minimizing the risk of excluding certain outcomes. Randomly select customers to receive feedback questionnaires and include different customer cohorts in your promotional activities and monitor all their behavior.
As should always be the case when analyzing data, it’s important to challenge assumptions about the data being analyzed. Does the data include successful and unsuccessful cases? Does the data show a reflective and comparative time period? Are you looking at all the available options?
If you’re able to regularly compare assumptions and conclusions against real-world data and outcomes then you can ensure that you are making decisions that are grounded in reality.
To help with this, try asking yourself the following two questions:
One of the key pillars of product management is open dialogue with involved parties, as this encourages individuals to share their experiences. This can contribute to a more transparent and realistic understanding of the complexities involved in various endeavors.
Survivorship bias poses a significant threat to rational decision-making when developing products. The key lies in embracing a more inclusive and realistic perspective that appreciates both success and failure.
Through a combination of critical thinking, diverse data analysis, and a willingness to learn from setbacks, you can navigate through survivorship bias and make more informed, balanced decisions.
Featured image source: IconScout
LogRocket identifies friction points in the user experience so you can make informed decisions about product and design changes that must happen to hit your goals.
With LogRocket, you can understand the scope of the issues affecting your product and prioritize the changes that need to be made. LogRocket simplifies workflows by allowing Engineering, Product, UX, and Design teams to work from the same data as you, eliminating any confusion about what needs to be done.
Get your teams on the same page — try LogRocket today.
As the name alludes to, the way you frame a product significantly changes the way your audience receives it.
A Pareto chart combines a bar chart with a line chart to visually represent the Pareto Principle (80/20 rule).
Brad Ferringo talks about how he helped develop modern “earconography” — sound language that creates context-driven audio notifications.
Without a clear prioritization strategy, your team will struggle to tackle competing demands and can end up confused and misaligned.
Learn more →
