As a product manager, you’re responsible for bringing your product to market, but also for effectively managing it once it launches. In order to do this, you need to be aware of the KPIs that enable you to have a comprehensive understanding of the health and performance of your product. System performance metrics help remove any guesswork that may arise and allow you to make informed decisions moving forward.
Within the category of KPIs, system performance metrics unpack the usability, functionality, reliability, and efficiency of a product. They focus on key operational knowledge that you need to understand your product. Some of the most important are mean time to repair (MTTR), mean time between failures (MTBF), mean time to failure (MTTF) and mean time to acknowledge (MTTA).
In this article, you will learn what MTTR is and how to calculate it, as well as other system performance metrics, through examples and best practices.
Table of contents
- What is mean time to repair (MTTR)?
- How to calculate system performance metrics
- System performance metrics examples
What is mean time to repair (MTTR)?
Mean time to repair (MTTR) is a business metric that measures the time required to identify a product failure and bring the product to its normal operating status. You measure MTTR from the moment you detect a problem, until everything has been resolved. This includes the time needed to diagnose, repair, and test the product.
MTTR provides you with valuable insights into the efficiency of repair work and gives you a sense of how quickly your team can respond to a problem. You want to have lower MTTR values, so that you can avoid customer complaints.
3 more system performance metrics
In addition to MTTR, there are three more system performance metrics that can help you monitor the health of your product:
Mean time between failures (MTBF)
Mean time between failures helps you predict a product’s failure rate by measuring the average time between system failures. You usually only use MTBF for products that can be repaired and returned to operation. For product managers, MTBF provides you with a sense of your product’s reliability over time.
MTBF can be used to forecast future failures, guide maintenance schedules, and identify components of your product that frequently fail and need improvement.
Mean time to acknowledge (MTTA)
Mean time to acknowledge (MTTA) tells you how much time passes between when you learn about a problem and when you actually start working on the problem. This metric gives you a sense of how long it takes your team to respond to an issue.
The better your observability is, the higher the probability that you have alerting mechanisms in place which can provide early indications that something might become a problem.
Mean time to failure (MTTF)
Mean time to failure (MTTF) tells you the average time a product can perform before suffering a non-repairable failure. As a product manager, MTTF allows you to estimate the longevity and reliability of a product that cannot be repaired once it fails.
A higher MTTF indicates a longer lifespan and this information provides you with valuable insights into warranty and lifecycle planning.
How to calculate system performance metrics
Now let’s walk through how to calculate the system performance metrics. In each case, we will need to have enough data available to calculate a statistical mean:
MTBF
To calculate MTBF, you need to:
- Reach agreement on what “failure” means
- Decide on the set of devices, services, components to calculate MTBF for
- Decide on the period to evaluate (six months, one year, three years, and so on)
- Gather data for the two measurables:
- Operational hours
- Failures (incidents)
- Run the MTBF calculation, by dividing the operational hours by the failures
To perform a simple sample calculation, let’s say you’re looking at a 1,000-hour period for operational hours, during which there were two failures.
Learn more →Based on that data, you divide 1,000 (operational hours) by 2 (failures), which gives you a MTBF of 500 hours.
Note: When calculating operational hours, you’ll need to be sure to exclude any planned maintenance windows from the calculation, because MTBF focuses on periods of unexpected down time.
MTTR
To calculate MTTR, you need to:
- Reach an agreement on what “repair” means
- Decide on the set of devices, services, components to calculate MTTR for
- Decide on the period to evaluate (six months, one year, three years, and so on)
- Gather data for the two measurables:
- Repair hours
- Repairs
- Run the MTTR calculation, by dividing the repair hours by the repairs
To perform a simple sample calculation, let’s say you’re looking at total repair hours of 3, during which there were 6 repairs.
Based on that data, you divide 3 (repair hours) by 6 (repairs), which gives you a MTTR of 0.5 hours.
MTTA
To calculate MTTA, you need to:
- Reach agreement on what “acknowledge” means.
- Decide on the set of devices, services, components to calculate MTTA for
- Decide on the period to evaluate (six months, one year, three years, and so on)
- Gather data for the two measurables:
- Acknowledgement time
- Repairs
- Run the MTTA calculation, by dividing the acknowledgement time by the repairs
To perform a simple sample calculation, let’s say you’re looking at a total acknowledgement time of 1.5, during which there were 6 repairs.
Based on that data, you divide 1.5 (acknowledge hours) by 6 (repairs), which gives you a MTTA of 0.25 hours.
Note: When considering how to calculate this metric, it’s especially important for you to have the same understanding of what constitutes an alert and what constitutes an acknowledgement of an alert.
MTTF
To calculate MTTF, you need to:
- Reach agreement on what “operational hours” (up time) means
- Decide on the set of devices, services, components to calculate MTTF for
- Decide on the period to evaluate (six months, one year, three years, and so on)
- Gather data for the two measurables:
- Operational hours
- Entities (devices/components/services)
- Run the MTTF calculation, by dividing the operational hours by the entities
To perform a simple sample calculation, let’s say you’re looking at total operational hours of 800,000, for which there were 20 entities.
Based on that data, you divide 800,000 (operational hours) by 20 (entities), which gives you a MTTF of 10,000 hours.
System performance metrics examples
Now that you know how to calculate system performance metrics, let’s explore how you might use each of these metrics in practice:
- Mean time between failures (MTBF)
- Mean time to respond (MTTR)
- Mean time to acknowledge (MTTA)
- Mean time to failure (MTTF)
Mean time between failures
MTBF is particularly helpful when evaluating the reliability and availability of your systems. For example:
- It can be helpful for planning purposes, as a means of reducing your organizational costs by implementing an effective preventative maintenance schedule
- Whether managing infrastructure yourself, or with one or more partners, having MTBF data available makes it easier to perform inventory management, so that you have enough spare devices and components, but also to avoid going overboard
- For internal teams, having MTBF data available is useful as part of a general path to continuous improvement, where it’s part of a larger narrative about lessons learned and root cause analysis, and also can provide input into employee onboarding guidance and surface areas
Mean time to respond
During periods of restructuring, which often involves a need to “do more with less,” you may see a negative impact on metrics such as MTTR. Examples of practical application of MTTR include:
- Surfacing areas where improved observability is necessary, by evaluating gaps in instrumentation and putting in place mechanisms to address those gaps
- Looking for patterns in repair items that emerge from incidents, where it may be possible to shrink incident response times by addressing areas where technical debt is known to exist
- Using MTTR data to consider modifications to on-call procedures and to evaluating ongoing staff needs, to help avoid employee burn-out
Mean time to acknowledge
The practical application of MTTA data is similar to MTTR data, in that worsening MTTA numbers may point to things like “alert fatigue.” Examples of additional insights that you can gain from MTTA data include:
- Making decisions about network architecture, where scalability and elasticity are key considerations. Depending on the circumstances, you may need to focus more on horizontal scaling or vertical scaling, where:
- Horizontal scaling means throwing more assets (nodes or machines) at a problem, often taken as an emergency measure
- Vertical scaling means making individual assets more capable of handling load, by upgrading machine/device processing power, storage, or memory, as part of medium to long-term planning
- Addressing any gaps in tooling that might exist, to provide better visibility and efficiency when parsing large volumes of data, such as log streams
- Surfacing integration challenges, where usage of different tools and systems may make it more difficult to get a quick read on what is happening in real time
Mean time to failure
MTTF data is particularly helpful as a means of assessing the relative likelihood of failures for one type of device or sub-component versus another. For example:
- Applying what you learn from analysis of MTTF data as a justification for revisiting the SLAs and similar arrangements we have in place with third parties
- In cases where you employ devices or sub-components from multiple manufacturers in infrastructure that we maintain ourselves, adjusting our future buying decisions as necessary
- Making horizontal or vertical scaling decisions to help minimize the load on machines and devices that are part of our cloud infrastructure
Conclusion
Success with any software product requires that you pay attention to multiple areas. Even if you invest heavily in usability, if you underinvest in other areas, such as the infrastructure and people, you jeopardize your ability to achieve long-term product success.
To summarize, you can:
- Use mean time between failures (MTBF) to help make big-picture decisions about spending on infrastructure
- Use mean time to repair (MTTR) to make informed decisions about IT staffing levels, partner agreements, and any adjustments that may be needed to operational procedures
- Use mean time to acknowledge (MTTA) to take a closer look at any gaps that might exist in network infrastructure or tooling
- Use mean time to failure (MTTF) to better understand the full lifecycle that we can reasonably expect from individual devices, components, and services that are part of your infrastructure
Featured image source: IconScout
LogRocket generates product insights that lead to meaningful action
LogRocket identifies friction points in the user experience so you can make informed decisions about product and design changes that must happen to hit your goals.
With LogRocket, you can understand the scope of the issues affecting your product and prioritize the changes that need to be made. LogRocket simplifies workflows by allowing Engineering, Product, UX, and Design teams to work from the same data as you, eliminating any confusion about what needs to be done.
Get your teams on the same page — try LogRocket today.
