How to add a harm score to product prioritization

Working on public-facing products and services has taught me that one wrong product decision can have a significant knock-on effort on a real household. Australia’s Robodebt scheme showed how much damage an automated decision can do when a system gets a person wrong.

In case you’re not familiar, Robodebt used annual tax data to estimate welfare recipients’ bi-monthly income and raised debts based on those estimates. If you only looked at the projected savings and scale of automation, it might resemble a strong prioritization bet. The upside was easy to quantify, however the severity of assigning a false debt to someone who couldn’t easily challenge it wasn’t.

The 2023 Royal Commission into the Robodebt Scheme called it a “crude and cruel mechanism” that was “neither fair nor legal.” In total, the scheme affected hundreds of thousands of people and caused severe financial and psychological distress.

That blind spot appears anywhere a product touches money, identity, health, safety, or legal status. To help mitigate this, I added a harm score to my product prioritization. A harm score rates the severity, reversibility, vulnerability, and recoverability of a feature’s worst realistic error, then changes what the team builds, gates, or refuses to trade off.

Why product prioritization frameworks can miss user harm

Most prioritization frameworks compare expected value with the cost of delivering it. RICE multiplies reach, impact, and confidence, then divides the result by effort. ICE and impact-effort models make similar value-versus-cost trade-offs. Other frameworks work differently. WSJF includes risk reduction in its calculation, while MoSCoW can classify legal or safety requirements as must-haves.

These frameworks can accommodate risk, but they don’t require you to account for serious harm that affects only a small group of users. You still have to identify that harm and express it as value, urgency, or necessity before the framework can account for it.

RICE makes the blind spot especially easy to see. Confidence adjusts for uncertainty in the estimates. However, it doesn’t measure how much harm a feature could cause if it works as designed but acts on the wrong person.

A feature that helps 90 percent of users and seriously harms two percent earns most of its score from the 90 percent. Any protection built for the smaller group appears to have low reach and falls down the backlog. In my own RICE exercises, safeguards for the most exposed users repeatedly landed near the bottom because those users were also the fewest.

Low reach can hide severe consequences

Benefits and routine product friction tend to spread across many users. A faster page or simpler flow helps a large group a little. On the other hand, severe harm tends to reach fewer people but affects each one far more.

Over 200k developers and product managers use LogRocket to create better digital experiences

Learn more →

This pattern appears well outside of government. In a 2021 letter to the Consumer Financial Protection Bureau, then-Senate Banking Committee Chair Sherrod Brown described reports that Chime had closed or frozen accounts over suspected fraud involving federal stimulus checks and unemployment insurance deposits, often without warning.

For a customer living close to the edge, losing access to that money could mean missed bills, late fees, damaged credit, or the risk of eviction. Yet a safeguard such as rapid human review might appear to have low reach because only customers caught by mistake would need it.

Reach asks how many people a feature affects. Consequence asks how badly each person could be affected and whether they can recover. You need a product framework that hears both sides of the equation.

Build a harm score with four factors

A harm score captures what an upside-focused model leaves out. It’s a decision trigger, not a precise prediction of how much harm will occur. I score four factors from one to five based on the worst realistic case where a product gets a person wrong.

I use 1 for low, 3 for moderate, and 5 for critical. Scores of 2 and 4 cover the cases between those anchors.

Reversibility and recoverability sound similar, but they measure different things. Reversibility asks whether the system can undo its action, whereas recoverability asks whether the person can return to their original position after it does.

Use the highest of the four ratings as the overall harm score. Make sure you don’t just average them. A five in severity shouldn’t disappear among lower scores for the other factors.

If four ratings feel like too much for an initial backlog review, I compress them into one question I ask out loud: If we are wrong about this person, who pays, and can they recover?

When a specific person could pay in money, time, health, access, or standing, I score the item regardless of its reach.

Modify, gate, or escalate based on harm

A harm score earns its place only if it changes the decision. I use the overall score to trigger one of three responses:

1. Low harm, scored 1 or 2 — Modify the design. Document the likely failure, add a proportionate safeguard, and let the feature compete through the normal prioritization process. This covers most of the backlog
2. Material harm, scored 3 or 4 — Gate the launch. The feature cannot ship until its safeguards ship with it. Those safeguards might include human review before an irreversible action, clear notice, an accessible appeal process, or a firm reversal deadline. The safeguard becomes part of the definition of done rather than a follow-up item
3. Critical harm, scored 5 — Set it apart. Route the item through a separate review governed by a written standard and owned by a named decision-maker. That reviewer can block the feature, require a redesign, or impose a non-negotiable constraint

That last lane is structural, and I learned why it matters the slow way. Put critical harm in the same calculation as a conversion lift, and enough projected upside can overwhelm it. A separate lane prevents that trade-off by giving the harm score veto power when the worst realistic outcome crosses a boundary the team has agreed not to accept.

How harm scoring changes a RICE decision

As an example, consider the kind of decision I face every quarter. A team has three candidate items scored with RICE:

Looking at RICE, there’s a clear order: B, A, then C. The issue isn’t that B comes first, but that A can be shipped as a standalone capability while C, its safeguard, remains at the bottom of the list. As a result, the capability launches while the protection is still waiting.

The harm review adds the failure case that the RICE table leaves out. Assume the account-freezing rule can misread irregular deposits, including gig income and benefit payments, and that reversing an error can take several days:

An overall score of 5 moves A into the critical-harm lane because the team can no longer prioritize it as a standalone item. It must reject or redesign the feature, or pair it with C and estimate the combined work.

B may still come first based on its RICE score. The difference is that C no longer competes against A as a low-reach feature. It becomes a dependency of A.

Unlike RICE, a harm score pulls the safeguard out of the backlog and binds it to the capability, so the brake and engine arrive together.

Add a harm review to your product backlog

You can put this to work this week without rebuilding your prioritization process:

1. Add a harm review column to the sheet you already use
2. For every item touching money, identity, health, safety, or legal status, ask: “If we are wrong about this person, who pays, and can they recover?”
3. If the answer identifies a concrete cost to the user, score the four harm factors in the item’s supporting details and record the highest rating as its overall harm score
4. Route the result. Scores of 1 or 2 require a proportionate design change. Scores of 3 or 4 gate the launch. A score of 5 enters the critical-harm lane for separate review
5. Name an owner for every gated or critical item. Review that lane regularly so it remains a decision mechanism rather than a holding area

More often than not, the math around your upside will favor the many over the few and the broad capability over the narrow safeguard. Instead of replacing RICE or another prioritization framework, a harem score adds a decision layer for consequences those frameworks may not capture.

Think of it like giving you the missing number while the decision is still a line on a roadmap. The process only costs a column and a conversation. Skipping it can cost users money, access, health, and trust that no product rollback can fully restore.

Featured image source: IconScout

LogRocket generates product insights that lead to meaningful action

LogRocket identifies friction points in the user experience so you can make informed decisions about product and design changes that must happen to hit your goals.

With LogRocket, you can understand the scope of the issues affecting your product and prioritize the changes that need to be made. LogRocket simplifies workflows by allowing Engineering, Product, UX, and Design teams to work from the same data as you, eliminating any confusion about what needs to be done.

Get your teams on the same page — try LogRocket today.