A guide to understanding the order of magnitude

The job of a product manager is to solve problems for the business and the user.

As you mature in your career, you unlock more and more ways of figuring out where the potential problems could be, such as user interviews, analytics, AB tests, etc.

The longer you work on a product, the more problems you’ll be aware of. However, the trickiest part of this job is figuring out which problems to solve and in what order.

Suppose you want to have an impact on your business and your users. In that case, you need to understand the potential order of magnitude of each problem so that you can easily ballpark how your possible solutions might drive your overall goals and metrics.

In this article, you will learn what the order of magnitude is and how you can prioritize your problem solving to achieve the best results for your product.

What is the order of magnitude?

The order of magnitude is the ability to guess the rough size of something before you know for sure.

Being able to guess the order of magnitude is a key skill in product management because you can’t know the exact impact of a project before you start working on it.

However, you should be able to roughly guess if it will be a large, medium, or small win.

There are multiple frameworks for order of magnitude estimation, such as:

• Tee Shirt Sizing — Small / medium / large / extra large — where you are sizing things relative to each other
• Fibonacci points — Where each point has to be a Fibonacci number, so 1, 2, 3, 5, 8, etc.
• Basis 100 Estimation — You have 100 points to get across all your projects, and your goal is to distribute these 100 points based on the expected impact

These systems work because our brains are much better at understanding how big something is in relation to something else versus being able to guess its size exactly.

Why does the order of magnitude matter?

When building a road map as a product manager, you’re frequently tasked with delivering a certain level of impact against an OKR or KPI, or however your company does goal setting.

These goals are often tied to a financial plan that involves growing the business to hit certain

milestones that are either public or private but material to the company.

You want to be able to choose the problems and solutions to these problems to give yourself the best likelihood of hitting these targets.

All product managers have a finite amount of resources, typically engineering hours, design hours, etc., that they can allocate to projects with the hope of hitting these targets.

If you need to increase your primary KPI by 10 percent, you can only afford to spend 70 percent of your engineering hours on a project that will increase it by 1 percent.

This sounds simple to say, but hands down, the most common mistake of new product managers is putting too much effort into solving a minor problem that only impacts a few users.

Estimating with the order of magnitude vs. exact values

Estimating impact most commonly happens during your planning process, be it quarterly, six-month, annual, etc.

To be extra sure of your estimation, it’s tempting to build a complex quantitative model to predict the potential impact of your projects.

This is typically a mistake for a few reasons:

• It’s too slow — Planning is one of the busiest phases of each year for product managers, and you won’t have time to take on another complex task
• It’s too inaccurate — It’s very counterintuitive, but because you’re estimating something that’s not known, the more variables you include in prediction, the more likely that minor errors in each will magnify each other. You’ll end up with an even more inaccurate prediction
• Lack of data availability — Even at the best companies, there are tracking problems, and the more specific you try to make your estimation, the more likely you will be searching for data that is either hard to find or not entirely trustworthy

Estimating with the order of magnitude solves all of these problems because all you have to do is stack rank the potential projects on your roadmap in order of potential impact.

This is faster and more accurate than trying to get into the weeds and finely estimate what each project will do.

When you look at the bigger picture, your overall velocity as a team is what matters. You learn more by shipping more things, so the faster you can make your overall process, the more you’ll learn in the end.

Examples of the order of magnitude

As mentioned above, there are many ways of using order of magnitude estimation, however the most common one in the industry is t-shirt sizing.

The process of using t-shirt sizing works like this:

1. Take all of the potential projects that could be on your roadmap
2. Dump all of those projects into a spreadsheet with enough details that you, your engineering manager, and your data scientist will understand what each of those projects are
3. Create two columns, one for “impact” and one for “cost”
4. Start with the impact column, review your goals as a team for the upcoming period, and walk through each project with your team. You can vote either out loud or asynchronously on what you think the project’s impacts will be
5. Repeat this process for project cost again, focusing anywhere where there is disagreement on the estimated cost
6. Sort this list to prioritize the highest impact and lowest cost first in the quarter

This is a very high-level summary of roadmap building. A few nuances that I picked up over the years:

• While each team member should get a vote, the tiebreaker should always be the team member with the best knowledge of that area
• It’s more important that you get projects roughly bucketed into groups of impact than it is to assign potential ranges of impact to each T-shirt size. You’ll frequently get a question about what each size “means”

Common pitfalls and solutions

They’re multiple ways that estimation can go wrong. Here are some tips that I found over the years that make this easier:

• Look at your metrics daily to build understanding — Hands down, the best way to improve your ability to ballpark the impact of projects is by understanding your product’s metrics and key drivers
• Understand the user life cycle — Understand when in the life cycle of a user they are likely to take specific actions
• Use common sense — A good ballpark for how often a user will do something is how frequently people did that behavior before the internet existed. You’re not going to be able to ship features that will push a user to do something more than they naturally want to
• Be conservative in your estimates — For all projects, especially the ones that the team is really excited about, be conservative about what you think the impact will be

Tips for using the order of magnitude with stakeholders

Frequently, estimation is done during the planning cycle, and projects might not be launched for multiple months after that.

If you put yourself in the shoes of your VP of Finance, they may have completely forgotten about the roadmap that they reviewed five months ago by the time your big project ships.

A lot else is happening in their world, and their attention is pulled in multiple directions.

When reviewing a feature’s impact, it’s helpful to remind stakeholders what you initially estimated this project would deliver, as their understanding might differ from yours.

You might have shipped a “medium” impact project that you thought would be “small.”

Your VP of Finance might have been expecting a “large” impact, and setting this context for them will be very helpful in making sure your projects are getting the attention that they deserve.

Additionally, any artifacts you’re producing, such as OKRs, product specs, etc., should link back to the estimation work and the ballpark expected impact.

When reviewing these artifacts, I ensure that any columns containing these ballpark costs and impact numbers are viewable by default.

Conclusion

Correctly being able to ballpark the impact and cost of projects through an order of magnitude estimation system allows you to help your team ship high-quality projects and focus them on the most critical areas of the application.

It’s less important which estimation framework you choose to convey the order of magnitude of your projects; what’s important is that you’re choosing one and getting comfortable with it.

Additionally, the more you familiarize yourself with how the product works, your metrics, and the problem space you’re within, the easier and faster you’ll be able to ballpark the impact of the work your team is doing.

Featured image source: IconScout