How to measure success after a product launch

Product managers use many key performance indicators (KPIs) to gauge the success of a product, but using them all at once can make you feel overpowered and disoriented.

Consider Netflix, for example. You should look at Netflix’s daily and monthly active user (DAU and MAU) statistics to determine whether the service is successful or not. This is because the main function of Netflix is to stream films and series to replace cable television. Its success depends on active users because people who are streaming on Netflix are not watching cable TV.

On the other hand, a hotel’s success cannot be measured by daily active users. Instead, the essential indicators for a hotel recommendation app would be Net Promoter Score (NPS).

The point is that the success of a product is subjective and not one-size-fits-all. In this guide, we’ll show you how to define what success looks like for your product and how to measure it post-launch.

What is a product launch?

To be clear, when we introduce our product, the objective is not to increase user engagement or to create a buzz. Instead, we ought to use product launches to get through the beta stage and determine whether the product is well-accepted in the wild without the help of the product team.

In this context, product teams frequently use beta tests or pilot groups to measure a product’s performance by obtaining feedback from users. The problem is that they’re dealing with a captive audience of users who have strong incentives to continue using the product.

The purpose of a product launch is to determine whether the product will succeed without the direct engagement of the product team. Therefore, it is important to remember that beta tests and pilot programs are not product launches.

This information allows us to precisely define a product launch as a significant release of a product to the market, where the product team does not have sustained direct engagement with users.

How does a product launch differ from mature products?

Building on top of an existing product is fundamentally different from launching a new product because the latter requires operating under a great deal of uncertainty.

You’ll experience ambiguity specifically in the following areas:

• Absence of historical user behavior information
• Insufficient proof that your target audience is the proper one
• Insufficient proof that you have chosen the right problem to target
• Insufficient proof that the value proposition and feature set you provide are in line with what the audience wants

Because of these variations, we wish to use particular sets of product indicators to assess the success of our product launch. Because the purpose of employing metrics is to demonstrate progress toward qualitative goals, our goal is to ascertain whether or not this solution or product resolves problems for the intended audience.

Metrics to measure success post-launch

There are three main categories of metrics to keep in mind when thinking about launching a new product:

• Conversion
• Retention
• Referral

Each of these measures demonstrates whether the target market we’ve chosen believes our product resonates with them and whether it is appealing enough for them to adopt and ultimately promote to others.

Conversion

When considering conversion, it’s important to keep in mind that freshly released products must be monetized in some way. This means that we have a desired action we want consumers to perform.

Every product must generate value for your business, and the best way to do this is to specify, in your product flow, which user activity is of the utmost importance. Therefore, it’s crucial to determine the percent conversion rate for the intended activity.

An important thing to remember is that the desired action can vary between products. For instance, Google Search is available to users without charge, but marketers must pay to appear in these searches. How frequently consumers search on Google is therefore the main conversion to track for Google’s search product because it facilitates monetization later on.

Some people might define conversion as engagement, which means they’re stating users are using the product and acting in some way. It’s perfectly acceptable to use that, but we must be careful to avoid treating all participation equally.

Based on the functionality of your product, choose the activity that adds the greatest value, and then assess how frequently that feature is utilized. Remember to measure it relative to the user base as well as in absolute terms (how many times it is used per week).

Retention

Retention is important to gauge whether users have genuinely embraced the product and altered their behavior. Usage and behavioral repetition are what we’re looking for. We can rapidly ascertain whether our product is genuinely increasing engagement by examining the churn rates of various cohorts of consumers.

Overall engagement is another option to measure success. The issue with total-level engagement indicators, such as weekly active users (WAU), is that they can go up if there are more new users than lost users..

However, overall engagement doesn’t indicate whether recent users have grown accustomed to the product over time. Therefore, we want to calculate the percentage of people who return within a certain time frame (e.g., every week or every month). Since you have more time to reactivate customers, in general, the longer the time period, the more people are likely to return to your product.

Note: Only compare seven-day retention in months one and two; do not compare seven-day retention in month one with 30-day retention in month two to ensure that your comparisons are fair. The idea is to compare oranges to oranges and not apples to oranges.

Referral

Referral metrics are effective tools for assessing whether your product is useful enough to generate popularity and word-of-mouth. In general, we don’t want to use net promoter score (NPS) as a leading predictor of referral metrics. NPS takes time to resolve and we don’t always want to saturate consumers with surveys as soon as the product is released.

Instead of having them use the product just once and become discouraged when a survey appears, we’d love to see them use it effectively several times. However, one thing we can do right now is provide users with the option to invite other users into the platform, whether they do so through email invites, social network posts, or any other method.

We’ll want to make sure we have this kind of capability since it can show that customers are finding our product to be so valuable that they’re ready to risk their social capital to promote it. To determine how frequently consumers invite other users to the product, we can measure the invite rate. In the coming months, we want to see the invite rate gradually rise along with the conversion rate and retention rate.

How should you prioritize these metrics?

Conversion should be given top priority among all other metrics. The goal of the product is to provide users with so much value that you can profit from their actions. A launched product won’t be able to grow unless you have means of monetization and conversion.

The product should be used to increase conversion rates as your first order of business. You can do this by removing friction from the main workflow, by automating data inputs, or by eliminating steps like account creation with a single-sign on implementation.

Linking your supporting workflows back to the main workflow is another approach to increase conversion rates. For example, let’s say your main workflow is a form where customers have to fill out information to receive mortgage quotes from various lenders. You might have a secondary workflow that explains interest rates to customers, or one that explains the value of their current home.

Make sure the end of those auxiliary workflows point to the major workflow so that the end user completes the action you want them to. For example, when you initially use Twitter, you are prompted to post your interests. The main conversion activity that generates value for Twitter as a business is the incentive to like the various tweets that show up on your feed when you’ve added enough interest.

You’ll want to make sure that you’re obtaining retention after you’ve managed to convert usage into income. In this method, you can be sure that your product is actively being used and isn’t just being used for its novelty. Your product launch’s primary goal is to create long-lasting, sustainable value.

Consider Clubhouse as an example of what happens when you just concentrate on conversion without taking retention into account. Although it had extremely high initial conversion rates for its main feature, creating audio talk rooms, it was unable to keep them for extended periods of time. This had a big negative impact on the company’s bottom line.

Finally, you should consider raising referral metrics. Referrals demonstrate that user engagement in your product is valuable enough for consumers to recommend it to other users. Word-of-mouth is still one of the biggest engines of organic growth for any product.

Other things to consider during a product launch

For recently launched items, in particular, qualitative input is essential.

The present moment is the best time to interact deeply with qualitative information like support tickets, comments, and user interviews because you’ll likely have a lot fewer users than you will in the future.

To ascertain whether you are marketing to the appropriate group of people and whether they are aware of the benefits of your product, you have to analyze the qualitative feedback. You want to know if you’ve successfully positioned your product to meet their demands and if they understand how your solution would solve their intended problem points.

Use this qualitative feedback to pinpoint any potential additional product flaws. Rapid iteration can then be used to quickly close these gaps, which should ultimately have an influence on conversion rates, retention rates, and referral rates.

Conclusion

It is key to incorporate the above metrics into the product development process. To ensure that your product will succeed in conversion, retention, and referral, you should ideally begin with design before writing a single line of code.

Additionally, don’t forget to collaborate with engineering to instrument these metrics before releasing your product into a live environment. After all, if you don’t have an appropriate analytics method , you won’t be able to track any measurements.

Working backwards ensures that our solutions will produce sustainable value for both our end users and for our business stakeholders. Start with a clear design and that will translate into good conversion, retention, and referral rates.

Featured image source: IconScout