Have you ever purchased and used a product and thought to yourself, “Who designed this? Who approved it for release?” Either the product just didn’t make sense to you as a user or it was full of bugs and errors. Sound familiar?
Look I just have one question. And I’ve been on it since childhood. Who approved this product as tried and tested to be SUPER SOFT? I’m waiting for the answer paaaaa. 😔 pic.twitter.com/C0owZuB9Jz
— Francis Abban (@francisabban) September 27, 2019
When I am using this product I am so angry I want to punch the Product Manager who approved it so that I can have some measure of satisfaction from something. https://t.co/HwgkVrNORk
— Alan Cooper (@MrAlanCooper) June 10, 2020
If you haven’t felt the rage and confusion that results from using a terrible, buggy, or imprecise product, the tweets above illustrate that emotional toll quite well.
As a product manager, the last thing you want is to be punched. It hurts, both physically and in terms of your ego. That’s why you should always test your product before releasing it to the market.
In this guide, I’ll show you how to avoid punches and maybe even earn some hugs from your customers by doing beta testing.
There are many tests product managers can and should conduct to ensure the team is on course to deliver a stellar product. In no particular order, these activities include:
As a product manager, you shouldn’t feel pressured to do all these tests for every single release. Pick the most relevant tests based on available resources and the data you’re trying to get from the test.
PMs need to gather information at every stage of the product development process to help measure, learn, and then design the right solutions for their users.
Beta testing is an important activity to help product managers validate the product hypothesis and gather initial feedback from real-life users. It’s very common for product teams to be so close to the product that they’re unable to see issues or identify weird behavior.
Beta testing gives PMs a fresh pair of eyes to view the product differently. Best of all, these extra eyes belong to the actual users of the product, and they’ll most likely tell you their true feelings. This might hurt a bit at first, but it’s critical to the long-term viability of the product and the business.
Beta testing is a great way to anticipate potential issues and bugs before you launch your product to the wider market.
Beta testers are often better equipped to observe things product teams overlook during development and other testing sessions. This enables the product team to collect meaningful user feedback and input about the product they wouldn’t otherwise receive from internal stakeholders.
In addition, beta testing is a valuable exercise because it enables product teams to:
Beta testing is conducted in three phases:
The preparation and design phase involves the following steps:
You and your team should be clear on what exactly is being tested for and why. What doesn’t get measured doesn’t get managed.
Some examples of metrics and KPIs for beta testing:
A common mistake PMs make is setting goals without the input of stakeholders.
You can get away with this for QA and AB tests, but beta tests require the input of customer-facing teams that are in touch with the users constantly. Naturally, these teams are well-positioned to receive customer feedback that is valuable to the product team.
In addition, the results of a beta test might dictate changes to the way operations teams work, so it’s always good to keep them in the loop from the get-go.
A beta test shouldn’t run for too long. Failure to properly time your beta test can result in skewed data and a decline in user interest. Not to mention, you have a product launch to plan.
At the same time, a beta test shouldn’t be too short; it needs to be long enough to allow you to collect data properly. I recommend between 14–60 days to beta test for a typical feature. The industry, goal, and type of product impacts this.
You don’t just let anyone into your space, so why would you let anyone join your beta group? PMs need to be specific about their target audience for a beta test.
Your product most likely has more than one user type, so you need to determine whether you want results from all user types or you’d rather focus on one user type.
I recommend recruiting a diverse, representative group of users in your beta test community.
Beta testers are a different breed. Where would product teams be without them? They are usually:
You can find beta users in the following places:
You might also want to have beta users sign legal T&Cs and NDAs, depending on the type of product you’re testing.
It is sometimes hard to find beta users. It’s a good practice to develop a pool of beta users you can reach out to over a period of time. That way, you don’t need to search for new users every time you want to beta test a product.
You should enable users to easily opt in or out of beta tests. If you have the resources, you might also consider offering an incentive. Ideally, this incentive should be something other than monetary compensation. People generally love free things, after all!
To save you from having to answer the same questions repeatedly during the beta testing period, you should prepare the following resources ahead of time for your beta users:
Beta tests happen before the product is launched which means the product shouldn’t be publicly available yet. You want to set up an environment that’s as close to production-ready and easily accessible as possible.
In product management, there are very few one-size-fits-all approaches. Beta testing is no exception.
There are myriad ways to do beta testing, depending on the type of product, what you are testing for, and the availability of your users. But as a general framework, here’s what you should do:
After you complete your beta test, you should take the following steps:
Featured image source: IconScout
LogRocket identifies friction points in the user experience so you can make informed decisions about product and design changes that must happen to hit your goals.
With LogRocket, you can understand the scope of the issues affecting your product and prioritize the changes that need to be made. LogRocket simplifies workflows by allowing Engineering, Product, UX, and Design teams to work from the same data as you, eliminating any confusion about what needs to be done.
Get your teams on the same page — try LogRocket today.
While rapid iteration and releases provide an edge, undue haste can result in technical debt, bad user experiences, and team burnout.
Jean-Yves Simon, Chief Product Officer at AB Tasty, discusses the importance of solving a need users don’t even know they have.
Joyce Schofield, VP of Product & Design at Dispatch, shares her approach to product strategy when designing for a two-sided marketplace.
Learn more about how the leading product leaders are using the 5 Whys framework, as well as tips for you can use it within your own team.
Learn more →
