Validation research techniques to test value and viability assumptions

When it comes to designing a new product, there are a number of decisions that need to be made on the basis of questions you answer among your team. Some of these might be things like “Why do we think our customers want these solutions?” or “Are there potential impacts of our product?” While you would like to have an objective answer, many of these require you to lean into what your gut tells you.

To do this, you’ll likely have to make value and viability assumptions. This article will help you understand the different types of assumptions, explain how to figure out which ideas will be successful, and provide you with methods for testing.

What is an assumption?

An assumption is something (conviction or belief) that you believe to be true or certain, with no proof of it being true. In the product world, an assumption must be true for our ideas to be successful. Because of this, assumptions can be dangerous and costly if they turn out to be incorrect.

That said, ultimately assumptions are often at the center of product decisions. Rather than cowering from them, you can develop an understanding of the different types and work on strengthening the accuracy of your assumptions.

5 different types of assumptions

In product development, assumptions usually can be categorized into five different types.

Viability assumption

With a viability assumption you answer questions like:

• Will this product or service be successful in the market?
• Why do we think a solution will be good for the business?

This assumption considers factors like market need, competition, and potential revenue streams.

Value (or desirability) assumption

In this case, you’re answering questions like:

• Will customers want and use a product or service?
• Why do we think our customers want this solution?
• Why do we think they’ll be willing to do what we need them to do to get value from it?

This type of assumption focuses on understanding customer needs, pain points, and motivations. It also explores whether customers will be willing to pay for the solution or adopt any necessary behaviors.

Feasibility assumption

Here, you would answer questions like:

• Can we build this product or service with our current resources and capabilities?
• Why do we think we can build this solution?

The focus is on the technical expertise and resources required to create the solution. It considers whether existing technology can support the product and if the development timeline is realistic.

Usability assumption

When it comes to usability assumptions the conversation would focus on:

• Can customers easily learn and use this product or service?
• Why do we think customers will be able to use a solution?

This assumption analyzes the user interface (UI) and user experience (UX) to ensure the solution is intuitive and user-friendly.

Ethical assumption

With ethics, you would deal with questioning like:

• Are there any potential negative impacts associated with building this product/service/solution?
• Do we ensure that this solution is accessible to all users, regardless of their socioeconomic status or physical abilities?

This assumption ensures the solution aligns with ethical principles and avoids causing harm to users, society, or the environment.

How to figure out which ideas fly or flop?

To figure out which ideas will fly versus flop, you need to know how to write assumption statements.

Then, in step two, you take your assumptions statements and map them on a two by two diagram. You should develop testing for the assumptions that fall within the top right quadrant. As a final step, you analyze the results and make a decision.

Now, let’s dive further into each step:

1. Writing assumption statements

Does it matter how you phrase an assumption? Well, it does matter a lot because the phrasing will affect how easy the assumption is to test.

A couple of things to keep in mind:

1. Phrase the assumptions in a way that they need to be true for your idea to be successful. For instance, every app requires a login, right? So, the best way to phrase an assumption is that “Users will remember their username and password.” Don’t phrase it as, “Users will forget their username and password.” The reason is that it’s easier to test if users will do something than to test if they won’t
2. Be specific. The more specific your assumption is, the smaller and quicker the test is. It’ll let you move fast

2. Assumption mapping

Then, you’ll take all of your assumption statements and map them in a two by two matrix.

To understand the risk and uncertainty of an idea you should ask, “What are the things that must be true for this idea to work/to be successful?” This will allow you to identify all five assumption types underlying a business idea: viability, value (desirability), feasibility, usability, and ethical considerations.

Before diving into any development, it’s important to work with valid assumptions. And, using assumption mapping can make a big difference in increasing the success of a product and its features.

Here’s the skeleton of an assumption map:

On the x-axis, you use the labels “have evidence” and “no evidence” because it all comes down to evidence. It’s about having observable data (a piece of evidence), qualitative or quantitative, to support your assumption.

On the y-axis, you use the labels “important”’ and “unimportant.” You might be thinking, “Well, we only have to write down the important assumptions, so this is redundant.” However, not everything is the most critical. By asking ourselves, “Which hypothesis, if proven wrong, will cause your idea to fail?” you can determine which are the most important assumptions.

Once you have this, focus on the top right quadrant! Your assumptions that are critical for success and yet have the least amount of evidence to support them, focus on those.

3. Develop tests

Doing assumption testing exercises regularly isn’t easy because it requires a mindset shift. It’s easy to think that if you use a product/feature, others would use it too.

You already know that ​​you make better decisions when you consider more than one option. That’s the same in product development too. That’s why you shop around when looking for a new job, a new place to rent, or a property to buy.

But at work, it’s a bit more challenging. When you hear about a user’s pain point, you usually jump right into the first solution that comes to mind based on your understanding of the world and the problem. After all, you’re problem solvers and like to go ahead with your guts and ideas.

But how do you know this is the best solution we can do if we don’t compare multiple options?

You just don’t know! Imagine if you rented a place like that. That’d be insane! But why do you tend to go ahead with the first solution anyways?

Maybe you don’t think you have enough time to consider more than one option within the constraints you have. Or, maybe you think assumption testing is time-consuming and slows down the team.

However, you can still move fast while considering options. The key is to stop testing whole ideas and just start testing bite-size pieces that your idea depends upon. Assumption testing allows you to evaluate which ideas will work and remove the ones that will not.

If you have a computer science background, the following idea will be familiar because it’s similar to setting up a unit test for your code:

1. Set up a condition for executing an assumption test
2. If the assumption is validated, develop/release the tested feature
3. If the assumption is falsified, don’t develop/release the tested feature

4. Analyze results

Finally, you need to synthesize the results from the test.

If the data confirms your assumptions, you can move forward with product development. If the data falsifies/contradicts your assumptions, you can pivot or stop focusing on that solution:

The example above highlights how assumption testing with features like personalized recommendations can help e-commerce websites validate or falsify customer behavior before a large investment is made in platform development.

You might be thinking, well, you don’t always have that much time and resources to develop a feature for A/B testing, right? If you’re thinking that, I’m glad, because I want to give you four more methods with low investment.

Low-investment methods for assumption testing

You know that untested assumptions about an idea, or feature can lead to unsuccessful or even disastrous results. So, how can you test assumptions to increase the chance of success? The answer is ask, show, offer, and deliver — a concept developed by Keith Hopper.

These represent the ways you can interact with your users to validate or falsify your assumptions and fall along a spectrum based on your level of certainty/evidence about the assumption.

Ask and show are ideal for the early stages when the concept is still evolving and when you don’t have lots of certainty about the idea. For example, when you have lots of ideas about what might be valuable for customers, but you have too many options and don’t know what would offer the most value.

Then, offer and deliver are more appropriate when you have a clearer direction and when you have more certainty about the direction, but you need to validate your idea/assumption to be sure users will use it.

The method involves structured interviews with individual customers. Here, the goal is to understand what truly matters to them. By asking specific questions, you can identify customer pain points and opportunities for innovation and what is meaningful for users.

When you’re at the ask stage, you can cover questions such as:

• What are our customers really struggling with, and how powerful is that struggle?
• What differentiates our customers from one another, and who should we target?
• What insights might inform solutions to their problems?

So, if you don’t have answers to these but have a bunch of assumptions about them, this is where you can start.

Go into the session with structured questions, so you don’t end up with casual conversations.

Always try to dig deeper, a couple of layers down, to have valuable data.

How to judge the strength of evidence

During assumption testing, the quality of the information you gather is just as important as the information itself. This is where the concept of evidence strength comes in. Just like in scientific research, not all evidence is created equal, not all evidence has the same strength.

Some insights provide a stronger foundation for decision-making than others. The ladder of evidence framework helps you prioritize user insights based on their reliability:

The weak evidence is at the bottom of the ladder and usually, what users say they do or want. These include surveys, interviews, and focus groups. While valuable for gathering initial ideas and understanding broad sentiment, these insights can be susceptible to biases such as social desirability (users reporting what they think is expected) and memory limitations.

As you ascend the ladder, the evidence becomes stronger and more actionable. Here, you find evidence based on what users do, providing a more objective picture of their behavior and needs.

Examples of strong evidence methods include:

• Observational research — Watching users interact with a product or service in their natural environment reveals their true behaviors and challenges
• Usability testing — Observing users complete specific tasks with a prototype or product allows us to identify usability issues firsthand
• A/B testing of a prototype — Testing different design variations with real users helps us determine which approach is most effective in driving desired behaviors

What is the best time to run these assumption-testing exercises?

My answer is as early as possible AND definitely not at the end of the product development process.

Here’s why.

Lots of product people go to designers and researchers saying, “Hey, could you validate this [idea] for me/could you run a quick user study, you know, just to validate this assumption?”

That’s user-centered performance. It’s too late to matter. It’s just a performative exercise regardless of the outcome.

When people do this, they aren’t interested in being wrong. Not at all. Because it’s too late for that. The feature is very likely to have already been scheduled for release. What people want is to check the box. And yes, it might be well-intentioned, but it’s just too late. So, do it as early as possible.

Key takeaways

If I can leave you with one thing, I want to leave you with, “Don’t validate! Falsify.”

Have a mindset shift, and decide that you want to be wrong with my assumptions! You want to do assumption testing that shows you how wrong my assumptions were in different ways.

That’s how you should approach insights, assumption testing, and research in general.

But lots of people don’t approach research this way. They want to validate, they want to be right.

So, be ready to be wrong! That’s where real learning lives.