What is exploratory testing? Benefits and how it works

When it comes to introducing a new feature for a software product, so much time is invested in defining requirements and determining a solution.

There’s nothing worse than releasing a new feature only to find a bug in it immediately after launch, and that’s where testing comes in. You really need to ensure that the feature is thoroughly tested to make it a success.

There’s a whole host of different approaches to testing software, but we’re going to look at exploratory testing specifically. We’ll learn what it is, what it can deliver for you, and when the best times to use it are.

What is exploratory testing?

Exploratory testing, as the name suggests, isn’t planned with a script to follow. Instead, it involves exploring all around the product and seeing where the testing takes you.

The phrase “exploratory testing” was first introduced in Cem Kaner’s book Testing Computer Software, first published in 1987. In it, Kaner says “No matter how many test cases of how many types you’ve created, you will run out of formally planned tests. You can keep testing. Run new tests as you think of them, without spending much time preparing or explaining the tests. Trust your instincts.”

Exploratory testing focuses on the explorational discovery, often from an experienced tester, to uncover defects that are not easily covered in the scope of other predetermined tests. The tester can use their experience to test the software with some specific goals in mind based on their prior experience — such as demonstrating that your search will return the results within a suitable time period because you’ve previously seen issues with latency.

A nice description to use when comparing exploratory testing with scripted testing is that scripted testing is similar to you making a speech. You prepare the words in advance and follow them as you present, whereas exploratory testing is more of a conversation. It’s spontaneous and involves reacting to the various outputs that you receive:

How exploratory testing benefits products and an organization

This ad hoc approach to testing is not perfect in 100 percent of situations. However, it really can come into its own in several situations, including:

• When documentation of the feature might not be present and you need to figure out what’s going on as you are testing
• When you’re unclear on what should be tested and so you can’t always prepare scripted tests
• When you want to follow the behavior of real users and not the scripted, perfect flows of a test writer

By giving the tester the freedom to explore, you open up possibilities that you might not have previously considered for testing. In turn, this will often result in the discovery of more bugs — or at least different types of bugs than conventional scripted testing would return.

In addition, exploratory testing is a valuable early-stage tool for organizations, particularly for those keen to understand how their application functions early in the development process.

The act of exploring early versions of a product may identify previously unconsidered areas that can be addressed during the development phase. The cost of resolving the issues is less at this point, especially compared to waiting for development to be complete and for the preparation of scripted tests.

Perhaps the most frequently cited benefit of exploratory testing is that it provides the organization with a great opportunity to learn about its application and perhaps fill in missing knowledge gaps on how it operates. This can feed back into the process and provide new functional specifications for future reference or the basis for future scripted testing.

Why exploratory testing is not always right to use

Of course, there is never one perfect approach to anything, and with exploratory testing comes several potential disadvantages.

The nature of exploration is that you are following an unknown path toward a potentially undefined destination. It might make the replication of any bug you find difficult to achieve. For example, what data did you input in that form two steps ago that is now generating this error? Or which link did you click on that’s now taking you from the US site to the UK site?

Another characteristic of exploration is that you are not following a map. You’re feeling your way through, which means you won’t always be traveling along the most efficient route toward any outcome.

An hour’s testing might return 10 bugs. It might return none. You won’t really understand how much of your application has actually been tested, whereas, with scripted testing, you’ll know that after an hour of following the test scripts, you’ve covered 25 percent of the application’s tests.

This uncertainty of not knowing what you’ve covered can also lead to the uncertainty of not knowing when your testing is finished. How do you know when you’ve tested enough of the application to feel comfortable releasing it? Should you keep testing or have you covered everything?

How to approach exploratory testing

If you’ve decided that exploratory testing might be the best approach for your organization or for your latest feature, there are a few simple steps to take to make it a success. You can deliver the greatest value to the organization and reduce some of the uncertainty surrounding it if you:

• Use an experienced tester — they’ll know from their know-how where issues are likely hiding. You’ll gain more value here than if you let an inexperienced tester wander aimlessly around your product
• Determine what ideas you want to test with your explorations — for example, “We will explore the process of searching for new products on our website and how the products interact with the shopping cart”
• Set time limits so you know when you’re done — you could allocate, say, four hours testing the application and then review the progress after that
• State what you’re planning to do with your test results on top of reviewing the bugs – maybe you can use the testing to draft some future automated tests
• Ensure you keep notes on what you’re testing and why — this is pretty crucial. A good example of a note is “We’re testing the shopping cart interactions to ensure that products can be added, edited, and removed”

Of course, these are all on top of the standard testing actions for tracking questions raised and issues identified.

So, should we do exploratory testing?

As with many of these subjects, the answer is always maybe.

Exploratory testing is great in some situations but it also isn’t the answer to everything. There are situations where it can provide a more successful result than other forms of testing, but it isn’t fit for each and every testing purpose.

Just make sure you think it through and review its use at the end. You’ll soon learn when the best times to go exploring are!