Guide to Exploratory Testing in Agile

Agile is the most popular methodology used in software development today—why so? Agile methodology promotes continuous iteration of development throughout the software development lifecycle of the project. Just as your development team practices 'Agile,' so should your QA team. Customers want quick fixes and faster updates, and testing in Agile makes a difference in your project's budget, time duration, and resource utilization.

Exploratory testing in Agile is an important activity in a fast-paced environment as it allows teams to adapt quickly to changes. In this post, we'll cover how you can use exploratory testing in Agile development environments, examples, tools and how it differs from traditional QA processes.

What is exploratory testing

According to the ISTQB Agile Tester Certification Syllabus, exploratory testing is defined as:

"[...] a simultaneous approach where the testers learn about the system, perform test design and write test cases. Exploratory testing may sometimes combine it with another experience-based testing such as analytical risk-based testing, analytical requirements-based testing, model-based testing, and regression-averse testing."

Exploratory testing is a method of evaluating software with no particular plan or test cases in mind. It's the art of breaking things to find out what works and doesn't work. To put it simply: you learn as you go.

When to use exploratory testing

Exploratory testing can be used when there are no test cases or scripts required. Testers have to explore a legacy system or read informal notes from various stakeholders to understand the Application Under Test (AUT).

Exploratory testing is also widely used in short-cycle development processes such as Agile. When it comes to testing types, exploratory testing is mainly used in functional testing to discover issues of newly developed features.

How to do exploratory testing

First and foremost, you need to define the test charter. It is a strategy for exploratory testing sessions. It describes the scope of testing and suggests some basic flows as the starting points for the sessions. It can also contain other information as a guide for testers, such as test environment, test data, variations of testing flows.

Then you need to define how much time you are willing to spend for each session. This is important because it will help testers to focus on critical flows and essential issues. Next, you need to prepare all the necessary tools to test and record the test results.

Once everything is ready, it's time for testing. You need to record test results, issues found, and any observation in a place where everyone in the team can access. Finally, writing the evaluation of the defects. Make sure to include any suggestions that would help improve the product's usability and quality of future testing sessions.

Exploratory testing example and tools

To perform exploratory testing, we need tools to create the test plans, store the test results and record the sessions and issues.

Here is an example of an exploratory test plan.

‍

There are many tools for test plan creation that have intuitive UI, such as Testrail, XRay. For free tools, use Google Sheets or Excel to accommodate what you need to create an exploratory test plan. To record the test results or issues during the session, you would need a screenshot and video recording tool. To record console logs and other technical information, you would need a different tool. With this information, developers can investigate and reproduce issues much faster.

Wondering which tools to use? It's easy to find the screenshot and video tool for both Windows and macOS. For Windows, you can use built-in tools such as Paint or Snipping tool or shortcut Alt + PrtScn to take a quick screenshot. Here is an excellent article that provides details on using shortcuts to take screenshots for macOS. For video recording, Quicktime is a built-in tool for macOS (shortcut Shift + Command + 5). Use Game Bar for video recording on Windows (shortcut Window + Alt + R).

An easy way to record logs is to open the dev console and record a video of the testing session—while replicating the issues. It's also essential to provide all system info of the machine you are using to perform testing (OS, browser version, screen size). Here at Bird Eats Bug, all these bits of information are automatically included in a session, saving users time reporting issues.

Finally, having a bug management tool is super helpful in keep track of issues found via exploratory testing. This ensures that issues are properly documented, reviewed, and fixed. Some popular bug management tools include Jira, Linear, Bugzilla, etc.‍

Testing web applications? Read our Web Application Testing: Guide to Manual Testing to dive in deeper.

Pros and cons of exploratory testing in agile

Benefits of exploratory testing

Exploratory testing can be very beneficial if used at the right time and correct way. First, it saves time to write test cases with a lot of details. Testers can use most of their time to focus on performing actual testing and finding issues solely.

Second, exploratory testing also encourages testers to use their creativity and their intuition over instructions. You should see this as a method of not constraining the tester rather than a lack of preparation.

The more questions testers ask themselves during testing, the better their testing coverage. After each exploratory testing session, testers will gain more knowledge about AUT. Eventually, testers will become product experts and help train users or new hires, if needed.

The agile testing pyramid

Created by Mike Cohn, the agile testing pyramid helps testers understand how to implement automated testing. The bottom part of the pyramid represents types of testing that benefit most from automation. As you move up the hierarchy, exploratory testing—on the top of the pyramid— is non-automated.

The testing pyramid acts as a 'single source of truth' for QA testers to refer back to and meet various needs. Yet, there are still chances that bugs are missing because the agile testing pyramid might not cover edge cases. Here is where manual exploratory testing comes in handy. It's the best type of testing to expose these particular edge cases.

Because exploratory testing does not require comprehensive test scripts, it will give the testers freedom to explore and break things using their creativity. Most testers will focus on actual testing activities to find bugs instead of formal documents. Last but not least, after each session, testers will gain more knowledge of how the system works in the areas where testers performed testing.

Challenges of exploratory testing in agile

Because there are no formal test scripts, exploratory testing might simultaneously overlap the tests performed by different testers. Therefore, some scenarios might be tested twice or more, which is not a good use of their time. The test effectiveness depends on the testers' knowledge and skills.

There might be a chance the testers cannot cover all edge cases if the testers are new or lack testing skills. Testers should have a certain level of product knowledge before assigned to exploratory testing. And more importantly, if the goal and duration are not clearly defined, testers might lose focus on essential flows and don't know when to stop.

Scripted testing vs. exploratory testing

Ad-hoc testing vs. exploratory testing

Both types of testing do not have formal test scripts. Testers execute the tests based on their own product knowledge and skills

Summary

Exploratory testing has become an indispensable part of the modern Agile methodology to get quick feedback and result. It can be a significant change for testers familiar with or prefer the old school method of relying on well-documented requirements upfront. Exploratory testing in agile is quick to implement and fluid in its approach. It helps an Agile team define and develop the application throughout its lifecycle.

Bonus: How BEB helps exploratory testing

As mentioned above, exploratory testing requires various tools to take screenshots, record videos of any issues found, including technical data from the dev console. With Bird Eats Bug, you can save time writing these bits of information manually, enabling a smooth handover between testers to software developers.

Bird Eats Bug is as easy to use as it is to install:
For testers

• Turn on the extension to record the video from the beginning.
  
• Bird will record the video, network, console log for all the steps you interact on the product, including the system information on your device.
  
• You can add the session title and description once you're done with a testing session, giving clarity to each session.
  
• Share with developers via link, or upload straight to Jira, GitHub, Slack, Linear, and more.

For developers

• Navigate in the session and view repro steps, technical information, and any other info needed to find and fix the problem quickly.

To learn more about bug reports, read our Guide to Bug Reporting, complete with examples and templates