Adhoc testing: is it worth it?

What is Adhoc testing?

The word "Ad Hoc" is a Latin phrase that means "for this" or "for this situation. An ad hoc activity is not planned but is done because of a purpose, need, or situation.

Adhoc testing is an informal testing technique conducted without prior planning or documentation. During Adhoc testing, Test Cases or Test Plans are not created before the test is conducted. Instead, Adhoc testing is an experience-based technique that anticipates the occurrence of bugs based on the tester's knowledge and learnings from their past experiences. Adhoc testing is an unstructured way of executing tests, and you can perform it at any stage of SDLC.  

Ad hoc testing only has one purpose: to reveal hidden defects in the software under test in any way possible.

Is Adhoc testing the same as exploratory testing?

The terms Adhoc and exploratory testing are often used interchangeably. Unlike scripted testing, both Adhoc and exploratory testing offer the tester more creativity, flexibility, and freedom. However, Adhoc and exploratory testing are not the same. In contrast to Adhoc testing, exploratory testing takes a disciplined approach. Exploratory Testing is the highly evolved version of Adhoc testing.

Exploratory testing involves simultaneous learning about the software, designing tests, and executing them. The result of the previous test influences the tester's choices for the next test. Unlike scripted testing, testers do not write test cases in advance.

Although both Adhoc testing and exploratory testing are a sort of freestyle testing, the significant difference between the both is the focus. During exploratory testing sessions, the focus is primarily on the test charter. The charter acts as a mission statement for exploratory testing. Charters offer direction for exploratory testing and ensure testing stays on course. Though testers may not formally document the results of exploratory tests, they produce informal notes, mind maps, etc.

On the other hand, Adhoc testing has very little structure. The Adhoc test is an on-the-fly process of identifying bugs and is unstructured. Adhoc testing does not have a specific goal, scope, or focus. However, it does produce any sort of documentation. A prerequisite for Adhoc testing is that testers need to know the tested software.

Adhoc testing often becomes challenging to manage, measure, or even track the performed testing.

Change your perspective on Adhoc testing

"Think outside the box" is cliche testers hear repeatedly. But no one tells testers how to think outside the box. Thinking outside the box means thinking differently or unconventionally. It takes unconventional methods like Adhoc testing to think unconventionally. Testing is ineffective when you rely only on executing the pre-defined(scripted) test cases —following a script. At the same time, testing will limit your thinking, and there is no room for creativity. Adhoc testing helps you generate creative ideas and discover hidden and unexpected bugs. If you want different results, you have to do things differently. It would be best if you deviated from the script.

Adhoc testing is not randomly clicking around an application, hoping to find a defect.

Adhoc testing is an experience-based test technique that relies on a tester's skill, intuition, and experience with similar applications. Although intuition is not reliable, it can still act as a guide for testers. It can guide us through similar situations with tremendous success. Furthermore, testers can completely concentrate on testing rather than on documentation.

Adhoc testing fosters creativity and critical thinking in testers. It offers the tester freedom and flexibility to apply their testing style to find bugs by appropriate means. So Adhoc testing discovers bugs usually not found with other methodical techniques. The biggest strength of Adhoc testing is the versatility that this testing approach offers.

Suppose the same tests are repeated over and over again. In that case, eventually, the same test cases will no longer find new bugs (Pesticide paradox). Adhoc testing comes in handy to keep your test cases updated. Learn from the bugs found through ad hoc testing to strengthen your test cases. Several Adhoc tests may be one-off tests that need not be a part of your test cases. Be wise in deciding if these tests must become part of your test cases.

Drawbacks of Adhoc testing

1. Several people view Adhoc testing as "Monkey testing" since it mimics the behavior of monkeys jumping from one tree to another in pursuit of a better fruit. The lack of preparation and structure in Adhoc testing can lead to many unproductive hours. Testers may waste a lot of time exploring to find bugs without a focus or purpose.
2. The effectiveness of Adhoc testing depends on a tester's craftsmanship and experience. Furthermore, testers must be familiar with the software or technology being tested. Inexperienced testers may run out of ideas quickly. As the tester becomes more acquainted with the product and different test methods, the effectiveness of Adhoc testing will increase.
3. Test coverage is difficult to assess and may not be measurable with these techniques. Testers may have a tough time justifying their time and effort spent on ad hoc cases.
4. It may become difficult to replicate the bugs later or remember the tested scenarios when testing without any documentation.
5. Adhoc testing does not provide the same confidence level as scripted testing or more formal testing techniques.

Dealing with the drawbacks of Adhoc testing

Exploratory testing offers similar benefits to Adhoc testing while being more scalable and manageable. In addition, exploratory testing sessions provide structure and accountability, allowing you to set boundaries and goals. Simply put, exploratory testing is Adhoc testing that is structured and documented.

How to perform Adhoc testing

There is no right or wrong way to execute ad hoc testing since it is unstructured and does not follow a specific approach. Likewise, Adhoc testing does not require you to strictly follow a particular set of steps, but here are a few suggestions to help you perform Adhoc testing effectively.

1. Create a rough plan

Theoretically, Adhoc testing does not need prior planning or documentation. Nevertheless, creating a rough plan beforehand enhances the effectiveness of Adhoc testing by giving it some structure.

Define the scope of testing by jotting down the following:

• Areas of the application you want to explore
• What kind of information or problems to look for.
• What tools, configurations, data, and test techniques do you plan to use?
• What aspect of the software do you want to focus on - Functional, security, performance, usability, etc.?

Please do not get too detailed in planning, as it can be counter-intuitive. The whole purpose of Adhoc testing is to provide greater flexibility and freedom.

2. Timebox your Adhoc testing

Timeboxing means assigning a fixed amount of time for an activity. When the time is up, the activity stops. Timeboxing mitigates some of the downsides of Adhoc testing. Timeboxing helps you proactively decide how much time you need to spend on Adhoc testing. Timeboxing allows you to organize better and be more productive with your Adhoc testing. Testers usually timebox Adhoc testing for 60-90 minutes. Experiment with different time intervals and settle on what works best for you.

3. Use smart tools to capture your sessions

When performing Adhoc tests, use powerful reporting tools like Bird Eats Bug to capture your entire session. It's easy to forget what you've discovered because you do not plan or document Adhoc testing. Therefore, reproducing the issues found can turn out to be challenging. In addition, writing high-quality bug reports after the session will take a long time. Therefore, recording your session will come in handy as it will enable you to relook at the testing performed.

Bird automates bug reporting by capturing all the information required to fix a bug. The report includes visual proof, steps to reproduce the error, and technical data like Console logs, network requests, system info, etc.

With Bird, you will never struggle to reproduce a bug again. Bird offers advanced features like bug watch that captures data in the background on your current page. With Bird's Instant Replay feature, you can always watch the recording with the technical logs of your session to replicate the bug. Additionally, it can alert and catch bugs that might not be visible in the interface. Furthermore, it saves your time in bug reporting by providing your developers with essential information to fix a bug in a single link.

Conclusion

Testing needs a combination of different types of testing to be effective. Use Adhoc testing to supplement and complement the other formal testing techniques to do testing well. Several organizations spare time for Adhoc testing before their releases. Some teams use Adhoc testing before starting scripted testing. As a result, they can assess the quality of the software upfront and gain a deeper understanding.

Adhoc testing is a powerful approach that enables testers to tap into their creativity and experience. Use Adhoc testing to check the thoroughness and effectiveness of your testing. Conduct practices like bug bashes and pair testing leverage the strength of Adhoc testing. Experience the effectiveness of ad hoc testing by exploring unexplored areas of your application or a bug cluster with chronic issues.