Guide to Black Box Testing: Techniques and Examples

What is black box testing? How does one perform it? What are some techniques for performing black box tests and examples of common cases? This blog post is geared towards QA testers who want an introduction into Black Box Testing techniques, and its different types. After reading this blog post, you'll be able to understand which technique would work best for your project based on its requirements.

What is Black Box testing?

In software testing, Black Box testing is a testing method that mainly focuses on the functionality of an application. It involves comparing an input value with an output value to examine the application's deliverables under various requests.

To perform Black Box testing, a tester will analyze the application under user perspectives to test the behavior of an application, without having knowledge of internal structure or code implementation. In addition, the tester will focus on test scenarios, requirements, and specifications to evaluate if the application responds as expected. For that reason, Black Box testing is also known as Specification-based or Behavioral testing.

"Black box testing is a testing method that mainly focuses on the functionality of an application"

Black Box vs White Box testing

The black and white box testing techniques may be used along with exploratory testing to make it more effective. Read our guide to Exploratory Testing here.

Types of Black Box testing

Black Box testing consists of three main types as following:

• Functional testing: this type focuses on doing testing based on functional requirements of an application.
• Non-functional testing: it is used to test other aspects to improve an application's performance, security, or usability.
• Regression testing: Black Box testing can be used to check if a new code or change will not impact the whole system.
  ‍

How to do Black Box testing

The Black Box testing can be a powerful technique if it is well-prepared and carried out as following steps:

1. Firstly, testers need to understand the requirements and specifications of the application. Software Requirements Specification (SRS) is strongly recommended being established in this step.
   
   
2. To save time and achieve good test coverage, testers should select valid and invalid input for both positive test scenarios and adverse test scenarios.
   
   
3. Develop test cases to cover a maximum range of inputs.
   
   
4. When test case execution is performed, it will provide outputs that will be compared with expected results. Then, the test case will be marked as pass or fail.
   
   
5. The final step is to retest the whole system with other testing techniques to verify it works as expected.
   ‍

Black Box testing techniques

When performing Black Box testing, it is impossible to conduct all test scenarios and test cases due to time and budget limitations. We need to effectively select the most necessary test cases from the test case pool to reduce time while ensuring the highest possible coverage. Following is some common techniques to help design a minimum number of test cases:

Equivalence Partitioning

In this technique, inputs are classified into different levels or groups. Testers will select any one value of a group to test the outcome. This technique's benefit is to help maintain the test coverage while reducing time and efforts to overwhelm testing on multiple inputs.

Example
We will test the behavior of withdrawing money from a bank account. An amount from $1 to $500 is considered valid. Otherwise, any value greater than $500 and less than $1 is considered invalid. It is impossible to test all values because it is a waste of time, and the test case number will exceed 500. Here is how we use Equivalence Partitioning techniques to maximize test coverage:

By applying Equivalence Partitioning, we divide possible inputs into groups or levels (3 groups as above) in which the system will respond as the same. Then we select one representative value from each partition to do testing.

The rule is: if one value in a group passes the test, then all other values will also pass. On the other hand, if one input fails, all other inputs in this group will fail as well.

There is no rule to test only one value of each group. We can select multiple values from each group to fulfill testing's needs and judgments.

Boundary Value Analysis

Errors usually happen at the boundaries of the input range—and that's a matter of fact. The Boundary Value Analysis technique focuses on the values of boundaries, in which the application's behaviors may change. Simply put, Boundary Value Analysis is the following part of Equivalence Partitioning when we select values on the edge of each group. This includes upper and lower values. Both valid inputs and invalid inputs will be examined to see if the system responds correctly.

Example
Let's consider the same example above. We will select boundary values in each group:

• Valid boundaries (in the valid group): 1 and 500
  
• Invalid boundaries (in the invalid group): 0 and 501

Instead of creating too many unnecessary test cases, this approach helps testers save time to cover edge cases and detect any issue around the boundary.

Both Equivalence partitioning and Boundary Value Analysis techniques work well together for applications with many inputs. Needless to say, these techniques help testers cleverly refine a huge amount of test cases into a sufficient amount and perform testing more efficiently.

Decision tables

In many applications, the outputs are generated based on a set of conditions. By identifying various input combinations and the outcome of each, the tester will develop a test case for each rule. A table is usually used to capture system behavior with a logical relationship between inputs and outputs. Therefore, it is also known as the Cause-Effect table. This technique is powerful in terms of dealing with complex conditions to design better test coverage.

Example
We will analyze how to test discount, and Jira integration access on the Bird Eats Bug application (BEB). There are 2 types of subscriptions on BEB: Free Plan and Paid Plan. For Paid Plan, depending on the number of members added to the team, a user will get different discount percentages on subscription.

A table is established to check all possible combinations.

Based on this table, there are some test scenarios will be created to test each combination:

• User is on Free Plan will get 0% discount and no access to Jira Integration
  ‍
  
• User is on Paid Plan and has less than 5 members: will get 0% discount and has access to Jira Integration
  ‍
  
• User is on Paid Plan and has from 5 to 10 members: will get 10% discount and has access to Jira Integration
  
  
• User is on Paid Plan and has more than 10 members will get 15% discount and has access to Jira Integration
  
  

This table is useful for complex requirements with multiple conditions and outputs in terms of designing test cases. However, the disadvantage is that the table itself will become a bit bulky.

State transition testing

When the system has a sequence of events that the same input will result in a different output depending on the previous event, this technique will be the best fit. Test scenarios are developed based on trigger events that will cause changes to the outcome. The best practice to follow is to create a State Transition Diagram (pictured below) to give an overview of how states are changed.

Example
We will examine the upgrade path from Free Plan to Paid Plan for the same BEB application. To upgrade, the user needs to input payment details and click upgrade. The user has two attempts to upgrade in case payment details are invalid (e.g., insufficient funds or incorrect card details). After two attempts, the user needs to contact support for assistance. If the payment details are valid in the first or second attempt, the user can then upgrade successfully.

State transition diagram

In this diagram, the states are shown in circles, and the transitions are the arrows. There are 3 test cases based on this diagram:

• Test cases 1: The user enters correct payment details the first time and then is able to upgrade to Paid Plan successfully
  
  
• Test case 2: User enters invalid payment details the first time and tries the second time with valid payment details. The user is then able to upgrade the plan.
  
  
• Test case 3: User enters invalid payment details for both attempts. The user will then be redirected to the Contact Support page.
  
  

By using this technique, testers can cover all conditions and results through a visualized diagram. However, when the system is large or does not have a sequential order, this technique will not be appropriate.

While Equivalence Partitioning and Boundary Value Analysis mainly focus on the user interface to verify if the application responds to the same behavior for a range of input, Decision tables and State transition testing are more suitable for business logic or business rules when the system responds differently. Besides learning how these Black Box testing techniques work, testers need to determine how to flexibly apply them in real projects when it comes to the size of the application and the complexity of inputs and combinations.

Want to create highly detailed effective bug reports?
Use our template and examples here.

Wrapping up

There is a common misconception that black box testing is easy to do. Unlike white box testing when you know the system inside out, making it easier to test for functionality and quality assurance, black box doesn't have this knowledge at all. This makes testing a lot harder as testers only see what happens once their tests run while not knowing why or how everything works beforehand. To perform quality black box testing, testers need to understand different types of testing to cover all aspects of the applications.

More importantly, it's almost impossible to test everything in a software application due to time constraints. With that in mind, we strongly recommended testers to not only learn the most common Black Box testing techniques but also where to apply these techniques when designing test cases so that they can maximize their testing coverage within a limited time.