Smoke vs Sanity Testing – What's the Difference?

What is Smoke testing?

Smoke testing is a type of testing in which QA will execute a group of the most necessary test cases. This is done to check that the key features of an application work appropriately. In software development, smoke testing mainly focuses on the core functionalities to detect any issues in the initial stage. This will help the development team save time and effort before further development and testing.

Smoke vs sanity testing techniques may be used along with exploratory testing to make it more effective. Read our guide to Exploratory Testing here.

How to perform Smoke testing

When the development team builds a new version of an application (we will call it a new build), testing all aspects of the implementation is time-consuming. Therefore, Smoke testing plays an essential role in helping testers verify the basic features and certify that the build is ready for detailed testing.

Smoke testing is done without creating new test cases. Testers will choose a minimal number of test cases from the existing test case suite based on the most critical functionalities of the application. The objective is to ensure that these test cases are enough to cover the significant workflow with positive scenarios and valid data.

If the Smoke tests pass, the QA will carry out in-depth testing. What if the tests fail and introduce issues? The development team would then need to fix them before renewing the build for the QA team a second time.

Example: Let's consider the Bird Eats Bug application, which has four important features like Login, Recording video, Uploading video, and Viewing video after uploading. To conduct Smoke testing, we will run main test cases in the following steps:

• Login page with valid input
  
• After that, we will check the recording, uploading, and viewing of a video.
  
• If all four critical features work fine, the application is stable to do exhaustive testing.
  

Advantages & disadvantages

Advantages

• Detect blocker or critical issues earlier. Smoke test suite contains the most critical tests that QA will perform when getting a new build before testing further.
  
• It saves testing efforts if critical bugs are discovered during smoke test execution. It's not worth it for QA to test a broken build because it's not ready to ship. The build will not be accepted and returned to developers immediately without doing deeper and detailed testing.
  
• In smoke tests, only a small subset of test cases are executed. They are usually executed and return results quickly. It helps teams to make decisions faster.
  
• Smoke tests are usually automated and integrated with CI/CD pipeline. Hence, this creates a constant feedback loop for every code change applied to the application. As a result, developers can detect the issues by themselves when making a change in the application without going through QA efforts.
  ‍

Disadvantage

• It only covers the most critical tests. Testers will find not all issues in smoke tests.
  ‍

What is Sanity testing?

Testers perform Sanity testing after a few functionalities or some minor bugs fixed are added into the build. The primary purpose is to verify that the changes work as expected while keeping the whole system intact.

Sanity testing will help to catch any issue early after the build is available for testing. It helps to save time if new functionalities or bug fixes are not working as intended. As a result, the build will be rejected before going to Regression testing, further prolonging the process.

How to perform Sanity testing

To perform Sanity testing effectively, QA should first identify the areas where changes happen and their dependencies. To do this right, QA needs to communicate with developers to understand the impact of the changes.

As Sanity testing aims to check newly implemented changes quickly, testers do not need to script the test cases. They simply execute the tests to verify the changes work as expected. Then, randomly test a few related features to verify that they are also working fine.

Sanity tests are usually used in the real world when there is a small change like a bug fix or a slight improvement of a feature. It's widely used in Agile development, which encourages changes and has time constraints to speed up the release process. Sanity testing helps reduce redundant testing efforts in those situations but still keeps the quality and meets the deadline.

Example: We already have a Search feature for the same Bird Eats Bug application that allows users to search bug reports by title, reporter, and domain. Now, users request to add a function to help them search bug reports by Jira and GitHub status via integration.

After the development team has implemented this requirement, and it is ready for testing, the tester will perform Sanity testing as below:

• Focus on this Search feature to verify that it works well with Jira and GitHub status.
  
• Testers will check other functionalities that are related to this change, such as the existing search by title, reporter, and the domain is working the same as before.

Advantages & disadvantages

Advantages

• Sanity testing only focuses on a few functionalities. Because of this, it helps to save time and effort when it comes to quick evaluation for the release.
  
• No documentation is required as it is unscripted.
  
• Work well when there is a slight change in the application.
  ‍

Disadvantages

• It is hard to detect any issue outside the scope of areas that Sanity testing is performed.
  
• As Sanity testing is usually unscripted, it is not ideal for future references.
  ‍

Differences between Smoke testing vs Sanity testing

Although both Smoke testing and Sanity Testing are types of Functional testing to check the functionality of a system working properly, they do have differences that we need to take into account:

Wrapping up

Testing is a crucial component of the software development process. Both these types of testing are used to quickly check the quality of an application. Additionally, they can be easily automated to help teams save time and effort.

In reality, some organizations don't understand the goal of Smoke and Sanity testing. They just let the QA team conduct thorough testing, although the build is not ready. This isn't great practice — It will waste QA efforts and slow down the delivery because teams have to do tons of rework. To avoid that, we should understand the goals of Smoke & Sanity tests and apply them correctly to improve QA productivity and product quality.