Are you skipping non-functional testing?

Published on

July 21, 2022

Are you skipping non-functional testing?

The first thing that comes to mind when we think about Software testing is Functional testing. Non-functional testing is just as vital as functional testing. However, non-functional testing is many times an afterthought and occurs at the end of the SDLC. Due to tight schedules and deadlines, non-functional testing is often rushed or overlooked. Several testers refrain from performing non-functional testing because they feel it is outside their scope of testing.

Software is considered reliable when it meets customers' expectations under various conditions. Non-functional testing plays a crucial role in meeting customer expectations. Non-functional testing overlooked not only leads to poor user experience but also can cause damage to the organization’s reputation.

In this article, we will explore what non-functional testing means and how we can prevent missing out on testing the non-functional requirements.

What is non-functional testing

Testing the functionality of an application against a set of requirements is functional testing. Non-functional testing examines different factors that functional testing doesn’t cover. Non-functional testing involves evaluating non-functional quality attributes of software, such as Usability, Maintainability, Efficiency, Reliability, and Portability. Refer to the ISO standard (ISO/IEC 9126) to comprehend all the ‘-ilities’.

The purpose of functional tests is to determine if the software does what it's supposed to do, while non-functional tests measure how well it performs its intended tasks. Non-functional testing is a general term that consists of all testing types that focus on how well the software performs in the real world.

Below are some of the noteworthy non-functional testing types -

Diagram of noteworthy non-functional testing types covering reliability, usability, efficiency, maintanability, and p

Performance testing - How well does the application perform under a specific workload in terms of its responsiveness and stability.
Load testing - What happens when a large number of users access the application concurrently?
Reliability testing - How reliable is the application? How long can it run without failure under specified conditions?
Stress testing - What is the maximum amount of stress the application can handle? What happens beyond normal operational capacity?
Recovery testing - How well can an application recover from crashes or failures.
Security testing - How secure is the application? Is the software vulnerable to cyber-attacks?
Compatibility testing - How does the application function in different environments or OS?
Usability testing - How user-friendly is the application? How easily can a user accomplish a given task?
Accessibility testing - Does the design of software, devices, and services enable people with disabilities to use the application?
Compliance testing - Does the application complies with the requirements of a specific standard or regulation
Internationalization and Localization - Can the application adapt to different languages and regional differences without requiring significant engineering changes?
Documentation testing - How easy is it to understand the software documentation?
And more.

Explore, discover, and define non-functional requirements

All the different non-functional testing requirements showing documentation, internationalization, stress, accessibility testing, and so on.

Customers communicate their requirements primarily through Functional requirements. The functional requirements define the tasks and feature users will interact with to achieve their goals using your software. Unlike functional requirements, non-functional requirements are often not defined in the requirement phase. There are certain qualities a system should have without having to specify them. In general, nonfunctional requirements are ambiguous and implicit. They can turn out to be tricky to define.

Just like your functional requirements, the nonfunctional requirements of your software needs should be defined and prioritized.

When gathering requirements, certain non-functional requirements can be identified by listening to customers and their feedback. Asking the right set of questions is the key to extracting non-functional requirements.
Some non-functional requirements might be hard to determine at the start of the requirement phase. They may evolve and get more apparent over time. Constantly evaluate and assess your product to discover non-functional requirements.

The non-functional requirements must be SMART (Specific, Measurable, Attainable, Realistic, Time-bound).

Non-functional requirements cannot be subjective. Customers may say things like, “The website should be fast." But you need measurable criteria to test. For instance, “Every page must load within 2 seconds”.
Understand the SLAs of your system/services that describe the non-functional requirements.

% of Uptime yearly for the Service should be 99.9% (acceptable downtime in a year is less than 8.76 hours) - Reliability
Not more than three major incidents per year - Reliability
The API response time should be lesser than 500ms for the service - Performance
The application must adhere to the Web Content Accessibility Guidelines WCAG 2.1- Accessibility
Database security must be compliant to HIPAA requirements- Security

As a tester, first, understand the SLA’s of your services/applications.
Understand how developers are measuring and monitoring the SLAs.
For new services, help developers to test if we are meeting the defined SLAs.
For existing services, conduct the required non-functional tests regularly.
Shift left and perform non-functional testing as early as possible. The late discovery of non-functional testing can turn out to be expensive.

You don't have to be an expert to get started

Let's be honest; not everyone has expertise in non-functional testing. Do not let inexperience hold you back. Because you do not feel like an expert, don’t be hesitant to start. Testers who succeed are the ones who constantly learn and acquire new skills. Learning new things has become easier than ever before. Numerous online courses and resources are available to help you learn the new skills needed to start non-functional testing.

While it's tempting to jump in and try out different tools, it's best to understand the fundamental concepts first. Below are some of my recommendations to get started-

Non-functional Testing	Recommended Reading for Beginners	Open Source tools
Performance testing	Performance Testing Guidance for Web Applications	Gatling, Locust, K6, Jmeter
Security testing	OWASP TOP 10	OWASP Zed Attack Proxy SonarQube Burpsite community edition
Accessibility testing	WCAG 2.1 Guidelines	WAVE ,axe, Pa11y
Usability testing	Usability.gov Usability testing 101	-

‍

There are a variety of non-functional testing types. Each one of them is essential in its own right. Start small; it might be overwhelming to start testing all the non-functional attributes simultaneously.

Additionally, the cost of performing each type of non-functional test is high. Thus, according to the risks and constraints, you should choose non-functional tests wisely according to the risks and constraints. Depending on your customers and their preferences, specific non-functional testing may be prioritized over others.

The wrap up

Non-functional testing is necessary to give your product a competitive advantage in today’s market. A system could meet all functional requirements but still not meet user expectations. To provide the best customer experience, don't forget the non-functional “ilities” that matter. When Non-functional requirements are neglected, they can impact the customer experience, and your customers look for alternatives.

Non-functional testing should be at the top of testers' priorities. Building a high-quality system that meets your functional and non-functional requirements is relatively easier if you plan for it beforehand. Learn about, comprehend, and ask questions about the non-functional requirements. Incorporate non-functional testing into your testing strategy, moreover, in the development process. Devote some time every day in the quest to learn non-functional testing. Don't dismiss non-functional testing anymore.