What is system testing?

Imagine you’re looking at a software system under a microscope, starting at the most extreme level of magnification, with the unit. This is the basic building block of the software system. Then the view expands outward to include the next level of magnification—the modules created by those individual units. Finally, by zooming out fully, you arrive at the system level. At this level of magnification, you can see everything within the system and how all the components created by those modules work together.

In some ways, system testing is kind of like that microscopic view—but with a key difference. System testing is a form of black box testing, which means that testers are less interested in the view of components involved in its assembly than in the overall functionality of the system. From this kind of “pass/fail” perspective, system behavior is only noteworthy in this context as far as it relates to system performance. (White box testing allows more visibility into the nature of the components within a system.)

System testing often involves the analysis of multiple separate software systems that may or may not work in unison within a particular software system.

Consider the countdown that precedes space launches. While everyone remembers the dramatic 10-step countdown before ignition and lift-off, fewer recall the numerous departmental checks that are asked by the flight chief and answered in the affirmative as a “go.” In a typical space launch, department heads are consulted about planned operations, mission safety, vehicle systems, and expected weather conditions, among numerous other matters. Each department is queried and each department head responds in turn.

Similarly, system testing can be considered the final checklist that precedes the launch of a new software system. The last round of cleaning up any known software bugs has been completed. And just like the historic check-off lists that early space pioneers originated, it all comes down to a final go from each “department” included within the system testing.

Each query is shaded in terms of the system’s functionality:

• Is each component working as expected?

• Are components performing in proper coordination with one another?

• Is the software poised to support a smooth user interface and deliver the desired user experience (UX)?

When discussing system testing, we will naturally encounter the topic of dependencies, which are relationships that exist within test cases. In such situations, the outcome of one test case can depend partially or wholly on the results of another test cases. Dependencies can also involve functionality, testing environments or security policies and can even impact the entire testing process an organization maintains.

System testing methodologies don’t provide a deep look into their inner workings (remember, this is a form of black box testing), but instead lets you know if a particular application works. The idea if for system testing to help locate gaps, errors or missing requirements as it determines the overall functionality of the software application.

System testing is usually performed following integration testing but before acceptance testing, thus ensuring all components function together properly. As we’ll see, it often encompasses both the functional and non-functional aspects of the system. Because it’s based on both the strictly functional and the broadly non-functional areas, it gets into aspects as far afield as usability, security and performance.

One of the main purposes of system testing is that it enables you to verify that a software's coding language can be translated into a usable program. However, the overarching goal of system testing is to make sure that the software being evaluated capably supports the business requirements of the users who will be relying upon it.

The testing process is intended to reflect the same production environment that will be used, to ensure the software functions as needed despite changing, real-world conditions. Similarly, test data is created to mimic real-world data and situations. Once test cases are conducted, defects in the software can be located and fixed.

System testing can be classified according to one of three main groups. The first, functional testing, is concerned with system performance, but seeks no deeper answer beyond learning if the software functions as promised. Here are some of the main types of functional testing:

• Acceptance testing: User acceptance testing seeks to incorporate performance testing conducted by the people who represent the intended demographic of the software being produced. These users bring added realism to the test process by testing the software in real-world conditions.

• Integration testing: One critically important area of functional testing studies how well different modules or components “fit” together when forced to work in close conjunction. That’s system integration testing. A fully integrated system provides testers with the capability to evaluate key interactions.

• Smoke testing: Smoke testing provides a means for checking if overall functionality has been maintained in the wake of a developer making some kind of code change. The change has been made and smoke testing lets you quickly see if there are any adverse effects resulting from that code change.

• Unit testing: In unit testing, a limited section of code is checked over within an isolated test environment. If the unit in question fails its test (as seen from a comparison of test data and functionality goals), more testing of individual components or modules is usually needed at a system-wide level.

While functional testing provides extremely important information, that data is basically limited to an up or down vote based strictly on whether the system performs as it should. That omits a huge number of pertinent variables—like system security, UX and performance aspects.

Non-functional system testing provides a means for judging how the system operates. The essential difference between functional testing and non-functional testing can be boiled down to the following simple comparison:

• Functional testing: Does the system work?

• Non-functional testing: Does the system work well?

With that in mind, here are some of the key forms of non-functional system testing:

• Accessibility testing: Is the digital content being presented accessible to everyone, including people with various disabilities? Accessibility testing investigates this question and works to ensure content is communicated to all possible users, in ways that traverse physical and mental handicaps and ensure consistency with standards for inclusion established by the Americans with Disabilities Act (ADA).

• Compatibility testing: When you consider how widely applications are distributed, and how many cross-platform journeys apps have to make, it becomes easy to understand the need for compatibility testing, which gauges how well such apps function, regardless of how many networks, browsers, operating systems and hardware configurations they have to travel across.

• Load testing: In a sub-discipline that addresses the unique physics presented by the notion of load and how it affects a system’s ability to process system requests, load testing focuses on what happens when scalability occurs on a massive scale. We assume that the system will have no problem processing a single system request, but how about when that load is multiplied by thousands of requests—or even significantly more?

• Localization testing: It’s a global economy in which more countries and cultures participate than ever before. Localization testing is about making sure that software content being exported to various locales around the world is appropriate for that specific area, according to general rules regarding UX.

• Performance testing: This type of non-functional testing pays particular attention to performance issues. For example, it’s essential to good performance that a system responds to requests quickly and smoothly. The test plan in performance testing evaluates how long users must wait for requests to be processed.

• Security testing: It’s no secret that these days, data security is of prime importance. Hence, it makes sense that a form of testing would cater specifically to security. Security testing methods are selected according to the type of potential threats received by the company.

• Stress testing: Just as a medical stress test seeks to understand how well a person’s heart works under the duress of activity, stress testing of software verifies how well a system can remain operationally resilient when pushed beyond its normal capacity to detect bottlenecks and vulnerabilities.

• Usability testing: This type of non-functional testing is completely about the quality of the user experience (UX). Usability testing is a manual testing process that’s most practically used on a small scale. Nonetheless, it should probably be applied often—especially when executing complicated moves like localizing applications.

Still other types of system testing are useful despite not falling into the categories of functioning tests or non-functioning tests. Here are some of the most noteworthy of these methodologies:

• API testing: Application programming interfaces (APIs) are extremely important, enabling software development by helping different apps or systems to connect. API testing ensures that API points of connection operate as intended. It also provides oversight concerning user permissions and the way that data gets managed through APIs.

• Automated testing: As its name implies, automated testing (also called test automation) puts the power of automation to work on testing software applications. It does this by the creation and use of test scripts designed to perform test cases on software apps, which it does on a massive scale, without human intervention. Structured sets of guidelines, tools and practices that help automate the testing process are called frameworks.

• Manual testing: In direct opposition to automated testing, manual testing relies upon human testing to experiment with the software being evaluated, as guided by various test scenarios and test scripts. The testers are encouraged to really put the software through its paces to identify issues that need remediation.

• Migration testing: Migration testing is needed by organizations engaged in updating or transforming its corporate digital culture. When companies are engaged in such transformative efforts, they need validation that key data and software are being properly transferred from an outgoing system to an incoming system.

• Regression testing: While code changes are intended to improve software or enhance its capabilities, they can inadvertently introduce difficulties if human error somehow gets added to the mix. Regression tests let testers confirm steady and correct operation by re-running tests as needed.

While the system testing process provides the most comprehensive black-box performance testing available, performing system testing is not without potential problems: