Integrate model-based testing to find quality problems early in development

Advantages of using the Rational Rhapsody TestConductor Add On with Rational Quality Manager throughout the software development process

Take advantage of the integration of the Rational Rhapsody TestConductor Add On with Rational Quality Manager to enhance your model-based testing inside a powerful generic framework. This article shows how the integration helps enable risk-based testing to assist QA teams as they prioritize testing of specific features and functions, based on their importance in the overall project and likelihood or impact of failure. Combining this with new reporting dashboards gives product managers a more realistic view of product performance according to set business objectives to better ensure that software development projects stay on track.

Share:

Marc Lettrari, PhD (marc.lettrari@btc-es.de), Vice President, Rhapsody and Statemate Testing Products, BTC Embedded Systems AG

author photoDr. Marc Lettrari received his master’s degree in computer science from the University Oldenburg, in Germany, in 2000 and his PhD in computer science in 2005. He previously worked at the OFFIS computer science institute in Oldenburg and conducted research in the field’s formal methods, model-based development, and testing. Since then, he has been working at BTC Embedded Systems AG, in Germany, where he is Vice President for Rhapsody and Statemate Testing Products.



05 April 2011

Also available in Chinese

Managing model-based tests with Rational Quality Manager

Quality assurance teams are often brought into the development lifecycle too late, after requirements are set and errors introduced into the design. To remedy this, IBM has integrated the Rational® Rhapsody TestConductor Add On with Rational® Quality Manager software to provide a live test that span the entire product life cycle. The result provides a consolidated view from requirements to product delivery. This article describes how to leverage this integration by bringing model-based tests into your overall testing process.

This integration works by using the Rational Quality Manager solution to manage different kinds of tests cases, their execution and results executions. This tool displays requirements, test cases, and other resources in one server-based view, which helps geographically dispersed team members exchange information in real time. The Rhapsody TestConductor Add On uses the UML testing profile to automatically specify the test architectures, to visualize test cases for improved understanding, and to execute the tests to pinpoint design model failures and implementation errors.

The article also shows how Rhapsody TestConductor test cases can be managed and incorporated into the overall quality management process within Rational Quality Manager. It explains how the integration helps enable risk-based testing, thus assisting QA teams as they prioritize testing of specific features and functions based on their importance in the overall project and likelihood or impact of failure. The ability to prioritize, combined with new reporting dashboards in the software, offers product managers a more realistic view into product performance against set business objectives to better ensure that your project stays successfully on track.

Quality management throughout the software development lifecycle

Today's tech-savvy consumers expect high quality from products they use. Delivering anything less than that can result a company reeling from a perception of poor quality for years to come. Consumers also expect advanced features, which drives complexity. The amount of software within embedded products is increasing and driving key product differentiation.

With these challenges of quality, complexity, and size, companies need to optimize their software testing to achieve desired business outcomes. Often, software testing is done late in the development cycle, which leads to finding the majority of defects near the end, when they are the most costly and riskiest to address. However, these defects were introduced into the design early in the development process. Software developers perform tests while developing software, such as unit tests, but these tests tend to be very isolated to the developer or inconsistent across developers. Software developers need to focus on developing, so testing becomes an area they take shortcuts on when a deadline looms.

The advantage of model-driven testing

Fortunately, emergence of the Unified Modeling Language (UML) and Systems Modeling Language (SysML) standards has provided the opportunity for breakthrough gains in streamlining the development process. These standards enable engineers and software designers to significantly improve their productivity by transitioning from a code-based development process to model-driven development (MDD). Using models, systems engineers and software developers can more clearly understand and analyze complex requirements, automate repetitive tasks, maintain consistency across design activities, test systems concepts earlier by using simulation, and automatically generate code for direct deployment on the target hardware.

Model-based testing extends model-based development beyond design to testing. The Rational Rhapsody TestConductor Add On adds model-based testing capabilities based on the UML Testing Profile to the Rational Rhapsody Developer, Rational Rhapsody Designer for Systems Engineers, or Rational Rhapsody Architect for Software editions. The testing profile tailors the development environment for testing by adding concepts such as test architectures and test behaviors into UML. Test architectures extend the existing UML 2.0 structural concepts to describe the elements involved in a test and their relationships. Similarly, the test behavior extends the existing UML 2.0 behavioral concepts to encompass all observations and activities during the test.

The TestConductor Add On automates the testing activities by creating a test architecture and enabling tests to be graphically captured by using UML sequence diagrams, state charts, or flowcharts. Capturing tests graphically allows better communication of the tests and further elicits the behavior of the design. The tests can be executed and results monitored to automate unit and regression testing.

By extending testing forward in the process to include the design model, then quality managers, and software engineers can check that each feature meets requirements early in development so they can assess the overall quality of the design early.

Overview of Rational Quality Manager

IBM® Rational® Quality Manager is a collaborative, role-based, business-driven environment for test planning and execution, workflow control, tracking, and metrics reporting. It provides a web-based centralized test and quality management environment for project managers, quality engineers, and other project stakeholders to collaborate and share information about the current quality of the project.

Rational Quality Manager quantifies how project decisions and deliverables affect and align with business objectives and outcomes. It helps the team reduce defects and accelerate product delivery schedules.

Using this software, you can create a customizable dashboard for quick and easy review of systems quality metrics and project status, such as builds ready to be tested. It also provides traceability features that can help test teams ensure that test plans are covering all requirements. The dashboard helps highlight areas where quality is at risk in the project and where priority should be focused.


Analyzing and elaborating requirements using executable models

The starting point of a development project is the requirements analysis phase. The objective of this phase is to analyze the process inputs. Stakeholder requirements are translated into system requirements that define what the system must do (functional requirements) and how well it must perform (quality of service requirements). In a model-based development process, using IBM Rational Rhapsody as the underlying UML modeling tool, this means that the initial stakeholder requirements (which are stored in IBM® Rational® DOORS® requirements management software, for example) are imported into the requirements model, which is depicted in Figure 1.

After the stakeholder requirements are imported, a model-based requirements analysis can be performed. This means that a UML model is created that captures the given requirements and can be used for analyzing them. With Rational Rhapsody, it is possible to create executable models, allowing analysis of the requirements by executing the model derived from these requirements for validation early in the development lifecycle.

Figure 1. Requirements from Rational DOORS are imported and further analyzed and validated in a Rhapsody model
Captures of DOORS and Rhapsody screens showing flow between the two

Validating requirements with model-based testing

As described in the previous section, based on the initial stakeholder requirements, an executable UML requirements model has been created that serves as a reference model for the intended behavior of the system to be developed. However, if the requirements model should serve in later development stages as a reference model, it must be verified that the behavior of the model indeed fulfills the given requirements. This verification step can be done and automated with model-based testing.

Model-based testing enables dynamic testing of the behavior of the model, meaning that test cases can be specified (for example, using UML sequence diagrams) and automatically executed on the model level. The UML Testing profile (see the second reference in Resources) provides definitions of all concepts and terms needed to model test architecture, test behavior, and test data within the UML modeling language. For UML models created with Rational Rhapsody, the TestConductor Add On provides all needed capabilities and a high degree of automation to help you create and execute model-based test cases.

Similar to dynamic software testing, dynamically testing the model ensures that the behavior of the model is indeed correct with regard to the given requirements. Furthermore, given that both requirements and test cases are part of the model, they can be linked directly, which ensures complete traceability from requirements to test cases and test results. Figure 2 shows an example of a test case that is described with a sequence diagram.

Figure 2. A sequence diagram can specify inputs and expected results of test cases
Sequence diagram

Integrating model-based testing into the quality management process

It is widely accepted that quality management throughout the whole development lifecycle is necessary to get a high-quality product. Rational Quality Manager helps optimizing project quality with a single, shared test management hub that provides integrated lifecycle support across many types of testing. It provides a customizable, role-driven solution for test planning, creation, and execution. as well as workflow control, tracking, and end-to-end traceability.

One important aspect of integrated quality management is proactively managing projects and quality risks with real-time, fact-based insights and reporting to ensure that project decisions and deliverables remain aligned with your business objectives. At any time during the development lifecycle, quality management should be able to answer the question: "Are we still on track?"

Especially in early development phases, such as during the functional system specification phase, answering this question can sometimes be hard, because dynamic software testing cannot be performed because there is no software implementation available at this stage. However, if a model-based approach is used for system specification, so that an executable model is already available, model-based testing can already be applied at this early stage to assess to what extent the original stakeholder requirements are met by the current system specification. This means that, despite being quite early in the development phase, you can already measure the quality status of the current project.

How to integrate model-based testing into the quality management process

For Rational Quality Manager and the Rhapsody TestConductor Add On, there is a specific adapter that you can use to access model-based tests directly from Rational Quality Manager. After the adapter is started, it automatically connects to Rational Quality Manager.

Thereafter, when you create a test script in Rational Quality Manager, you can connect it to an executable model-based test in Rhapsody by following these steps:

  1. Create a test script in Rational Quality Manager (see Figure 3). In Rational Quality Manager, each test case needs a test script in order to execute the test case.
Figure 3. Create a test script in Rational Quality Manager to execute the test case
Create test script command

Larger view of Figure 3.

  1. After specifying details such as the test script name and so forth, select Rhapsody TestConductor in the Type field of the test script (see Figure 4). This tells Rational Quality Manager that the test script is actually implemented as a model-based test in Rational Rhapsody.
Figure 4. Specify Rational TestConductor as the type of test script
Create Test script dialog with TestConductor selected

Larger view of Figure 4.

  1. After specifying that the test script is based on a model-based test, you need to define which model-based test actually implements the test script. To do this, from the Create Test Script dialog window, click Select adapter.
  2. When a dialog window opens that shows all available adapters, simply select the adapter that you have started (see Figure 5).
Figure 5. Select the adapter for the test case
Import Test Script dialog window

Larger view of Figure 5.

  1. After you have selected the adapter, the adapter lets you choose the Rhapsody model (Figure 6) that contains the model-based test.
  2. After selecting the model, select the model-based test case that should be connected to the test script (see Figure 7).
Figure 6. Specify the Rhapsody model where the test case is located
Dialog window with the Rhapsody model selected

Larger view of Figure 6.

Figure 7. Specify test case to associate with test script
Listing of test cases

Larger view of Figure 7.

That's it. Now Rational Quality Manager recognizes that this test script is actually implemented by a model-based test that can be executed with Rhapsody TestConductor.

When you run the test script, Rational Quality Manager automatically detects that this is a model-based test and uses the TestConductor adapter to execute it on the referenced executable Rhapsody model, as depicted in Figure 8.

Figure 8. Model-based test cases can be run from Rational Quality Manager
Dialog listing test cases of test plan

Larger view of Figure 8.

After executing the test script in Rational Quality Manager, all of the test results, such as status (passed or failed) and log files, are directly accessible in Rational Quality Manager (see Figure 9). This means that you can use all of the reporting capabilities of Rational Quality Manager to assess the quality status of the project, even in early development phases.

Figure 9. Results of TestConductor tests can be stored and managed in Rational Quality Manager
Window shows test results

Larger view of Figure 9.


Reusing model-based tests for software verification

The previous section described how model-based tests can be integrated into an overall quality management process that is based on Rational Quality Manager so that you can measure the quality status in early development phases. Model-based tests can also be used to measure the quality status of the software implementations as soon as they are available. The model-based tests can be adjusted easily so that, rather than testing the underlying model, they test the real system and software implementations, respectively. If these adjustments are made, the model-based tests can be executed from Rational Quality Manager in exactly the same way as before, but not to test the real implementation.

And again, Rational Quality Manager can use the results of these tests along with the metrics measuring reporting capabilities included to assess the quality status of the implementations. Thus, with Rational Quality Manager, model-based tests specified early can be used throughout the software development lifecycle to help assess the project's quality.

Summary

Integrating model-based testing with the Rational Rhapsody TestConductor Add On and using with Rational Quality Manager for quality management helps bring make the quality status of the project visible earlier in the development phases. Teams can then identify quality risks in the project earlier, respond to changes quickly, and assess the impact from a quality perspective. By seeing the tests that fail, teams can recognize critical areas of concern and work to mitigate those risks earlier in the development lifecycle.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational, DevOps
ArticleID=644264
ArticleTitle=Integrate model-based testing to find quality problems early in development
publish-date=04052011