Managing model-based tests with Rational Quality Manager
Quality assurance teams are often brought into the development lifecycle too late, after requirements are set and errors introduced into the design. To remedy this, IBM has integrated the Rational® Rhapsody TestConductor Add On with Rational® Quality Manager software to provide a live test that span the entire product life cycle. The result provides a consolidated view from requirements to product delivery. This article describes how to leverage this integration by bringing model-based tests into your overall testing process.
This integration works by using the Rational Quality Manager solution to manage different kinds of tests cases, their execution and results executions. This tool displays requirements, test cases, and other resources in one server-based view, which helps geographically dispersed team members exchange information in real time. The Rhapsody TestConductor Add On uses the UML testing profile to automatically specify the test architectures, to visualize test cases for improved understanding, and to execute the tests to pinpoint design model failures and implementation errors.
The article also shows how Rhapsody TestConductor test cases can be managed and incorporated into the overall quality management process within Rational Quality Manager. It explains how the integration helps enable risk-based testing, thus assisting QA teams as they prioritize testing of specific features and functions based on their importance in the overall project and likelihood or impact of failure. The ability to prioritize, combined with new reporting dashboards in the software, offers product managers a more realistic view into product performance against set business objectives to better ensure that your project stays successfully on track.
Quality management throughout the software development lifecycle
Today's tech-savvy consumers expect high quality from products they use. Delivering anything less than that can result a company reeling from a perception of poor quality for years to come. Consumers also expect advanced features, which drives complexity. The amount of software within embedded products is increasing and driving key product differentiation.
With these challenges of quality, complexity, and size, companies need to optimize their software testing to achieve desired business outcomes. Often, software testing is done late in the development cycle, which leads to finding the majority of defects near the end, when they are the most costly and riskiest to address. However, these defects were introduced into the design early in the development process. Software developers perform tests while developing software, such as unit tests, but these tests tend to be very isolated to the developer or inconsistent across developers. Software developers need to focus on developing, so testing becomes an area they take shortcuts on when a deadline looms.
The advantage of model-driven testing
Fortunately, emergence of the Unified Modeling Language (UML) and Systems Modeling Language (SysML) standards has provided the opportunity for breakthrough gains in streamlining the development process. These standards enable engineers and software designers to significantly improve their productivity by transitioning from a code-based development process to model-driven development (MDD). Using models, systems engineers and software developers can more clearly understand and analyze complex requirements, automate repetitive tasks, maintain consistency across design activities, test systems concepts earlier by using simulation, and automatically generate code for direct deployment on the target hardware.
Model-based testing extends model-based development beyond design to testing. The Rational Rhapsody TestConductor Add On adds model-based testing capabilities based on the UML Testing Profile to the Rational Rhapsody Developer, Rational Rhapsody Designer for Systems Engineers, or Rational Rhapsody Architect for Software editions. The testing profile tailors the development environment for testing by adding concepts such as test architectures and test behaviors into UML. Test architectures extend the existing UML 2.0 structural concepts to describe the elements involved in a test and their relationships. Similarly, the test behavior extends the existing UML 2.0 behavioral concepts to encompass all observations and activities during the test.
The TestConductor Add On automates the testing activities by creating a test architecture and enabling tests to be graphically captured by using UML sequence diagrams, state charts, or flowcharts. Capturing tests graphically allows better communication of the tests and further elicits the behavior of the design. The tests can be executed and results monitored to automate unit and regression testing.
By extending testing forward in the process to include the design model, then quality managers, and software engineers can check that each feature meets requirements early in development so they can assess the overall quality of the design early.
Overview of Rational Quality Manager
IBM® Rational® Quality Manager is a collaborative, role-based, business-driven environment for test planning and execution, workflow control, tracking, and metrics reporting. It provides a web-based centralized test and quality management environment for project managers, quality engineers, and other project stakeholders to collaborate and share information about the current quality of the project.
Rational Quality Manager quantifies how project decisions and deliverables affect and align with business objectives and outcomes. It helps the team reduce defects and accelerate product delivery schedules.
Using this software, you can create a customizable dashboard for quick and easy review of systems quality metrics and project status, such as builds ready to be tested. It also provides traceability features that can help test teams ensure that test plans are covering all requirements. The dashboard helps highlight areas where quality is at risk in the project and where priority should be focused.
Analyzing and elaborating requirements using executable models
The starting point of a development project is the requirements analysis phase. The objective of this phase is to analyze the process inputs. Stakeholder requirements are translated into system requirements that define what the system must do (functional requirements) and how well it must perform (quality of service requirements). In a model-based development process, using IBM Rational Rhapsody as the underlying UML modeling tool, this means that the initial stakeholder requirements (which are stored in IBM® Rational® DOORS® requirements management software, for example) are imported into the requirements model, which is depicted in Figure 1.
After the stakeholder requirements are imported, a model-based requirements analysis can be performed. This means that a UML model is created that captures the given requirements and can be used for analyzing them. With Rational Rhapsody, it is possible to create executable models, allowing analysis of the requirements by executing the model derived from these requirements for validation early in the development lifecycle.
Figure 1. Requirements from Rational DOORS are imported and further analyzed and validated in a Rhapsody model
Validating requirements with model-based testing
As described in the previous section, based on the initial stakeholder requirements, an executable UML requirements model has been created that serves as a reference model for the intended behavior of the system to be developed. However, if the requirements model should serve in later development stages as a reference model, it must be verified that the behavior of the model indeed fulfills the given requirements. This verification step can be done and automated with model-based testing.
Model-based testing enables dynamic testing of the behavior of the model, meaning that test cases can be specified (for example, using UML sequence diagrams) and automatically executed on the model level. The UML Testing profile (see the second reference in Resources) provides definitions of all concepts and terms needed to model test architecture, test behavior, and test data within the UML modeling language. For UML models created with Rational Rhapsody, the TestConductor Add On provides all needed capabilities and a high degree of automation to help you create and execute model-based test cases.
Similar to dynamic software testing, dynamically testing the model ensures that the behavior of the model is indeed correct with regard to the given requirements. Furthermore, given that both requirements and test cases are part of the model, they can be linked directly, which ensures complete traceability from requirements to test cases and test results. Figure 2 shows an example of a test case that is described with a sequence diagram.
Figure 2. A sequence diagram can specify inputs and expected results of test cases
Integrating model-based testing into the quality management process
It is widely accepted that quality management throughout the whole development lifecycle is necessary to get a high-quality product. Rational Quality Manager helps optimizing project quality with a single, shared test management hub that provides integrated lifecycle support across many types of testing. It provides a customizable, role-driven solution for test planning, creation, and execution. as well as workflow control, tracking, and end-to-end traceability.
One important aspect of integrated quality management is proactively managing projects and quality risks with real-time, fact-based insights and reporting to ensure that project decisions and deliverables remain aligned with your business objectives. At any time during the development lifecycle, quality management should be able to answer the question: "Are we still on track?"
Especially in early development phases, such as during the functional system specification phase, answering this question can sometimes be hard, because dynamic software testing cannot be performed because there is no software implementation available at this stage. However, if a model-based approach is used for system specification, so that an executable model is already available, model-based testing can already be applied at this early stage to assess to what extent the original stakeholder requirements are met by the current system specification. This means that, despite being quite early in the development phase, you can already measure the quality status of the current project.
How to integrate model-based testing into the quality management process
For Rational Quality Manager and the Rhapsody TestConductor Add On, there is a specific adapter that you can use to access model-based tests directly from Rational Quality Manager. After the adapter is started, it automatically connects to Rational Quality Manager.
Thereafter, when you create a test script in Rational Quality Manager, you can connect it to an executable model-based test in Rhapsody by following these steps:
- Create a test script in Rational Quality Manager (see Figure 3). In Rational Quality Manager, each test case needs a test script in order to execute the test case.
Figure 3. Create a test script in Rational Quality Manager to execute the test case
- After specifying details such as the test script name and so forth, select Rhapsody TestConductor in the Type field of the test script (see Figure 4). This tells Rational Quality Manager that the test script is actually implemented as a model-based test in Rational Rhapsody.
Figure 4. Specify Rational TestConductor as the type of test script
- After specifying that the test script is based on a model-based test, you need to define which model-based test actually implements the test script. To do this, from the Create Test Script dialog window, click Select adapter.
- When a dialog window opens that shows all available adapters, simply select the adapter that you have started (see Figure 5).
Figure 5. Select the adapter for the test case
- After you have selected the adapter, the adapter lets you choose the Rhapsody model (Figure 6) that contains the model-based test.
- After selecting the model, select the model-based test case that should be connected to the test script (see Figure 7).
Figure 6. Specify the Rhapsody model where the test case is located
Figure 7. Specify test case to associate with test script
That's it. Now Rational Quality Manager recognizes that this test script is actually implemented by a model-based test that can be executed with Rhapsody TestConductor.
When you run the test script, Rational Quality Manager automatically detects that this is a model-based test and uses the TestConductor adapter to execute it on the referenced executable Rhapsody model, as depicted in Figure 8.
Figure 8. Model-based test cases can be run from Rational Quality Manager
After executing the test script in Rational Quality Manager, all of the test results, such as status (passed or failed) and log files, are directly accessible in Rational Quality Manager (see Figure 9). This means that you can use all of the reporting capabilities of Rational Quality Manager to assess the quality status of the project, even in early development phases.
Figure 9. Results of TestConductor tests can be stored and managed in Rational Quality Manager
Reusing model-based tests for software verification
The previous section described how model-based tests can be integrated into an overall quality management process that is based on Rational Quality Manager so that you can measure the quality status in early development phases. Model-based tests can also be used to measure the quality status of the software implementations as soon as they are available. The model-based tests can be adjusted easily so that, rather than testing the underlying model, they test the real system and software implementations, respectively. If these adjustments are made, the model-based tests can be executed from Rational Quality Manager in exactly the same way as before, but not to test the real implementation.
And again, Rational Quality Manager can use the results of these tests along with the metrics measuring reporting capabilities included to assess the quality status of the implementations. Thus, with Rational Quality Manager, model-based tests specified early can be used throughout the software development lifecycle to help assess the project's quality.
Integrating model-based testing with the Rational Rhapsody TestConductor Add On and using with Rational Quality Manager for quality management helps bring make the quality status of the project visible earlier in the development phases. Teams can then identify quality risks in the project earlier, respond to changes quickly, and assess the impact from a quality perspective. By seeing the tests that fail, teams can recognize critical areas of concern and work to mitigate those risks earlier in the development lifecycle.
- To learn more about the Rational Rhapsody collaborative, model-driven development tool for embedded systems, start with these overviews: Introduction to Rational Rhapsody, the Rational Rhapsody Developer product page, and the Rational Rhapsody page on IBM developerWorks.
- Explore the various versions, too:
- IBM® Rational® Rhapsody® Architect for Software, a visual development environment for embedded systems and software
- IBM® Rational® Rhapsody® Architect for Systems Engineers
- IBM® Rational® Rhapsody® Designer for Systems Engineers
- IBM® Rational® Rhapsody® Developer edition. This edition is required for Eclipse users and editions are available to create specialized projects in C, C++, Java, and Ada languages.
- Read the PDF file titled Rhapsody TestConductor Add On Adapter for Rational Quality Manager Overview, which is available in the Rational Rhapsody installation when the TestConductor Add On is installed.
- Explore the various versions, too:
- Check the Rational Quality Manager page on IBM® developerWorks® for links to product documentation, articles, tutorials, courses, downloads, and other useful areas. Also, explore the Rational Quality Manager Information Center for technical details and browse the IBM Quality Management page to learn more about what is available. For tips, read these developerWorks articles by Michael Kelly:
- Getting started with IBM Rational Quality Manager (December 2008)
- For Managing your first project with IBM Rational Quality Manager (October 2008)
- Visit the Rational software area on developerWorksfor technical resources and best practices for Rational Software Delivery Platform products.
- Stay current with developerWorks technical events and webcasts focused on a variety of IBM products and IT industry topics.
- Improve your skills. Check the Rational training and certification catalog, which includes many types of courses on a wide range of topics. You can take some of them anywhere, any time, and many of the "Getting Started" ones are free.
Get products and technologies
- Download Rational Rhapsody Developer and try it free for 30 days.
- Download the free trial version of Rational Quality Manager.
- Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.
- Join the discussion in the Rational Rhapsody forum.
- Join the Rational Quality Manager forum, which also includes discussions about Rational Test Lab Manager.
- Share your knowledge and help others who use Rational software by writing a developerWorks article. You'll get worldwide exposure, RSS syndication, a byline and a bio, and the benefit of professional editing and production on the developerWorks Rational website.
- Follow Rational software on Facebook and Twitter (@ibmrational), and add your comments and requests.
- Ask and answer questions and increase your expertise when you get involved in the Rational forums, cafés, and wikis.
- Connect with others who share your interests by joining the developerWorks community and responding to the developer-driven blogs.