A complete testing process with Rational Quality Manager

You get complete overview of the testing process by using IBM Rational Quality Manager throughout the software development lifecycle. An online bookstore application provides an example application to describe the effort from a test plan to a test report. You also learn how to integrate software requirements from IBM Rational RequisitePro with Rational Quality Manager. You submit a defect from Rational Team Concert, which is also integrated with Rational Quality Manager.

Share:

Yan Ting Zhang, Software Testing Engineer, IBM

author photoYan Ting Zhang is a software testing engineer from IBM's Globalization Laboratory, User Technologies, China Development Lab, Shanghai, China. She has 2 years of experience in testing.



Yang Gu (guyang@cn.ibm.com), Staff Software Engineer, IBM

author photoYang Gu is a project manager from IBM’s Globalization Laboratory, User Technologies, China Development Lab, Shanghai, China. She has rich experience in project management and the adoption of relative tooling.



Jie Hu, Staff Software Engineer, IBM  

Photo of Jie HuJie Hu is a software engineer from IBM China Development Lab. He has 5 years of experience in Java Platform, Enterprise Edition, development.



11 March 2010

Also available in Portuguese Spanish

IBM® Rational® Quality Manager is a collaborative, Web-based, quality-management tool that offers comprehensive test planning and test-asset management throughout the software lifecycle. Built on the IBM Jazz™ platform, it is designed for test teams of all sizes to use. Rational Quality Manager supports a variety of user roles, such as test manager, test architect, test lead, tester, lab manager, and roles outside of the test organization.

Introduction

In this article, you look at the complete testing process using IBM® Rational® Quality Manager throughout the software development lifecycle. An online bookstore provides the example for the full process, including creating a test plan, executing test cases, submitting defects, and generating test reports. You also can create a test plan in IBM Rational RequisitePro when IBM Rational RequisitePro is integrated with RQM. Similarly, you see how to submit a defect to Rational Quality Manager from an integrated version IBM Rational Team Concert™. Before you start following the steps in the article, you need to install Rational RequisitePro v7.1 and IBM Rational Team Concert. In one procedure, you integrate Rational Quality Manager and IBM Rational Functional Tester. Make sure Rational Functional Tester is installed also.


Road map

In experiencing the complete testing process, you complete these major steps:

  • Learn about the online bookstore application.
  • Plan your test.
  • Import a requirement from the integrated Rational RequisitePro application.
  • Create and execute a test case.
  • Submit a defect to the integrated Rational Team Concert application.
  • Report the results of the test.

Authors' note

This article was written using IBM Rational Quality Manager Open Beta Version 1.0.1.20090310 running on a Microsoft® Windows® XP Professional SP2 server, using the Microsoft® Windows® XP Professional SP2 operating system with the Mozilla FireFox 3.0.4 browser. In addition, this article refers to a version of a bookstore demonstration application that was publicly available at the time of writing.


Welcome to the online bookstore

The online bookstore is a Web-based application that a team must test before deployment. These four steps capture the basic flow of the bookstore application. Users complete these procedures in the bookstore:

  1. Log in to the bookstore.
  2. Search for a book in the bookstore: Who Says Elephants Can't Dance?
  3. Add the book to shopping list.
  4. Pay for the book.
Figure 1. The bookstore search page
The bookstore search page showing search results

Plan your test

The test plan describes the overall scope of the test and the test schedule. The test plan provides a record of the test planning process. The plan also identifies test environments, entry and exit criteria, quality goals, and other aspects of the test.

When you create a test plan in Rational Quality Manager, you can base the plan on a default template or you can create a test plan. You can also define a default template or develop a new template. This flexibility makes creating test plans in Rational Quality Manager suitable for different teams that need to conduct one or numerous types of tests, such as functional verification tests, performance tests, system verification tests, globalization verification tests, and so on.

Figure 2. Sections of the default test-plan template
The menu of default test-plan sections

Each section has an editor, an area where you can add details and information about the test. Some sections, such as Business Objectives and Test Objections, provide a rich-text editor for input. These editors provide common capabilities for formatting text, such as creating tables, setting fonts, and creating bulleted and numbered lists.

Some sections, such as Requirements and Tests Cases, provide links to additional test artifacts. Other sections include tables where you can establish criteria and measure testing progress against those criteria, such as Quality Objectives, Entrance Criteria, Exit Criteria, and Test Schedules.

Tip:
You can customize your own test-plan template by using the Manage Sections feature.

This screen capture shows the test plan for the online bookstore application:

Figure 3. The test-plan requirements and schedules
Windows show test-plan requirements and schedules

In addition to the test-plan requirements and schedules, you need to specify the environment or environments in which to test. Browsers that customers use and the software that you run to support the online bookstore are integral application components that an effective test must cover.

Figure 4. The test-plan environments and team members
Windows showing test environments and testers

Essential decisions in test planning include deciding when a test is ready to undergo the test you are developing. Moreover, you must decide when an application under test is ready to exit this testing stage. You can make the entry and exit criteria simple or complex depending on the test.

Figure 5. The entry and exit criteria in the test plan
Table entries list test entry and exit criteria

Integrate requirements from Rational RequisitePro

For many teams, requirements management helps to ensure life-cycle traceability. By linking the test cases in your test plan with requirements that product managers, program managers, or other members of the larger, cross-functional team have assigned, you can verify that all requirements are tested.

If you define your requirements in an external tool, such as Rational RequisitePro, you can import the requirements into Rational Quality Manager, where they are accessible in the Requirements view. You can also associate requirements with test plans and test cases or add requirements to your test plan manually.

When requirements are updated or deleted in the original requirements application, the status of the requirement in Rational Quality Manager is updated with the Suspect icon (Suspect icon). You need to update the test plans or test cases that are associated with these suspect requirements.

Note:
Rational Quality Manager integrates with Rational RequisitePro Version 7.1 and requires the Rational RequisitePro client for Web to implement the integration. Be sure to select the Web Components option when you install Rational RequisitePro.

After you install and configure Rational RequisitePro, you can use the application to manage your requirements and import those requirements into Rational Quality Manager.

The following screen capture from Rational RequisitePro shows this requirement: The online bookstore application must provide a way for customers to search the extensive list of books:

Figure 6. The book-search requirement in Rational RequisitePro
A page that shows the requirement attributes

To integrate Rational RequisitePro and Rational Quality manager and import requirements:

  1. Install Rational RequisitePro v7.1, and configure a Web server for the application.
  2. Create a Rational RequisitePro v7.1 project, and add one or more users to the project.
  3. Add requirements to the Rational RequisitePro project.
  4. In Rational Quality Manager, point to the Requirements icon (Requirements icon), and click Import Requirements.
  5. Select Import from an external requirement system, and click Browse; then select Rational RequisitePro.
Figure 7. Requirement source configuration
A window for configuring the requirement source
  1. In the Source, select RequisitePro.
  2. In Host Name, type the host name or IP address of the host.
  3. In Port, type the port number of the host, and click Next.
  4. Select the name of the project in the requirements repository.
  5. Type your user ID and password for the requirements repository, and click Next.
Figure 8. Requirement view customization
A window shows the requirement list for selection
  1. In Type, select the requirement type of the requirements to import. A list of requirements is displayed.
  2. Select the requirements from the list. To import all the requirements that are listed, select the check box next to the Status column.
  3. Click Import, and then click OK. The requirements are imported into Rational Quality Manager.
  4. Point to the Requirements icon (Requirements icon), and click All Requirements. The requirements that you imported are displayed in the Requirements editor.
Figure 9. The Requirements view list
The requirements list in Rational Quality Manager

The next step is to associate each imported requirement with a test plan. You associate requirements with test plans to ensure that you test all product requirements thoroughly.

To associate one or more requirements with a test plan:

  1. Open a test plan.
  2. From the Table of Contents, click Requirements to display the Requirements section.
  3. Click the Add Requirement(s) icon (Requirements icon).
  4. Select the requirements to associate with the test plan, and click OK.

The requirements are associated with the test plan.

Create a test case

Test cases are essential to maintaining high quality throughout the testing process. Test cases define what you need to validate to ensure that the system under test works correctly.

Rational Quality Manager provides a test-case template with some defined sections. For example, the template includes sections for preconditions and post-conditions and test execution and descriptions of expected results. You can also create a test case for a current test plan and associate your requirements with the test case. When you add a test script to a test case, it is listed in the Test Scripts section of the test case.

Note:
Rational Quality Manager can integrate with IBM Rational Functional Tester version 8.0 or later, IBM Rational Robot version 7.0.2 or later, IBM Rational Performance Tester version 8.0 or later, and IBM Rational Service Tester version 8.0.

Here is a Rational Quality Manager test case and test script that pertains to a search for this book by using the online bookstore application: Who says elephants can't dance?.

Figure 10. A book-search test case in Rational Quality Manager
A Rational Quality Manager test-case design window

Listing 1. Test script of a search for a book titled, Who Says Elephants Can't Dance?

Click to see code listing

public void testMain(Object[] args) 
{
	startBrowser("Internet Explorer","http://localhost:8081/BookStore/");
	text_username().click(atPoint(21,2));
	browser_htmlBrowser(document_demosBookStore(),DEFAULT_FLAGS).inputKeys("yantingz{TAB}");
            browser_htmlBrowser(document_demosBookStore(),DEFAULT_FLAGS).inputChars("yantingz");
	button_submit().click();
	text_widget_searchText().click(atPoint(41,14));
	browser_htmlBrowser(document_demosBookStore(),DEFAULT_FLAGS).inputChars("who says elephants can't dance");
	button_search().click();
	try{
	if(image_clickToShowDetailOfTheBo().performTest(ClickToShowDetailOfTheBook_textVP())){
	logTestResult("Search for book named who says elephants can't dance.", true, "Pass");
	}
	else{
	logTestResult("Search for book named who says elephants can't dance.", true, "Fail");
	}
	}catch(Exception e) {
	logException(e);
	}
}

Associate a test script with your test case

If you have test scripts, you can add them to your test case. Of course, you can also create a script. This example provides all the steps to associate a test script with your test case.

  1. Create a test case, and then open it.
  2. Select the Test Scripts section in the Table of Contents.
  3. Click the Add Existing Test Script icon (Add Existing Test Script icon). The Add Test Scripts window opens.
  4. Click Run to view all the test scripts in the project or use the View Builder to filter the result.
  5. Select the script or scripts to add to the test case, and click OK.
Figure 11. The Add Test Scripts window
A script is listed in the Add Test Scripts window
  1. Save the changes to the test case.
  2. Point to the Construction icon (Construction icon), and click View Test Case. The test case that you created is displayed in the Test Case editor, as shown in Figure 12.
Figure 12. A test-case list
Test cases listed in the View Builder

Replace literal values with external test data in manual test scripts

To use external data in a manual test, you must associate test data with the manual test script. After you establish this association, you can use the values in the test data to replace literal values in the manual test script.

Before completing these steps, create test data and associate a test case with a manual test script.

To replace literal values with test data in a manual test script:

  1. Create the external test data, the file must be in a comma-separated value (CSV) format with the variable information in the first row.
  2. From the Test Scripts section of the test case, open the test script by clicking the name. The test script opens in the Manual Test editor.
  3. From the Test Data list, click the test name, and click Save. This action associates the test data with the test script.
  4. Insert the test data variable into the manual test script:
    1. Click the step to which to add a test data variable, and then move the cursor to the location to insert the variable.
    2. From the toolbar, click the Insert Test Data Column icon (Insert Test Data Column icon).
    3. Select the variable, and click OK. The variable is inserted at that point in the step.
    4. Repeat these steps to add additional variables.
Figure 13. Adding external data to test scripts
Replacing literal values with external data

During test execution, the literal values are replaced with the test data.

Figure 14. Running test-script steps with variables
User input values are replaced with external data

Run a test case

Rational Quality Manager provides a variety of ways to run test cases, making the tool highly adaptable for many types of test teams. In this example, you run a Rational Functional Tester script.

  1. Integrate Rational Quality Manager with Rational Functional Tester:
    1. Click the startadapter.bat batch file that is installed by default into the C:\Program Files\IBM\SDP\FunctionalTester\RQMAdapter directory. The Rational Functional Tester Adapter window opens as shown in Figure 15.
Figure 15.The Rational Functional Tester Adapter window
A Rational Functional Tester adapter setup window
  1. Enter the Server URL for the Rational Quality Manager server, a valid login ID and password; then specify a unique adapter name. The adapter name is the name that is displayed in the Rational Quality Manager Web user interface (UI) to identify this instance of the adapter.
  2. If you do not want to enter your password each time you start the adapter, select Save Password.
  3. After you complete all fields, click Start Adapter.
  4. When the connection is made, the Connected is displayed at the bottom of the Rational Functional Tester Adapter window. You can also click the Adapter Console tab (Figure 16) to see the adapter status.
Figure 16. Checking the adapter status in the Rational Functional Tester adapter
The Adapter Console page gives status information
  1. Point to the Construction icon (Construction icon), and click View Test Case.
  2. Choose the test cases you want to run.
  3. In the upper-right corner of the test case, click Run Test Case, which opens the Run Test Case dialog box.
  4. Make selections based on your project. For example, choose selections in Test environment and Associate Results with a Test Plan (Figure 17).
Figure 17. The Run Test Case window
Input fields to complete for running a test case
  1. Click OK. The Script Execution page opens as shown in Figure 18.
Figure 18. The Script Execution page
Summary of specifications for the script to run
  1. After you run the test script, click Close and show results. You can see the test results in Rational Quality Manager as shown in Figure 19.
Figure 19. Test-run results in Rational Quality Manager
A page displays test results and test details
  1. On the Execution Result page, click the Rational Functional Tester Simple Log link. The test results open in your browser as shown in Figure 20.
Figure 20. The Rational Functional Tester simple log
The simple log provides brief test-run results

After running your test, you can add attachments, log defects, and add comments to the script.


Submit defects in Rational Team Concert

You can set up communication between Rational Quality Manager and Rational Team Concert so you can create and track defects in Rational Team Concert. After you set up communication, you can also create and track defects in the Rational Quality Manager user interface, even though the defects themselves are maintained in Rational Team Concert.

Note:
Before you begin, verify that you can log in to Rational Quality Manager from the computer that hosts the Rational Team Concert server. Likewise, verify that you can log in to the Rational Team Concert Admin Web UI from the computer that hosts the Rational Quality Manager server.

To set up defect tracking in Rational Team Concert:

  1. Set up Rational Team Concert to accept cross-domain requests from Rational Quality Manager:
    1. In the Rational Team Concert Admin Web UI, click the Server tab.
    2. In the Configuration pane, click Advanced Properties.
    3. Under Core Repository Component, scroll to the com.ibm.team.repository.service.internal.xdomain.DynamicProxyFrameService component.
    4. In Current Value of the Cross Domain Host Whitelist property, type the various URLs that can be used to access the Rational Quality Manager Jazz Team Server. Separate each path with a comma, and try to include all possible paths. Consider these URLs, for example:
      https://myserver:9443/jazz,
      https://myserver.ourdomain.mycompany.com:9443/jazz,
      https://9.12.345.67:9443/jazz

      These examples include the simple host name, the host name with the domain name, and the IP address.

      To configure multiple Rational Quality Manager servers, use commas to separate each server URL.

      Note:
      These paths must be the same that users use to access Rational Quality Manager.
    5. For the Enable Cross Domain Communication property, in Current Value select true.
    6. Scroll to the com.ibm.team.repository.servlet.internal.ServletConfigurationService component.
    7. Type the host name of the Rational Team Concert server.
    8. In Use Canonical Host Name, select false.
Figure 21. Rational Team Concert administrator configuration window
Setup for cross-domain requests
    1. Click Save.
    Set up Rational Quality Manager to communicate with Rational Team Concert.
    1. In the Rational Quality Manager user interface, click Admin, and then click Jazz Server Administration.
    2. Click Advanced Properties.
    3. In Adapter for RTC Component, go to the com.ibm.rqm.adapter.rtc.service.internal.RTCDefectsCollector component, and modify the properties as necessary.
      • If necessary, in Current Value, type the correct user name and password for the Rational Team Concert Jazz Server administrator. At minimum, user-name account must have JazzUsers repository permissions and also have an assigned Client Access License.
      • If necessary, specify a period in Delay or accept the default value.
        The delay value determines how frequently the Rational Quality Manager server synchronizes with the Rational Team Concert server. The default period is 500 seconds. To see changes more frequently, decrease the delay. If decreasing the delay produces too great a load on the Rational Quality Manager server, you can increase the delay.
      • If necessary, update the period in Draft Link Lifespan or accept the default value.
        This value specifies the time before a draft defect expires automatically. The default period is 1000 seconds. If a user creates a defect or associates an existing defect, the user must save the defect before the draft link lifespan times out.
    4. Also in Adapter for RTC Component, go to the com.ibm.rqm.adapter.rtc.service.internal.rest.ConnectionRestService component, and modify the properties as necessary. In Current Value, type the correct user name and password for the Rational Team Concert Jazz Server user. This user-name account must have JazzUsers repository permissions and also have a Client Access License assigned to it.
Figure 22. Rational Quality Manager administrator configuration
Configuring the Rational Team Concert adapter
    1. Scroll to the com.ibm.team.repository.servlet.internal.ServletConfigurationService component.
    2. Type the host name of the Rational Quality Manager server.
    3. In Use Canonical Host Name, select false.
    4. Click Save.
    Set up the linkage between the Rational Team Concert project area and the Rational Quality Manager project area. After you set up this linkage, the defects that users create in Rational Quality Manager are stored in the corresponding Rational Team Concert project area.
    1. In Rational Quality Manager, click Admin, and then click System Properties.
    2. Click Defects Provider, and in Provider select Rational Team Concert (Defects).
    3. Click Configure.
      This action starts a wizard for setting up Rational Team Concert as the defect provider.
    4. In URL, type the full URL of the Rational Team Concert server, for example, https://rtc_server1:9443/jazz.
    5. In Name, type a name for this particular Rational Team Concert server.
    6. Optional: In Description, type a description of the server.
    7. Click Next.

      Note:
      If you receive a warning message, Security Error: Domain Name Mismatch, click OK. If you receive the Verify the Logged in User request, click Continue or click Log in as different user.
    8. On the next wizard page, select the Rational Team Concert project area for which the defects will be created. The default value is the RQM Defects project area.
    9. Click Finish.
Figure 23. Rational Quality Manager administrator configuration, continued>
Configuring the adapter, continued

After you complete the wizard, the Defects Provider page is updated, showing that the status is OK.

After you complete these steps, you can create and track defects that are saved in Rational Team Concert.

Note:
If you create defects after running a test script, you can create and track the defects in Rational Team Concert. If you click Defects on the left navigation, the defect is only in Rational Quality Manager, and is not tracked in Rational Team Concert.


Report test results

Test managers and test leads are intensely interested in the status and results of tests. Reports from Rational Quality Manager provide this essential information.

Rational Quality Manager provides an array of reports that can be run anytime. Three reports represent some of the information that test managers and test leads can obtain from Rational Quality Manager. The following charts are generated from preformatted reports in Rational Quality Manger. You can also save report-generation parameters and export reports.

Plan Requirements Coverage by Test Case

This report lists the number of requirements that are associated with the test case.

Figure 24. Plan requirements coverage by test case
A pie chart shows test coverage for requirements

The Execution status by Test Schedule report

This report lists the status of execution work items by test schedule. You can select several plans to see the execution results for multiple plans.

Figure 25. The Execution by Test Schedule report
The execution work item status by test schedule

Execution and defects by owner

This report lists the status of execution work items and defects by different owners.

Figure 26. A report showing execution work item and defects by owner
This report shows defects, defect IDs, and owners

Depending on your focus and requirements, you can look at the same information by tester, owner, plan, or computer that was tested. With these different views, you have different ways to get data for comparison.

The software developers receive information about the defects that testing found and work to correct the problems. After that, the application is tested again to ensure that the developers corrected the problem and introduced no other problems.

When quality criteria are met and all defects are resolved, the team can deploy the software.


Summary

You established requirements for the application and captured them in Rational RequisitePro, which can be integrated with Rational Quality Manager. You have developed test cases and test scripts for the test cases. After running a test case and script, you logged defects for the product. You provided reports to managers and others on the team. Rational Quality Manager, working with Rational RequisitePro, Rational Functional Tester, and Rational Team Concert, provides critical, end-to-end tools for developing and maintaining high quality software.


Download

DescriptionNameSize
Files to download for articleacompletetestingprocesswithrationalqualitymanager.zip8MB

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational, DevOps
ArticleID=470980
ArticleTitle=A complete testing process with Rational Quality Manager
publish-date=03112010