![Close [x]](http://dw1.s81c.com/i/c.gif){kind=small}
Software testing is an important part of the software development lifecycle, because it assures developers and owners that the software works as expected. With the increased complexity of today's software and the software environment, it is becoming difficult to manage testing manually, alone. Organizations are seeking solutions to automate the testing to mitigate the risk of regression defects leaking into the end product.
Fortunately, there are many automation tools available to automate the tests, including IBM® Rational® Functional Tester and Rational® Performance Tester. There are initial problems associated with automating the test and with maintaining the tests as applications change. However, after a well-proven suite of automated tests is created, it proves useful in long run.
After tests has been automated, the next challenge arises while running the automated tests on all supported platforms, such as Microsoft Windows 2008 Server, Android, Firefox 10, IBM® WebSphere® Application Server, Oracle, IBM® DB2, and so forth. Every year, the list of supported platforms changes due to the changes in versions, support for new platforms, and other factors.
As a recent example, Mozilla has released many versions of the Firefox browser since Version 3.0. This means that it is not only important to run the test, but it is equally important to run all tests on all of the supported test environments. This article explains how you can make sure that your automated tests automatically cover the platforms in your test plan, thus ensure maximum test coverage and detection of defects before you release software.
Figure 1. Manual script
Automated testing using Rational Quality Manager
IBM® Rational® Quality Manager is a web-based test management tool. It integrates with most of the leading automation tools, including Rational Functional Tester, Rational Performance Tester, Rational® Robot, HP Quick Test Professional, and many more. The Rational Quality Manager open RESTful web API makes it easy to integrate with any other tools.
Essentially Rational Quality Manager provides two types of scripts:
- A native manual script with natural language steps, as shown in Figure 1, above
- A remote script, which represents an external automated script as shown in Figure 2, below
Figure 2. Remote functional test script
A manual script captures manual steps in natural language statements, as Figure 1 shows, and is useful for manual testing. Remote scripts, on the other hand, represents placeholders for the actual external automation scripts that can be run by specific external automation tools as shown in Figure 2, above. Each remote script has a type that is based on the type of automation tool. Rational Quality Manager cannot run the remote script; it merely coordinates the execution by delegating it to the automation tool that can actually run the remote script.
Automated tests are typically run on lab machines where the application under test (AUT) and the automation tool are configured to run automated tests. To integrate with Rational Quality Manager, each automation tool needs to have a lightweight utility called an execution adapter to bridge the gap between Rational Quality Manager and the automation tool. Each Rational automation tool has Rational Quality Manager-specific adapter to connect the two.
Planning automated test execution
Rational Quality Manager provides test cases that can be associated with manual or remote scripts to execute the test cases. Planning results in the creation of test case execution records. These records typically cover which test cases to be executed, when to execute them, who will execute them, and — most relevant to this article — which test environment to use. A single test case often has multiple execution records, with each execution record covering a unique test environment.
Registering an execution adapter
When an automation tool-specific adapter is started, it registers to Rational Quality Manager and thereby makes itself available for execution of automated tests. As a result of this registration, Rational Quality Manager creates two resources: adapter and lab resource (test machine). The adapter represents the registered adapter, and it is linked to the lab resource, which captures machine details where the adapter is running. By default, the adapter captures only the machine name, IP address, domain name, and so forth.
To automatically provision full machine details, Rational Quality Manager can integrate with IBM® Tivoli® Provisioning Manager, Tivoli Application Dependency Discovery Manager, and open source StAX (Streaming API for XML) or STAF (Software Testing Automation Framework). The Integration section in Rational Quality Manager online help provides more information (see Resources for a link). Alternatively, you can manually edit the lab resource in Rational Quality Manager to provide details about the machine where the adapter is running.
The command-line adapter (CLA) is included with Rational Quality Manager (see the link in Resources for where to get details). CLA can execute any script that can be run from the command line. It could be as simple as ping command to as complex as a path to a .bat file that contains a complex script. Automation tool-specific adapters are often included when you install the automation tool.
Follow these steps to register a command-line adapter to the Rational Quality Manager server:
- Copy the command-line adapter .zip file from the Rational Quality Manager installation directory to test machine:
...\JazzTeamServer\server\conf\qm\adapters\ RQMCommandLineAdapter.zip - Extract the file into a folder.
- Make sure that Java Runtime Environment (JRE) version 1.5 or later is installed on the test machine
- Open the command window and change the directory to the location where you extracted and saved the contents of the file.
- Run the following command in command window (italic type indicates variables for you to replace with your information):
start.bat -repository https://rqmservername:9443/qm -user user ID -password password -adapter uniqueAdapter ID -adapterName your adapter -projectArea name of the Rational Quality Manager project area
Upon successful registration, the adapter console will display a confirmation message, as shown in Figure 3.
Figure 3. Registering the command-line adapter
Viewing registered execution adapters in the adapter console
In this section, you will learn how to view all execution adapters registered from different machines to Rational Quality Manager server.
- Open your web browser, and enter the Rational Quality Manager URL, for example:
https://server:9443/qm/web/console - Provide valid login information to log into the project area
- Click Execution > Adapter Console to open the console
- Verify that the adapter that you connected from test machine is listed, as shown in Figure 4, and that the Health column for it shows a green icon. (A red icon means that the adapter is not active for a long time after registration.)
Figure 4. Viewing adapter console
Updating test environment information
After execution adapters are registered, Rational Quality Manager creates a lab resource that corresponds to each machine where the adapter is running. By using this lab resource, Rational Quality Manager determines the test environment where the adapter is running. As mentioned earlier, registering an adapter to Rational Quality Manager provides merely minimal information about the machine, such as the host name and IP address. Such information is not sufficient. You can update test environment information in two ways. One way is to use provisioning tools from Tivoli that will update Rational Quality Manager automatically (see the link to
"Rational Quality Manager and Tivoli" in the Resources section).
This article covers only how to update the test environment information for lab resources manually:
- Follows steps in previous section to open the adapter console.
- Click the Machine Name link to open the lab resource associated with the adapter.
- Click the Operating System section. Update it with operating system information.
- Click the Hardware section and provide any details about the machine,
- Click the Software section and add all software that you can to update, such as Browser, Database, Application Server, and so on.
- Click Save to save the test environment information.
Repeat these steps for each adapter that you have registered.
Configuring custom test environment coverage
This section provides step-by-step guidance to make sure that when Rational Quality Manager selects or proposes an execution adapter, the machine where the adapter is running complies with the test environment specified for the test case execution record.
- Log in to Rational Quality Manager as a Project Administrator.
- Click Application Administration > Manage Project Properties (see Figure 5).
Figure 5. Manage project properties
- In the Manage Project Properties view, under the Properties section, click Execution Preference as shown in Figure 6.
Figure 6. Execution preference
- By default, Rational Quality Manager does not make any attempt to match the test environment when executing automated tests. To change this, select the Custom Search radio button, and then select the check box for the type of test environments (operating system, browsers, and so forth) that you want to make sure are matched for searching for the adapter. If you want to match a specific test environment, choose the Environment radio button instead.
- Click Save to update execution preferences.
By following these steps, you have configured Rational Quality Manager to run automated tests on the correct machine. This means that if you create an execution record to execute on Windows XP with the Firefox browser, Rational Quality Manager will verify that the chosen machine adheres to the test environment requirement.
Using a test cell for preparing robust test bed
Often, multiple teams work together on single project. Each team will have its own set of execution adapters connected to a Rational Quality Manager project. You do not want your testing to interfere with someone else's test environment, of course. By using a test cell, you can limit the scope of machines where automated tests will be executed.
To create a test cell:
- Click Lab Management > Create > Test Cell.
- Provide a logical name for the test cell.
- Click the green plus sign icon under the Lab Resource section (see Figure 7).
- Add all machines where you want to run your tests.
- Click the Save button to save the test cell.
Figure 7. Test cell
After test case execution records are generated, you can run individual records manually by clicking the Run button from the Browse Test Case Execution Records view. This is useful when you want to run individual automated tests on an as needed basis.
Often, you will want to create a suite of automated tests and run the full suite to save time. In some situations, you might want to run automated test regularly, according to a schedule. You might also want to run automated tests whenever a new build is created to make sure that there is no regression in the build and to detect problems early, before the whole test team is engaged in testing. In this section, you learn how to run test suites manually for automated test, as well as more advanced options so you can design a workflow of automated tests that can be scheduled to run many times.
Running an automated test by using a test suite
- Create a test suite by following instruction given in the information center (see Resources). Add all of the automated test cases to it, and make sure that you specify the test environment for each test case.
- To specify the test environment:
- Check rows and then click Change Execution Properties.
- Select the test environment.
- Click OK, and Save your test suite.
- Click Run in the test suite editor.
- By default, Rational Quality Manager will select an adapter for each test. To change the default adapter, click Edit Adapter. Alternatively, let Rational Quality Manager distribute tests at run time by selecting the test cell. This will allow selection of adapters to be done by the software within the specified test cell at run time, based on the availability.
- Select Run in parallel, in case tests do not depend on each other, to minimize the overall suite execution time.
- Click OK.
Scheduling an automated test
- Click Execution > Create Execution Schedule.
- Provide a name for the test schedule.
- Select the scheduling options available (see Figure 8):
- Schedule for one time at specified time
- Schedule daily for given time
- Schedule weekly on specified days
- Schedule based on build availability
Note:
The first three options are based on the time schedule. The last option is more useful when you want to run tests each time that a new build is created.
Figure 8. Types of schedules
- In the Steps section, click the + (plus sign) button to add a test suite execution record, as shown in Figure 9. Filter to get the record that you want, and add to the Steps section. You can add as many execution records as you need.
Figure 9. Add a schedule step
- Rational Quality Manager automatically selects available adapters to use for execution. However, to make better use of available adapters at run time, select Choose machines at run time in toolbar. The selected adapter will be set to "Search runtime" as shown in Figure 10.
Figure 10. Search for the adapter at run time
- Associate a test cell with the execution schedule if you wish to isolate the machines where Rational Quality Manager should search for available adapters and not interfere with another test environment.
- Click Save.
Now all steps added in the Execution Schedule will be executed in sequence whenever the schedule is triggered.
Tips:
To manage a scheduled job, select Next scheduled time. Or to cancel schedules, click Execution > Browse > Scheduled Jobs.
Summary of what you have learned
In this article, you learned how to use Rational Quality Manager to run and schedule automated tests for various types of test automation tools, including command-line scripts. The article explained how you can configure Rational Quality Manager so that automated tests run in the test environment according to your test plan and to ensure that all supported platforms have been properly covered. You also learned how to better manage automated testing by using test cells.
| Description | Name | Size | Download method |
|---|---|---|---|
| Code sample for a test cell. | TestCell | 1080 | HTTP |
Information about download methods
Learn
- To learn more on benefits of automated testing, see Manage time and resources better by scheduling automated tests
- See the information center for these instructions mentioned in this article:
- Running Rational command-line execution tests for more about the command line adapter.
- Referencing automated test scripts
- Creating a test suite
- Check the Rational Quality Manager page on IBM® developerWorks® for links to product documentation, articles, tutorials, courses, downloads, and other useful areas. Also, explore the Rational Quality Manager Information Center for technical details and browse the IBM Quality Management page to learn more about what is available.
- For tips, read these developerWorks articles by Michael Kelly:
- Getting started with IBM Rational Quality Manager (December 2008)
- Managing your first project with IBM Rational Quality Manager (October 2008)
- To learn how to integrate Rational automation tools with Rational Quality Manager, read
- Integrating IBM Rational Functional Tester and IBM Rational Performance Tester with IBM Rational Quality Manager
- See Seven Steps to Test Automation Success by Bret Pettichord (June 2001)
- Explore the Rational solution for Collaborative Lifecycle Management.
- Check what is new and noteworthy in Rational Quality Manager.
- To learn how to use Rational Quality Manager with Tivoli provisioning tools, read Rational Quality Manager and Tivoli.
- Subscribe to the developerWorks weekly email newsletter, and choose the topics to follow.
Get products and technologies
- Download the free trial version of Rational Quality Manager.
- Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.
Discuss
- Join the Rational Quality Manager forum, which also includes discussions about Rational Test Lab Manager.
- Rate or review Rational software. It's quick and easy.
- Share your knowledge and help others who use Rational software by writing a developerWorks article. Find out what makes a good developerWorks article and how to proceed.
- Follow Rational software on Facebook, Twitter (@ibmrational), and YouTube, and add your comments and requests.
- Ask and answer questions and increase your expertise
when you get involved in the Rational forums, cafés, and wikis.
- Get social about thought leadership. Join the Rational community to share your Rational software expertise and get connected with your peers.
Pramod Chandoria works at the Rational Software Lab for IBM, in India. He represents Test Automation in defining the first version of the Automation OSLC specification. He has been leading design and development of Rational Quality Manager since Version 1.0. Test execution and integration with test automation tools are his areas of expertise. Pramod is a graduate of the Indian Institute of Technology, Roorkee, with a degree in computer science, and has nearly nine years of product development experience.
