Data Population for Jazz-based Products

Create large volumes of test data by using Rational Performance Tester, Jazz Foundation Services, and the Mozilla Firefox Poster plug-in

Preparing to test a large volume of data is an important part of performance tests. Still, there are few efficient ways to do this work well, especially during the early part of the software development lifecycle. IBM® Jazz™ technology-based software is built on a RESTful, resource-oriented architecture, where a create, relieve, update, and delete operation can be implemented by using HTTP protocols. This article explains a method that leverages both IBM® Rational® Performance Tester and the Mozilla Firefox Poster plug-in. This method does not depend on product build changes and application interface changes, and it also provides better testing performance than other current approaches.

Li Ping Li (liliping@cn.ibm.com), Staff Software Engineer, IBM

photo of Li Ping LiLi Ping Li is a Rational system performance test engineer in the IBM China Development Lab, where she works primarily with IBM Jazz-based products, especially IBM Rational Requirements Composer, Rational Quality Manager, and Rational Project Conductor. She is working on performance benchmark tests for Jazz-based products.



Dan Liu (liudcdl@cn.ibm.com), Software Engineer, IBM  

author photoDan Liu has worked as a system tester with IBM Rational software for two years. Currently, Ms. Liu focuses on performance tests for Jazz-based products and test automation.



Zhao Hui Ding (zhding@cn.ibm.com), Contractor, The Symbio Group

photo of Zhao Hui DingZhao Hui Ding is on the system performance test team at the IBM China Development Lab. She works primarily on the performance tests for Jazz-based products.



11 March 2010

Also available in Chinese Portuguese Spanish

Compare test data generation methods

For a performance test, you must find efficient ways to create large data volumes for testing during the early stages of software development. The specific data volumes must simulate a real product environment, and the data sizes must vary from small to large so that you can find the performance bottleneck of the product that is being tested.

Table 1 contains examples of IBM® Rational® Quality Manager data volume definitions.

Table 1. Data volume definitions from Rational Quality Manager
ItemType Low Med High
WorkItem 6888 19068 31048
TestCase 3964 9152 17701
CategoryType 10 10 10
Category 0 0 0
TestPlan 101 148 196
TestExecutionRecord 2394 2666 4661
ExecutionResult 0 0 1
TestPhase 202 296 392
AssetConfiguration 404 450 1363
RequirementItem 0 0 0
TestEnvironment 270 270 553
LabResource 601 2156 2410
Request 4971 13062 11256
Reservation 954 3355 4116

Vaughn Rokosz, an IBM Rational systems verification tester, has defined the following approaches for adding test data into a system:

  • Use test automation tools, such as IBM® Rational® Performance Tester or IBM® Rational® Functional Tester, to populate the test environment by simulating user activities.
  • Use public APIs or command line tools to develop specialized population tools.
  • Import test data from customers if the application supports importing.
  • Use the bulk load tools that are provided with relational databases to initialize database tables.
  • Use internal APIs to develop specialized population tools.

Some of these approaches for adding test data to a system can be challenging to implement. For example, applications that are based on Jazz Foundation Services access repositories through storage services. This logic data model makes it impractical to directly load database tables by using bulk load tools that are described in the fourth method.

Another challenge can occur at the beginning of the application development lifecycle. The user interface is typically not available, or it is unstable to use. This challenge makes methods 1 and 3 not feasible because both methods are based on client operations. Even when the application is stable enough, these two methods are time-consuming.

To address these challenges, you need to develop a method for populating data by using the API against the current build code. The risk of using this method is that it can require a large effort to accommodate build changes. To address this problem, you need to find an effective tool that can work independently from build changes.


Discover the value of Jazz Foundation Services

IBM® Jazz™ technology-based applications are considered to be extensions of Jazz Foundation Services, which are based on a Representational State Transfer, or RESTful specifications, and resource-oriented architecture. Each application provides services and data as REST APIs. In these cases, data can be exposed as addressable resources, especially as a URI. On the other side, it works as a Web service application that provides a create, read, update, and delete interface for its resources. These interfaces are implemented by the POST, GET, UPDATE, and DELETE methods within the HTTP protocol. This loosely coupled architecture enables different products to integrate with each other and enables clients to communicate with the Jazz Team Server in the same way no matter how the service is implemented.

Figure 1. Jazz Foundation Services
Diagram: Each Jazz technology-based applications provides REST API

Populate data based on the REST API

Based on the REST API that is provided by Jazz-based applications, we proposed a data population method that takes the advantage of approaches 1 and 2, which were outlined in the first section. To illustrate how you can achieve this method, you can use Rational Quality Manager as the targeting server.

For this exercise, the data population goal is to create 20 test plans and add 1000 test cases to each plan. Other tools that we can use to illustrate this method include Rational Performance Tester, the Firefox browser, and the Firefox Poster plug-in.

For this example, the authors assume that you can create a template that can be assured before a system verification test (SVT) entry. The template file for this test plan example is testplan_001, which contains one test case, named testcase_001.

Get a resource template through REST API

Some applications provide the REST API during installation. For example, you can get an XML template of various resources from the Samples folder, which is in the installation directory. You can also access the XML schema of the test plan by URI:
https://<RQMServer>:<Port>/jazz/secure/service/com.ibm.rqm.integration.service.IIntegrationService/resources/Quality+Manager/testplan?metadata=schema

If you do not have this template information, you can get the XML template of the target resource through the REST API as long as there is a target resource in the repository. You must also ensure that the URI of this resource is available. You can get the URI through one of the following methods:

  • An API document that you can get from a developer
  • A link to information that is on the client side
  • The HTTP protocol data that is recorded by Rational Performance Tester to get the URI of the specified resource

After you get the URI, you can use it to get the resource template for each record type. The template is independent of the product build change because it is retrieved from the current repository. Here is an example from the record type test case in Rational Quality Manager:

  1. Use the URI to get the test case list in the browser.
Figure 2. Get the test case list
All test cases are displayed
  1. In the results, select testcase_001. To open it, right-click on the page and select View > Page source. As Figure 3 shows, you can get the source code for this test case record in the XML format, save this page source to a file named testcase_001.xml which will be the resource template file for test case record.
Figure 3. Code in the XML template of testcase_001
XML content of testcase_001

Larger view of Figure 3.

  1. You can use these same steps to get the XML template file for the test plan, testplan_001.xml.

Record operations with the Firefox Poster plug-in

Poster is a plug-in for the Mozilla Firefox browser that can simulate the interaction between an application and an HTTP server. You can send and receive content from a URI by using the standard HTTP actions: PUT, POST, GET, or DELETE.

To use Rational Performance Tester and the Poster plug-in to automatically create the data repository, follow these steps:

  1. Start the Rational Performance Tester HTTP recorder. Firefox with the Poster add-on opens and records all of the requests and responses that are sent and received by the browser.
  2. If you are connecting to the Jazz server for the first time, you will be prompted to log in. Because Poster runs as a Firefox add-on, it shares the authenticated session from your login to the Jazz server. Rational Performance Tester can record and handle this session automatically.
  3. Check the resource operations that were performed by Poster. Review these three areas in the Poster dialog (see Figure 4):
    • URL: Make sure that this corresponds to the resource that you want to interact with.
    • Action: Select GET/POST/PUT to respectively retrieve, add, or modify the content.
    • Content to send: Used in a POST/PUT where content can be added or modified. An element value such as identifier or title must be changed in this step to make the result correct and easily verified,
  4. Click Go, and it will send the request to the server. If you get a response code of 201, then a new resource was created by the POST action. If you get a response code of 200, then the resource was modified by the PUT action.
Figure 4. Use the Poster plug-in to generate testcase_001
Poster window with URL, action, and content fields

By using Rational Performance Tester and Poster to record operations as below, you have generated a test script that simulates the scenario for our example. See yellow highlited page names of Rational Performance Tester test script in Figure 5:

  1. Login
  2. Create testplan: create one test plan (POST by Poster)
  3. Get testplan: get new created test plan (GET by Poster or by browser)
  4. Create testcase: create one test case (POST by Poster)
  5. Modify testplan: add new created test case to test plan (PUT by Poster)

Edit the test script

When editing test scripts, you must edit these elements (Figure 5) to incorporate the Rational Performance Tester data pool, custom code, reference, and other all functions that can be used.

  1. Add a loop or make a schedule to generate large data volumes.
  2. See the loop example in Figure 5. It can generate 20 test plans that each have 1000 test cases.
  3. Enter some values in the request URI or data content.
  4. The script is independent of resource type. You can substitute URI with other resource types to generate various resources. A resource ID can be substituted to generate resources with different IDs. To make the resource unique, edit the resource title, description, and other elements. See the right panel of Figure 5 for an example.
  5. Create a data correlation to support the resource relationship.
Figure 5. Edit the test script
Loop, URL, and Data edits highlighted onscreen

Larger view of Figure 5.

In Figure 5, notice the custom code ModifyCase and ModifyPlan to make each test plan has its 1000 test cases. In each test plan XML data content, one test case is linked by adding one XML element shown as test case template in Figure 6.

The custom code in ModifyCase uses test case template as the input parameter to generate the 1000 test cases XML document. Its output will be merged into an test plan XML document that is retrieved from the get testplan request by the ModifyPlan custom code to generate a modified test plan. The code merges the XML documents, you can also find flow diagram of how the XML file merges in Figure 6. Flexible input argument settings in Rational Performance Tester make the target resource content independent.

Figure 6. Create a resource relationship by using custom code and data correlation
Flow diagram of how the XML file merges

Summary

This data generation method can automatically populate data that is independent of the development environment and application interface. To implement this method, all you need is the REST API of the product, because this method is independent of any build change. Also, because this method works directly on the server resource through the API, its performance is better than other approaches that were outlined in the first section. For example, you can populate 1,000 Rational Requirement Composer resources in 4 minutes by using this method. If we had used one of the other uploading approaches that we described in the first section, the process would have taken longer than one hour.

As Jazz and RESTful services become popular in application development, the method that you learned in this article can be valuable not only for testing but also for other stages of the application development lifecycle.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=472011
ArticleTitle=Data Population for Jazz-based Products
publish-date=03112010