Create modular performance tests to save time and increase reuse: Part 1. Use Rational Performance Tester to exchange data between modular tests

This article series focuses on creating modular performance tests by using the IBM® Rational® Performance Tester split test functionality and then dynamically exchanging data between tests by using the test variable functionality. It explains how to do these tasks: derive test modules from existing tests while keeping the original tests intact, split and annotate test in real time during recording by using the Rational Performance Tester Annotation toolbar, link test modules together by using the test variable functionality, and construct user scenarios by using various test modules and modeling a workload. The tutorial (Part 2) linked with this article gives step-by-step instructions that show how to use a test variable to share data across tests.

Share:

Karline Vilme (kvilme@us.ibm.com), Software Quality Engineer, IBM

Author photoKarline is a staff software quality engineer with the IBM Software Group. She focuses primarily on system verification, integration, and performance testing for the Rational Performance Tester tool. She joined IBM in 2000 and has held several system administrator and senior customer support positions with the IBM Global Services group.



05 October 2010

Also available in Chinese Russian Portuguese

The advantages of using modular tests

One of the core activities of performance testing is test development and customization. As a tester, you probably spend a lot of time developing and customizing tests that often need to be rerecorded because of changes in the application or system under test or to the use cases. For example, you might need to change your test scenario to target a different part of the system to investigate a new performance issue discovered during test runs. This will probably require a new set of tests, which you will have to record and customize, thus taking valuable time away from test execution and results analysis.

Test modularity helps in several ways:

  • Minimizes the need to rerecord your tests each time something changes
  • Eliminates redundant steps from your tests
  • Reduces the size of your tests
  • Improves robustness and reusability of your tests
  • Makes it easier to debug your tests and to change the focus of test scenarios.

Using modular tests involves creating smaller test modules. You can then use the test modules as building blocks to build larger or smaller scenarios, depending on the test objectives. This article explains how IBM® Rational® Performance Tester functionalities, such as split test and test variable, can help you create modular tests that are more robust and reusable.


Terminology

Before delving into these functionalities, study the meanings of the following definitions in the context of Rational Performance Tester:

Test module
A smaller test that is split from a larger test, with intent to reuse it in different scenarios. Rational Performance Tester does not treat this test module any different from a regular test.
Split test
Ability to create multiple test modules from a single test.
Test variable
A user-defined name-value pair that stores and refers to information throughout a test and between tests.

Creating modular test recordings

As of the Rational Performance Tester Version 8.1 release, you can use the split test functionality to divide a larger test into smaller test modules while keeping the original test unchanged. By using the Annotation toolbar (as of Version 8.1.1), you can also identify test module boundaries during the recording process. You can then use the test variables functionality to link various test modules together.

Test variables

  • Do not include the test variables with the selections for the split test.
  • Always select the server connection with the login transaction for the login test for SAP.
  • For an HTTP test, the server connections are automatically included as test variables and will persist with any split test.

In the next three sections, you will learn how to do these tasks:

  • Split a SAP test after recording
  • Split an HTT test after and during recording (in real time)
  • Annotate an HTTP test during recording

Although this article focuses on the SAP GUI and HTTP extension, all other Rational Performance Tester extensions also support the split test and test variable functionality.

Splitting a SAP test

Split tests

  • The steps listed are for Rational Performance Tester v8.1.1, but they are similar for v8.1 and later, except that you will select split test for the last step.
  • SAP tests can be split only at the transaction level. This is necessary because SAP requests and responses are captured in the context of screens. If the screens get out of order, the test will malfunction during playback.
  • Do not include the test variables container with the selections to save as a new split test.
  • For a SAP test, always select the server connection with the login transaction for the login test.
  • For an HTTP test, the server connections are automatically included as a sub-container under the test variables container. It will persist with any split test.

After recording, you can split a SAP test by selecting one or more transactions in the test editor and save the selections as a new test. When selecting multiple transactions, they must be contiguous. Otherwise, the option to save the selections as a new test will be disabled. This is important because there might be dependencies between the transactions.

The steps to split a test are very simple and quick:

  1. Open the test in the test editor.
  2. Select a test transaction (see Figure 1).
  3. Right-click and save the selection as a new test, which brings up the split test dialog so that you can rename and add comments to the test.
Figure 1. Selecting transactions for a split test
Screen captures of both single and multiple selections

Figure 2 shows an example where three different test modules are derived from a test from a single SAP test recording. The test modules include Login, Create order, and Logout tests.

Figure 2. Multiple test modules derived from a single SAP recording
Single test split into three test modules

Split test variables

  • A test variables container persists with each split test module. It contains the test variables from the original test. You can add and remove variables as necessary.
  • HTTP split tests include Rational Performance Tester-generated test variables, which are created automatically during the splitting process. There are stored in the "Variables Created by Split Test" container.
  • Rational Performance Tester automatically determines which variables need to be persisted among the split test and creates the links so that you do not have to write custom code. You can add or remove variables as necessary.

Splitting an HTTP test

The steps for splitting an HTTP test script are similar to those listed previously for SAP, with only a few differences:

  • HTTP tests can be split during recording by using the Rational Performance Tester Annotation toolbar to insert split points.
  • HTTP tests can be split at the page level (see Figure 3). However, it is possible to split them at the transaction level if the test contains transactions.
Figure 3. Selecting pages for an HTTP split test
'Save Selection As New Test' selected

Figure 4 shows an example of multiple split test modules derived from a test from a single HTTP test recording. The modules include Login, Import, Create, and Logout tests.

Figure 4. Multiple test modules derived from a single HTTP recording
A single HTTP test split into four test modules

All of the test modules in Figure 4 include Rational Performance Tester-generated test variables. They are test variables automatically generated and set while splitting a test to create links among the test modules. In addition, some of the test modules include variables set by Rational Performance Tester that are used by subsequent test modules.

Splitting an HTTP test during recording

Using the Rational Performance Tester Annotation toolbar, you can split HTTP test in real time during recording. While recording a test, you can use the Annotation toolbar to insert split points into the test. After recording is finished, Rational Performance Tester automatically generates the individual test modules, based on the inserted split points and test names provided.

Notations and names

  • The Rational Performance Tester Annotation toolbar must be installed (in the browser) to create HTTP split tests during recording.
  • You are prompted to install the Annotation toolbar in the browser upon the first (HTTP) recording after installing Rational Performance Tester.
  • Use comments to annotate the tests during recording. This will make it easier to understand and split a test after recording.
  • It is possible to regenerate a single complete test from a recording that contains split points.
  • When naming a split test, the name is for the test that will be generated for the previous section, not the next section.

If, after recording is finished, you realize that you split the test incorrectly or for some other reasons want a single test for the complete recording instead, you can use the recmodel file from the original recording that contains the split points to regenerate a complete test without splits.

The Annotation toolbar also provides additional functionalities that you can use to annotate your test during recording. It enables you to insert comments and transactions to your test in real time. In addition, you can use it to rename pages while recording to give them names that are more meaningful. These tasks can be performed after recording the test as well but it is often more convenient and efficient to perform them while recording.

When providing a name for a split test during recording, it is important to remember that this name will be associated with the test generated for the previous section of the recording session, not the upcoming section.

Figure 4 shows an example of a recording that was annotated with comments using the Annotation toolbar. For example, the first two pages were renamed with more meaningful names such as Login and Home. The comments make it easier to understand the purpose of the test and where you can possibly split the test after the recording session completes.

Figure 5. Recorder Test Annotation toolbar
Labels show what toolbar icons mean

Reminders

  • When combining test modules from different HTTP test recordings, always review the Rational Performance Tester-generated split test variables. Ensure that the variables from a subsequent test module are created and assigned with values by a test module that is executed before it.
  • It is critical that you document dependencies between the test modules. This will help with maintenance and reusability.
  • You can use comments or the test properties description field associated with each test to document dependencies and other information about your tests and schedules.

Now that you have learned about using the Rational Performance Tester split test functionality to create modular test scripts, in the next sections, you will learn how to use them for these purposes:

  • Constructing test scenarios by using various test modules
  • Linking various test modules together by using the Test Variable functionality so that they can exchange data

Constructing a test scenario from test modules

As depicted in Figure 2 and Figure 4, you can create test modules from tests after and during recording. Each split test represents an interchangeable test module that you can reorganize and combine into bigger or smaller scenarios. Figure 6 shows an example of various test modules (A through D) being used to create different test scenarios. It also shows that some of the modules, such as A and D, are reused in several scenarios combination.

Figure 6. Creating test scenarios by using test modules
Using test modules to build test scenarios

Best practices

Consider these best practices when recording and splitting your test scripts:

  • When recording a test, always return to the main screen or page at the end of each scenario or before a logout. Having common beginning and ending screens or pages provides a known state to serve as a reliable entry point for other tests. This will help with long-term maintenance as you add more modules.
  • Always document dependencies and other information about your split test modules. This will help with test script maintenance and reusability. You can do this by inserting comments in the test or schedule or by using the properties description field associated with each test. Inserting comments before each test module is more useful, because those comments are more visible.

Dynamically exchanging data between tests by using test variables

The "test variable" functionality plays an essential role in the split test process. After you split a test recording into smaller test modules, you need a mechanism for sharing data between them. The Rational Performance Tester test variable provides this functionality. You can define a variable in one test to capture and store test data and then reference the test variable in a second test. If you set the visibility (scope) of the test variable to "All tests for this user" rather than "This test only," Rational Performance Tester persists the value of the test variable at the end of the test. The same test variable referenced in a future test will share the same value for that virtual user, provided that the test variable was also defined with a visibility of "All tests for this user."

Test variable use

From a programming perspective, using a test variable in Rational Performance Tester is similar to those in other programming languages. You can declare and initialize a test variable, set its scope, and change its value. The differences between using variables in Rational Performance Tester tests and other programs are that you can perform all of these tasks using a graphical user interface and the software takes care of the coding for you. Also specific to Rational Performance Tester is the ability to handle errors at run time and to share variables across test modules.

Figure 7. Test variable use flow diagram
Flow diagram

Larger view of Figure 7.

Variable declaration

After you have identified the test modules that you want to link together, the next step is declaring variables and assigning them accordingly.

The steps for declaring variable are quick and simple:

  1. Right-click the Test Variables container.
  2. Click Add.
  3. Click Variable Declaration and enter a variable name.
Figure 8. Variable declaration
Drop-down menus showing the selections described

The Test Variables container in a test that is generated from a recording cannot be deleted, but it can be renamed. As you define more test variables for a test, you can add sub-containers to help organize them. It is always good to use meaningful names for your test variables and the containers that you define.

A variable is initialized with an empty text string as its default value when declared. You can substitute this value with your own text string or a value from another data source, such as a data pool, reference, custom code, or another test variable. Optionally, you can change or uninitialize the test variable by using the variable assignment operator. See Figure 11 for an example. It is also possible to access and change a test variable through custom code.

In addition, you have the option of handling runtime errors when a variable is not initialized. You can choose to do nothing, issue a test log error or warning, or exit the test. Although runtime error handling is optional, handling runtime errors is especially important when sharing data across tests. An error could occur in the first test and prevent the variable's value from being set, thus resulting in the other tests receiving a null value. See Figure 9 for an example.

Variable scope

A test variable is stored in the data area during execution. Its scope determines whether it is visible to other tests or not.

Figure 7 and Figure 9 show two options for setting a variable scope. You can set a variable scope to be visible in "This test only" or "All tests for this user." This concept is similar to that of local or global/class variables in other programming languages.

Figure 9. Setting the scope of variables
Fields associated with a test variable

Using test variables

  • Test variables can be referenced throughout a test, and values can be persisted between tests.
  • A local test variable is not stored in the user data area at runtime. It must be explicitly passed into custom code as an argument, and the variable can be set to the custom code returns value. Figure 11 shows an example of a test variable set to a custom code, which returns a value.

Local variable use

A local variable is a variable that has its scope set to be visible in "This test only." This means that you can reference this variable only in the test that it is declared but not across other tests. This is because data set by one test is not visible to other tests, even if the other test contains a variable with the same name. The test variable is only accessible through custom code.

Figure 10 shows an example of local variable use, where test variables are initialized by using values from a data pool (login credentials). The test variable is then substituted for the test's recorded values (username and password). By just changing the scope of these variables, you can reference them in other tests that use the same login information.

Figure 10. Test variable initialized with data pool value
Assigning a test variable value from a data pool

Per-user variable use

The scope of a per-user variable is set to be visible in "All tests for this user," which makes it visible to all tests being executed by the same virtual user. You can reference a per-user variable from another test by declaring a variable with the same name and setting its scope to "All tests for this user" in the second test. The scope of the variable from the first test must also be set to "All tests for this user" for it to be visible to the second test.

Shared variables

  • To share variables across tests, both tests must contain a variable with the same name and the scope of both variables must be set to be visible to "All tests for this user."
  • Test variables are not shared between users but among different tests by the same virtual user
  • Per-user test variables are stored in the user data area. You can also access and change their values using custom code.
  • Always initialize and handle runtime errors for your test variables.

Handling runtime errors for test variables is very important, especially when sharing them with other tests. There is a possibility that some errors could occur during execution of the first test, thus preventing the variable value from being set. If the variable is not initialized, you need to decide what the next step should be. Rational Performance Tester provides the following options that you can choose from (see Figure 7):

  • Do nothing
  • Issue test log warning/error
  • Exit the test

Figures 11 and 12 show examples of per-user use, where data from one test is passed on to the other test by using test variables.

In Figure 11, the orderNumber test variable is declared and initialized with a "no-value" text string. Subsequently, its value is set to an order number returned by a custom code action by using a variable assignment action.

Figure 11. Example of test variable use in a test
A test script showing test variable use

Figure 12 shows a second test module (VA02) that is using a variable with the same name as the VA01 test module. The VA02 test module deletes the orders that the VA01 test module creates. In this example, the VA02 test module references data from the VA01 test rather than, for example, from a user-generated data pool.

Figure 12. Referencing a test variable from another test
Test referencing variable set in another test

Now that you have learned how to create modular test cases and link them by using test variables, the next sections show you an example of modeling your schedule by using the test modules.


Modeling your workload

When you are finished with developing and customizing your test modules, you can organize and combine various test modules together to emulate a workload scenario.

Figure 13 shows an example of various test modules being used to model a workload scenario. In this example, the Login, Create order, Logout test modules were all derived from a single test (VA01), using the split test functionality. These modules are then organized and combined with the Delete order test module (VA02) derived from another test.

Schedule structure

When building your schedule, it's essential that you maintain the general structure of the test.

For example, SAP test structure includes a login, other tasks, and a logout. This structure must be maintained to ensure that the test plays back correctly

As previously shown in Figure 11 and Figure 12, the Create order and Delete order test modules are linked through a test variable named orderNumber.

Figure 13. Schedule models with various SAP test modules
Example of a schedule using SAP test modules

This workload scenario can be expanded later to include other functions by just inserting other test modules between the login and logout. Likewise, it can be made smaller by removing or disabling the test modules that are not needed for a particular scenario.

When modeling a workload using HTTP test recordings, always review the Rational Performance Tester-generated split test variables to ensure that the variables from a subsequent test module are created and assigned with values by a test module that is executed before it.

Best practices

  • Always maintain the general structure of a test when modeling your workload scenario using the test modules. It should include a beginning and an end. The SAP scenario, for example, should always include a login and a logout test.
  • When building your workload scenarios using various test modules, ensure that they are organized in a flow that is valid and consistent with the application use.
  • Be aware of dependencies between the modules. If test modules are reordered, ensure that shared test variables used are initialized and set appropriately. This is one of the reasons why documenting your test module's dependencies is necessary.
  • Always examine test scripts before execution to ensure that you have created and assigned all expected variables accordingly.

Benefits of using split tests and test variables

These are some of the benefits you can achieve using the split test and test variable functionality:

  • Derive test modules from a single recording, while keeping the original test intact.
  • Eliminate redundant steps from tests in these three ways:
    • Reduces the size of tests and maintenance time
    • Improves reusability and robustness of tests
    • Makes tests easier to debug
  • Build bigger user scenarios by reorganizing and combining test modules
  • Make it easier to quickly change user scenarios, targeting different parts of the system when issues arise.

Summary and tutorial

This article focused on creating modular performance tests with Rational Performance Tester using the "split test" functionality and exchanging data between the test modules using the "test variable" functionality.

By using the split test functionality, you can derive test modules from existing recordings while keeping the original scripts unchanged. In addition, you can also split and annotate a test while recording using the Annotation toolbar.

The test variable functionality facilitates the exchange of data between the test modules. The test variable is used to reference data throughout a test and between tests. The test variable is also used to exchange data between regular tests that have not been modularized.

The test modules can be used interchangeably to construct bigger or smaller user scenarios, depending on the test objectives.

Test modularity provides several benefits:

  • Helps minimize the need to rerecord your tests each time something changes
  • Eliminates redundant steps from your tests
  • Reduces the size of your tests, improves robustness and reusability of your tests
  • Makes it easier to debug your tests and to change the focus of test scenarios

Now you're ready to proceed to Part 2: Tutorial, which is a real-world example of using Rational Performance Tester test variables. Work through it step by step to get hands-on experience.


Acknowledgements

Thank you to the following people for taking the time to review and provide comments for this article: Kent Siefkes, Sanjay Joshi, Alex Bernstein, Joel Duquene, and Jean-Philippe Delpiroux. In addition, thanks to Arif Muhammed for providing the HTTP screenshots depicted in the tutorial example Figures 10 and 11.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=548623
ArticleTitle=Create modular performance tests to save time and increase reuse: Part 1. Use Rational Performance Tester to exchange data between modular tests
publish-date=10052010