![Close [x]](http://dw1.s81c.com/i/c.gif){kind=small}
The advantages of using modular tests
One of the core activities of performance testing is test development and customization. As a tester, you probably spend a lot of time developing and customizing tests that often need to be rerecorded because of changes in the application or system under test or to the use cases. For example, you might need to change your test scenario to target a different part of the system to investigate a new performance issue discovered during test runs. This will probably require a new set of tests, which you will have to record and customize, thus taking valuable time away from test execution and results analysis.
Test modularity helps in several ways:
- Minimizes the need to rerecord your tests each time something changes
- Eliminates redundant steps from your tests
- Reduces the size of your tests
- Improves robustness and reusability of your tests
- Makes it easier to debug your tests and to change the focus of test scenarios.
Using modular tests involves creating smaller test modules. You can then use the test modules as building blocks to build larger or smaller scenarios, depending on the test objectives. This article explains how IBM® Rational® Performance Tester functionalities, such as split test and test variable, can help you create modular tests that are more robust and reusable.
Before delving into these functionalities, study the meanings of the following definitions in the context of Rational Performance Tester:
- Test module
- A smaller test that is split from a larger test, with intent to reuse it in different scenarios. Rational Performance Tester does not treat this test module any different from a regular test.
- Split test
- Ability to create multiple test modules from a single test.
- Test variable
- A user-defined name-value pair that stores and refers to information throughout a test and between tests.
Creating modular test recordings
As of the Rational Performance Tester Version 8.1 release, you can use the split test functionality to divide a larger test into smaller test modules while keeping the original test unchanged. By using the Annotation toolbar (as of Version 8.1.1), you can also identify test module boundaries during the recording process. You can then use the test variables functionality to link various test modules together.
In the next three sections, you will learn how to do these tasks:
- Split a SAP test after recording
- Split an HTT test after and during recording (in real time)
- Annotate an HTTP test during recording
Although this article focuses on the SAP GUI and HTTP extension, all other Rational Performance Tester extensions also support the split test and test variable functionality.
After recording, you can split a SAP test by selecting one or more transactions in the test editor and save the selections as a new test. When selecting multiple transactions, they must be contiguous. Otherwise, the option to save the selections as a new test will be disabled. This is important because there might be dependencies between the transactions.
The steps to split a test are very simple and quick:
- Open the test in the test editor.
- Select a test transaction (see Figure 1).
- Right-click and save the selection as a new test, which brings up the split test dialog so that you can rename and add comments to the test.
Figure 1. Selecting transactions for a split test
Figure 2 shows an example where three different test modules are derived from a test from a single SAP test recording. The test modules include Login, Create order, and Logout tests.
Figure 2. Multiple test modules derived from a single SAP recording
The steps for splitting an HTTP test script are similar to those listed previously for SAP, with only a few differences:
- HTTP tests can be split during recording by using the Rational Performance Tester Annotation toolbar to insert split points.
- HTTP tests can be split at the page level (see Figure 3). However, it is possible to split them at the transaction level if the test contains transactions.
Figure 3. Selecting pages for an HTTP split test
Figure 4 shows an example of multiple split test modules derived from a test from a single HTTP test recording. The modules include Login, Import, Create, and Logout tests.
Figure 4. Multiple test modules derived from a single HTTP recording
All of the test modules in Figure 4 include Rational Performance Tester-generated test variables. They are test variables automatically generated and set while splitting a test to create links among the test modules. In addition, some of the test modules include variables set by Rational Performance Tester that are used by subsequent test modules.
Splitting an HTTP test during recording
Using the Rational Performance Tester Annotation toolbar, you can split HTTP test in real time during recording. While recording a test, you can use the Annotation toolbar to insert split points into the test. After recording is finished, Rational Performance Tester automatically generates the individual test modules, based on the inserted split points and test names provided.
If, after recording is finished, you realize that you split the test incorrectly or for some other reasons want a single test for the complete recording instead, you can use the recmodel file from the original recording that contains the split points to regenerate a complete test without splits.
The Annotation toolbar also provides additional functionalities that you can use to annotate your test during recording. It enables you to insert comments and transactions to your test in real time. In addition, you can use it to rename pages while recording to give them names that are more meaningful. These tasks can be performed after recording the test as well but it is often more convenient and efficient to perform them while recording.
When providing a name for a split test during recording, it is important to remember that this name will be associated with the test generated for the previous section of the recording session, not the upcoming section.
Figure 4 shows an example of a recording that was annotated with comments using the Annotation toolbar. For example, the first two pages were renamed with more meaningful names such as Login and Home. The comments make it easier to understand the purpose of the test and where you can possibly split the test after the recording session completes.
Figure 5. Recorder Test Annotation toolbar
Now that you have learned about using the Rational Performance Tester split test functionality to create modular test scripts, in the next sections, you will learn how to use them for these purposes:
- Constructing test scenarios by using various test modules
- Linking various test modules together by using the Test Variable functionality so that they can exchange data
Constructing a test scenario from test modules
As depicted in Figure 2 and Figure 4, you can create test modules from tests after and during recording. Each split test represents an interchangeable test module that you can reorganize and combine into bigger or smaller scenarios. Figure 6 shows an example of various test modules (A through D) being used to create different test scenarios. It also shows that some of the modules, such as A and D, are reused in several scenarios combination.
Figure 6. Creating test scenarios by using test modules
Consider these best practices when recording and splitting your test scripts:
- When recording a test, always return to the main screen or page at the end of each scenario or before a logout. Having common beginning and ending screens or pages provides a known state to serve as a reliable entry point for other tests. This will help with long-term maintenance as you add more modules.
- Always document dependencies and other information about your split test modules. This will help with test script maintenance and reusability. You can do this by inserting comments in the test or schedule or by using the properties description field associated with each test. Inserting comments before each test module is more useful, because those comments are more visible.
Dynamically exchanging data between tests by using test variables
The "test variable" functionality plays an essential role in the split test process. After you split a test recording into smaller test modules, you need a mechanism for sharing data between them. The Rational Performance Tester test variable provides this functionality. You can define a variable in one test to capture and store test data and then reference the test variable in a second test. If you set the visibility (scope) of the test variable to "All tests for this user" rather than "This test only," Rational Performance Tester persists the value of the test variable at the end of the test. The same test variable referenced in a future test will share the same value for that virtual user, provided that the test variable was also defined with a visibility of "All tests for this user."
From a programming perspective, using a test variable in Rational Performance Tester is similar to those in other programming languages. You can declare and initialize a test variable, set its scope, and change its value. The differences between using variables in Rational Performance Tester tests and other programs are that you can perform all of these tasks using a graphical user interface and the software takes care of the coding for you. Also specific to Rational Performance Tester is the ability to handle errors at run time and to share variables across test modules.
Figure 7. Test variable use flow diagram
Variable declaration
After you have identified the test modules that you want to link together, the next step is declaring variables and assigning them accordingly.
The steps for declaring variable are quick and simple:
- Right-click the Test Variables container.
- Click Add.
- Click Variable Declaration and enter a variable name.
Figure 8. Variable declaration
The Test Variables container in a test that is generated from a recording cannot be deleted, but it can be renamed. As you define more test variables for a test, you can add sub-containers to help organize them. It is always good to use meaningful names for your test variables and the containers that you define.
A variable is initialized with an empty text string as its default value when declared. You can substitute this value with your own text string or a value from another data source, such as a data pool, reference, custom code, or another test variable. Optionally, you can change or uninitialize the test variable by using the variable assignment operator. See Figure 11 for an example. It is also possible to access and change a test variable through custom code.
In addition, you have the option of handling runtime errors when a variable is not initialized. You can choose to do nothing, issue a test log error or warning, or exit the test. Although runtime error handling is optional, handling runtime errors is especially important when sharing data across tests. An error could occur in the first test and prevent the variable's value from being set, thus resulting in the other tests receiving a null value. See Figure 9 for an example.
Variable scope
A test variable is stored in the data area during execution. Its scope determines whether it is visible to other tests or not.
Figure 7 and Figure 9 show two options for setting a variable scope. You can set a variable scope to be visible in "This test only" or "All tests for this user." This concept is similar to that of local or global/class variables in other programming languages.
Figure 9. Setting the scope of variables
Local variable use
A local variable is a variable that has its scope set to be visible in "This test only." This means that you can reference this variable only in the test that it is declared but not across other tests. This is because data set by one test is not visible to other tests, even if the other test contains a variable with the same name. The test variable is only accessible through custom code.
Figure 10 shows an example of local variable use, where test variables are initialized by using values from a data pool (login credentials). The test variable is then substituted for the test's recorded values (username and password). By just changing the scope of these variables, you can reference them in other tests that use the same login information.
Figure 10. Test variable initialized with data pool value
Per-user variable use
The scope of a per-user variable is set to be visible in "All tests for this user," which makes it visible to all tests being executed by the same virtual user. You can reference a per-user variable from another test by declaring a variable with the same name and setting its scope to "All tests for this user" in the second test. The scope of the variable from the first test must also be set to "All tests for this user" for it to be visible to the second test.
Handling runtime errors for test variables is very important, especially when sharing them with other tests. There is a possibility that some errors could occur during execution of the first test, thus preventing the variable value from being set. If the variable is not initialized, you need to decide what the next step should be. Rational Performance Tester provides the following options that you can choose from (see Figure 7):
- Do nothing
- Issue test log warning/error
- Exit the test
Figures 11 and 12 show examples of per-user use, where data from one test is passed on to the other test by using test variables.
In Figure 11, the orderNumber test variable is declared and initialized with a "no-value" text string. Subsequently, its value is set to an order number returned by a custom code action by using a variable assignment action.
Figure 11. Example of test variable use in a test
Figure 12 shows a second test module (VA02) that is using a variable with the same name as the VA01 test module. The VA02 test module deletes the orders that the VA01 test module creates. In this example, the VA02 test module references data from the VA01 test rather than, for example, from a user-generated data pool.
Figure 12. Referencing a test variable from another test
Now that you have learned how to create modular test cases and link them by using test variables, the next sections show you an example of modeling your schedule by using the test modules.
When you are finished with developing and customizing your test modules, you can organize and combine various test modules together to emulate a workload scenario.
Figure 13 shows an example of various test modules being used to model a workload scenario. In this example, the Login, Create order, Logout test modules were all derived from a single test (VA01), using the split test functionality. These modules are then organized and combined with the Delete order test module (VA02) derived from another test.
As previously shown in Figure 11 and Figure 12, the Create order and Delete order test modules are linked through a test variable named orderNumber.
Figure 13. Schedule models with various SAP test modules
This workload scenario can be expanded later to include other functions by just inserting other test modules between the login and logout. Likewise, it can be made smaller by removing or disabling the test modules that are not needed for a particular scenario.
When modeling a workload using HTTP test recordings, always review the Rational Performance Tester-generated split test variables to ensure that the variables from a subsequent test module are created and assigned with values by a test module that is executed before it.
- Always maintain the general structure of a test when modeling your workload scenario using the test modules. It should include a beginning and an end. The SAP scenario, for example, should always include a login and a logout test.
- When building your workload scenarios using various test modules, ensure that they are organized in a flow that is valid and consistent with the application use.
- Be aware of dependencies between the modules. If test modules are reordered, ensure that shared test variables used are initialized and set appropriately. This is one of the reasons why documenting your test module's dependencies is necessary.
- Always examine test scripts before execution to ensure that you have created and assigned all expected variables accordingly.
Benefits of using split tests and test variables
These are some of the benefits you can achieve using the split test and test variable functionality:
- Derive test modules from a single recording, while keeping the original test intact.
- Eliminate redundant steps from tests in these three ways:
- Reduces the size of tests and maintenance time
- Improves reusability and robustness of tests
- Makes tests easier to debug
- Build bigger user scenarios by reorganizing and combining test modules
- Make it easier to quickly change user scenarios, targeting different parts of the system when issues arise.
This article focused on creating modular performance tests with Rational Performance Tester using the "split test" functionality and exchanging data between the test modules using the "test variable" functionality.
By using the split test functionality, you can derive test modules from existing recordings while keeping the original scripts unchanged. In addition, you can also split and annotate a test while recording using the Annotation toolbar.
The test variable functionality facilitates the exchange of data between the test modules. The test variable is used to reference data throughout a test and between tests. The test variable is also used to exchange data between regular tests that have not been modularized.
The test modules can be used interchangeably to construct bigger or smaller user scenarios, depending on the test objectives.
Test modularity provides several benefits:
- Helps minimize the need to rerecord your tests each time something changes
- Eliminates redundant steps from your tests
- Reduces the size of your tests, improves robustness and reusability of your tests
- Makes it easier to debug your tests and to change the focus of test scenarios
Now you're ready to proceed to Part 2: Tutorial, which is a real-world example of using Rational Performance Tester test variables. Work through it step by step to get hands-on experience.
Thank you to the following people for taking the time to review and provide comments for this article: Kent Siefkes, Sanjay Joshi, Alex Bernstein, Joel Duquene, and Jean-Philippe Delpiroux. In addition, thanks to Arif Muhammed for providing the HTTP screenshots depicted in the tutorial example Figures 10 and 11.
Learn
- Learn more on the Rational Performance Tester product page.
- Explore the Rational Performance Tester page on IBM® developerWorks® for links to technical information for software developers and testers and to related IBM software.
- Browse the Rational Performance Tester Information Center.
- Learn about other applications in the IBM Rational Software Delivery Platform, including collaboration tools for parallel development and geographically dispersed teams, plus specialized software for architecture management, asset management, change and release management, integrated requirements management, process and portfolio management, and quality management.
- Visit the Rational software area on developerWorks for technical resources and best practices for Rational Software Delivery Platform products.
- Explore Rational computer-based, Web-based, and instructor-led online courses. Hone your skills and learn more about Rational tools with these courses, which range from introductory to advanced. The courses on this catalog are available for purchase through computer-based training or Web-based training. Additionally, some "Getting Started" courses are available free of charge.
- Subscribe to the IBM developerWorks newsletter, a weekly update on the best of developerWorks tutorials, articles, downloads, community activities, webcasts and events.
Get products and technologies
- Download the trial version of IBM Rational Performance Tester.
- Download trial versions of other IBM Rational software.
- Download these IBM product evaluation versions and get your hands on application development tools and middleware products from DB2®, Lotus®, Tivoli®, and WebSphere®.
Discuss
- Join the Performance Testing forum, where you can share you questions and knowledge about IBM performance testing products.
- Check out developerWorks blogs and get involved in the developerWorks community
Karline is a staff software quality engineer with the IBM Software Group. She focuses primarily on system verification, integration, and performance testing for the Rational Performance Tester tool. She joined IBM in 2000 and has held several system administrator and senior customer support positions with the IBM Global Services group.
Table of contents
- The advantages of using modular tests
- Terminology
- Creating modular test recordings
- Constructing a test scenario from test modules
- Dynamically exchanging data between tests by using test variables
- Modeling your workload
- Benefits of using split tests and test variables
- Summary and tutorial
- Acknowledgements
- Resources
- About the author
- Comments
