Test data is an important part of most software testing, especially performance testing, which often requires large amounts of test data. IBM® Rational® Performance Tester leverages the Eclipse Test and Performance Tools Platform (TPTP) datapool structures for handling of test data. Rational Performance Tester includes many features that make using these datapools very easy and flexible.
Once you start using datapools with more than 10,000 rows of data, however, there can be long delays in starting up a test. For performance testing of large systems, it is not inconceivable to have test data requirements of 1 million records or more. In these situations, the TPTP datapools may not be the most effective solution.
This two-part series of articles will show you how to create Rational Performance Tester tests that use files instead of datapools for storing and handling test data. This technique can be used to address the need for using large sets of test data in performance testing with Rational Performance Tester.
Part 1 will begin with creating a simple test and a standard datapool to provide variable test data. In Part 2, you will modify the same test to use a test data file in place of the datapool.
Note: This article was developed using Performance Tester version 7.0.0. This should work with future versions although specific procedures could change. You can also use this article with RPT version 6.1.2, although some screen shots and procedures may not be exactly the same. Performance Tester version 6.1.1 and earlier will not work due to changes in the Performance Tester custom code API.
Handling large volumes of test data with Rational Performance Tester
Start by recording a simple test with one variable data item, a search string. The first test you develop will use a datapool to hold multiple search items to use when running the test with a load of users.
Setting up the article files
There are several files included with this article, described as follows:
- TestData.csv: A file containing 50 random names used for search strings.
- GetTestData.java: A completed custom code module from which you can copy and paste, used in Part 2.
- SetupTestDataArea.java: A completed custom code module from which you can copy and paste, used in Part 2.
- Download and unzip these files into your C:\temp\ directory. You could put these files into another location, but you will have to update this in several steps, so it’s probably easier to use the default.
Create a test using a datapool for test data
- Create a Rational Performance Tester project.
- Launch Rational Performance Tester and make sure you are in the Test perspective, which is the default Eclipse perspective for Rational Performance Tester.
- From the menu, select File > New > Performance Test Project, as shown in Figure 1. You may enter any Name and Contents directory that you want.
Figure 1. Creating a new test project
- In the Create New Test From Recording window shown in Figure 2, click Cancel. You will now see the new project listed in the Test Navigator pane on the left.
Figure 2. Cancel create new test from recording
- Create a new folder by right-clicking the project name in the Test Navigator pane and selecting New > Folder. Repeat this to create the following folders, as shown in Figure 3:
Figure 3. Additional project folders
Note: You may also create any other additional folders you want to help organize the test artifacts.
- Review the system under test and plan the test steps.
As mentioned earlier, you will use the developerWorks Web site as the system under test (SUT), and perform a simple search for the test. Before recording a test, you should always walk through the test steps manually at least once to ensure that the environment is properly configured and ready for test recording. The test steps you will perform are listed below. Perform only these steps.
- Launch a browser and go to the Rational brand section of developerWorks at: www.ibm.com\developerworks\rational\ .
- Click the Performance Tester link located under Software Quality near the center of the page, as shown in Figure 4.
Figure 4. System under test -- developerWorks
- In the Search field near the top of the page, enter your name (or whatever search term you want) and click Search.
- After the search results have been returned, close the browser.
- Create a datapool from the included TestData.csv file.
You don’t want to use the exact same search string for every test, so you will use a set of test data (in this case random names) for the search string. You can generate your own data, or use the files included with this article. Comma-separated value (CSV) files are a common format for data in files, and most databases can save data into this format. You will import these into datapools to utilize some of the ease-of-use features of Rational Performance Tester.
- Create a new datapool by right-clicking the testdata folder in the Test Navigator pane and selecting New > Datapool.
- In the New Datapool window shown in Figure 5, make sure that you have selected the testdata folder. Enter the name
TestData, then click Next.
Figure 5. New datapool
- In the next New Datapool window, enter any description you want then click Next.
- In the final New Datapool window, shown in Figure 6, clear the First row contains variable names and suggested types checkbox.
- Enter or browse to the location of the article test data file (as described earlier, the default location is C:\temp\TestData.csv). If you saved the file to a different location, then enter that location here.
- Click Finish to create the datapool. It may take a short time for the import to complete.
Figure 6. Importing data for a new datapool
- You will now see the Datapool open in the center workspace showing the rows of data. The new datapool contains only one variable (column), and you want to give this a more meaningful name. Click the variable name (column header) labeled “Variable1::String” once to open the Edit Variable window.
- Change the name to
SearchStringas shown in Figure 7, and click OK.
- Save the change to the datapool and close it.
Figure 7. Renaming a datapool variable
You now have your test data ready in the form of a datapool.
- Record a test.
- Record a new test by clicking the Create New Test From Recording toolbar button.
- In the Create New Test window, make sure that Create Test From New Recording and HTTP Recording are selected, as shown in Figure 8. Click Next.
Figure 8. New HTTP recorder test
- In the next Create New Test window, make sure you have selected the tests folder. Enter the name
LargeTestData_datapool, then click Finish.
- Repeat the test steps exactly as described in section III to launch a browser and search, recording against the developerWorks website. Enter any name or value for the search string.
- If you are using Microsoft® Internet Explorer, then the HTTP recorder will automatically stop when you close the browser. If you are using Firefox or another browser, then you will have to click the Stop Recording button (located near the bottom of the Rational Performance Tester window) in the Recorder Control tab, as shown in Figure 9.
- The test will take a moment to generate and will then be open for editing.
Figure 9. Recorder control
- Replace recorded search value with datapool value.
The recorded test will by default use the recorded test data for the search string until you edit the test to select a datapool to use.
- In the newly recorded test, select the last page in the Test Contents pane, which should be titled IBM developerWorks > Search results. In the Test Data section of the test editor, shown in Figure 10, you will notice the search string you entered as the Value for the item query.
Figure 10. Identifying parameters for test data
- Select the row with the Name query, then click the Substitute From button.
- From the sub-menu that opens when you click this button, select Datapool Variable.
- In the Select datapool column window, click the Add Datapool button.
- In the Import Datapool window, select the datapool TestData, then click Select, leaving the default datapool options.
- In the Select datapool column window, select SearchString, then click Use Column, as shown in Figure 11.
Figure 11. Datapool column substitution
- In the Test Data section of the test editor, you should now see the item query is substituted with the
SearchStringvariable of the TestData datapool, as shown in Figure 12.
Figure 12. Substituting parameters with datapool test data
- Create a schedule to run the test.
- Create a new schedule by right-clicking the schedules folder in the Test Navigator pane and selecting New > Performance Schedule.
- In the Performance Schedule window, make sure you have selected the schedules folder, and enter the name
LargeTestData_datapool. Click Finish.
- In the Schedule Element Details to the right, on the General tab, select the Add a delay between starting each user checkbox and enter a delay of
500milliseconds, as shown in Figure 13.
Figure 13. Performance schedule
- Click the Test Log tab under Schedule Element Details to set the log level. Set the Log Level for the And also show all other types to Action Details, as shown in Figure 14. You can ignore the recommendation warning, since this test will not log excessive amounts of data.
Figure 14. Performance schedule -- Log Levels
- Add a loop by right-clicking User Group 1 in the Schedule Content section and selecting Add > Loop.
- With the loop still selected, in the Schedule Element Details to the right, enter
3for the Number of iterations. Select the Control the rate of iterations checkbox and enter an iteration rate of
10per minute, as shown in Figure 15.
Figure 15. Performance schedule loop iteration rate
- Add a test by right-clicking the loop in the Schedule Content section and selecting Add > Test.
- From the Select Performance Tests window, select LargeTestData_datapool and click OK.
- Run the test and verify the test data that is used.
- Run the schedule by clicking the Run toolbar button.
Note: Do not select Run As > Performance Schedule, because this will save the results in the root folder of the project, and you want to save test results in the result folder.
- In the Run configurations window, select Performance Schedule on the left, then click the New button to create a new configuration, as shown in Figure 16.
Figure 16. Create a new run configuration
- Enter a name for the new configuration. Click the Schedule tab and select the schedule LargeTestData_datapool.
- Click the Test Logs tab, clear the Use defaults checkbox, and select the results folder for the Location.
- Click Apply to save this run configuration, as shown in Figure 17.
Figure 17. Saving performance tests to a results folder
- Now that you have configured the schedule to save results in the intended folder, run the schedule by clicking Run. The test will immediately begin execution. The default Overall Performance Report will indicate when the test has completed.
Important! The IBM developerWorks web site is not to be used as a load testing site. Please do not run any more users than specified in this article. Also, do not run repeated tests beyond what is necessary to complete these steps.
- Open the execution history by right-clicking LargeTestData_datapool [date+time] in the Performance Test Runs view and selecting Display Test Log, as shown in Figure 18.
Figure 18. Display execution history
- Show the test run details by selecting the Events tab at the bottom of the execution history.
- Expand the events until you see the page titled IBM developerWorks > Search results. Expand this and select the first request, as shown in Figure 19. Here you will see the URL encoded query value that was actually used in the test run.
Figure 19. Visually verifying test data used in a test run
- Repeat this for several different instances of this page and verify that the query value is different, and that it matches values from the datapool.
- Another way you can verify the test data that was actually used is to select the page request (you may have to select the second request if the first returns a “The document has moved” response).
- Next, click the Protocol Data tab and select Browser near the bottom of the workspace, as shown in Figure 20. This will render the response received during the test run in a browser view. This technique takes longer, especially for larger test results.
Figure 20. Visually verifying test data used with the browser view
- Run the schedule by clicking the Run toolbar button.
- Close the test LargeTestData_datapool.
Test data with datapools summary
Using datapools in Rational Performance Tester is the easiest way to manage test data and substitute values in your tests. Once test data has been imported through CSV files, you can simply select test parameters and pick the datapool columns to use. However, there can be limitations with very large datapools, even though we don’t attempt to push these limits in this article.
Rational Performance Tester datapools are an easy way to link test data with your tests, which can help reduce the amount of time required to develop tests for performance and load testing. However, they can become inefficient with very large sizes (in general over 10,000 records). The good news is that Rational Performance Tester does not limit you to just datapools for test data; you can also use test data files directly through custom coding to handle whatever volume of data is required for your testing. This technique is explained in Part 2 of this series.
|50 random names used for search strings||TestData.zip||1 KB|
|A completed custom code module||GetTestData.zip||2 KB|
|A completed custom code module||SetupTestDataArea.zip||1 KB|
Get products and technologies
- Get a trial download of Rational Performance Tester.
- Here is the IBM Performance Optimization Toolkit.