Faster processing of numerous test data parameters for performance tests

Use OS memory with Rational Performance Tester to process parameters at run time

With the increase in use of the IBM Rational Performance Tester tool for performance testing of diversified protocols and large-volume load simulations, one of the major challenges is in setting parameters for the test data at run time. This is primarily because of the volume of test data involved, both in terms of the number of types of test data and the amount of test data involved for each type. The amount of data often spans to millions of rows. Reading that data at run time involves very intensive I/O activity. This article describes ways to optimize test data parameters to for lower use of operating system resources and to drive more loads through the testing software. This leads to successful performance tests even with large volumes of test data and request generation activities.

Bharath Raj Keshavamurthy (bharathrajbk@in.ibm.com), Enterprise Solutions Performance Analyst, IBM

author photoBharath Raj works at the IBM India Software Labs with the HiPODS team (High Performance On Demand Solutions) as an enterprise solutions performance analyst. He started his career with IBM as a J2EE application developer using Web 2.0 technologies. He continued into the role of performance of middleware, Java, and databases used for enterprise solutions in telecom, retail, and other applications. With an intensive background and experience on Performance Tuning, Capacity planning and Sizing, Testing and Performance Engineering methodologies, Bharath brings a holistic end-to-end Topology from anything and everything in performance of an Enterprise solution, as well as design and review of the solutions based on performance. He continues to use cutting edge technologies in enterprise web applications to achieve an optimized, faster, and a much more reliable solution.



12 July 2011

Also available in Chinese

Using system memory for test data parameters at run time

In case of very large data pools, where the number of records for each type of test data is 100,000 or more with a specific pattern, it is better to use custom code, with logic based on the pattern, to parameterize runtime data in the script, rather than creating a data pool. The data pool is a file that contains the test data. It needs to be transferred from the workbench machine to the agents and read at run time by the agents to the generated load, using the test data in the file. These read operations involve heavy I/O activity during generation of high throughput requests, leading to significant CPU utilization on the operating system.

In this event, you cannot drive the desired amount of high throughput load by using the performance test software, because it "chokes" the operating system resources and becomes a bottleneck in running performance tests. To completely alleviate this I/O activity, you can use custom codes in the test scripts to generate data though quick computations (based on logic), where the data is stored and read from physical memory. Physical memory read/write operations are far faster than operations on disk, so this improves the overall performance of load generation requests.

Example: Generate data pools from system memory

Consider a simple example where the username for a web application needs to have parameters set. If the number of users is 100,000 or more, then it is best to generate a user ID on each iteration by executing logic with a common pattern. Follow the steps described here for implementation –

Identify a common pattern

Let's say that all user names start with the word username, followed by a number. For example: username1, username2, username3.

  1. Create a new test variable and initialize its value to Username1, as Figure 1 shows:
    1. Open the test script.
    2. Click on the Test Variables section and expand it.
    3. Notice the test variables listed below Test Variables section. If there are none, click Add when Test Variables is selected.
    4. Click Variable Declaration so that you can add a new variable.
Figure 1. Creation of new test variable in Rational Performance Tester
Test Contents view, Variable Declaration selected
Figure 2. Note that you can initialize a test variable to a text or data pool value
Initialize username variable only if required

Create custom code

  1. Create a custom code in the page that loads after logging into the application, and paste the code from Listing 1 in the exec method of the custom code class.
Listing 1. Exec method in the custom code
public String exec(ITestExecutionServices tes, String[] args) {
	
  /**
  * This code snippet obtains the previously used username for logging into the 
    application and increments it by 1 to get the next username in the test data 
  */
	
  String number = "";
	
  /**
  * The assumption here is that all usernames begin with the word - "Username" and 
    has '8' characters in it.
  * Variable "number" is the value 'n' for 'nth' user or nth row of the data pool or can 
    be also thought of as the 'nth' iteration of the test
  */
	
  for (int i=0;i<((args[0].length())-8);i++)
  {
    number+=Character.toString(args[0].charAt(8+i));
  }
  int num = Integer.parseInt(number);
  num++;
	
  return "Username"+Integer.toString(num);
	
  /**
  * This returns the username appended with the next value of 'n' in the 'nth' iteration
  */
}
  1. In the Custom Code – Select Arguments window, under Test Variables, check the box by Username_Variable to add it as the argument to this custom code, as Figure 3 shows.
Figure 3. Addition of the Username variable as an argument to custom code
Username_Variable check box checked
  1. Select the Username variable in the list of data pool variables on the page that loads after logging in, and substitute the same from the custom code as Figure 4 shows:
    1. Select the page that shows the Username variable as test data. Tip: Another way to find out whether the Username variable is a test data or not is by looking at its color code. The color code adopted in Rational Performance Tester software for variables that can be parameterized at run time is green.
    2. Select the username variable under test data table.
    3. Click the Substitute button and choose Custom code in the list of options as shown.
Figure 4. Substitution of the username variable with output of the custom code
Substitute Username variable with code output
  1. Assign the variable to the custom code by inserting Variable assignment (see Figure 5). A test variable must be reused in the test script. This means that its value needs to be updated with the new value whenever logic is executed. To update the value, follow these steps:
    1. Select the custom code that will change the test variable data when it is executed.
    2. Click Insert, select Variable Assignment.
Figure 5. Username variable assignment to the custom code
Variable Assignment selected from drop-down menu
  1. Select the Username_Variable from the list of options, and set the Username_Variable to the output of the custom code by clicking the DataSource Value radio button and selecting the custom code in the options provided.

When this test is launched, the Username variable in the test will be substituted from the value returned by the execution of the custom code at run time.

Important:
Inserting the variable assignment is crucial in ensuring that the updated value of the global variable in the custom code takes effect after exiting the custom code, when the assignment of the global variable happens.

By doing this, large data pool file sizes and corresponding I/O activity can be avoided. This also improves the performance Rational Performance Tester agent in generating the test load, because the data pool values are served faster from memory than from disk, thereby alleviating disk reading latencies.

Note:
For the "Visible in" field, if the test script is looped at a schedule level, select the option for All tests for this user, and if the loop is within the test script, select the This test only option, as shown in Figure 6.

Figure 6. Different levels of visibility of the test Username variable
Different levels of visibility, Username variable

In scenarios where there is a good likelihood of some of the test data being repeated over and over again for many iterations, it is better to store the data pool entries (row entries) in a variable string array, and read the values from this array and substitute at run time.

This approach prevents performance degradation from repeated data read I/O on the data pool files due to disk latencies. Instead, it reads the data from physical memory, which is much faster to access and leads to better performance of the Rational Performance Tester agent.


Other good practices to adopt for large volume data pools

  • When there is a test scenario that requires parameters for many values (usually > 30-40 values), then it is better not to use a monolithic large data pool file. Instead, use many small data pool files in which the entire test data is divided and organized by the variables, with parameters set at run time.

    This helps in increased parallel operation in I/O while reading test data from the data pools hence leading to better performance of Rational Performance Tester agents.
  • Although it is good to use custom codes for data parameters, this option should be carefully chosen wherever applicable. This is because custom codes create significant overhead (primarily on process utilization and memory) during testing, depending on the complexity of the logic executed in the code and the number of such custom code segments.
  • For scripts that are already heavy in terms of many custom codes, correlations, and so forth, avoid Content verification points insofar as possible, because using them parses the entire response from the server to validate certain criteria. Parsing of the entire response for a limited-length string consumes significant CPU resources to run the process. Use Response Code or Title verification points wherever possible, instead.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=696520
ArticleTitle=Faster processing of numerous test data parameters for performance tests
publish-date=07122011