This article describes a custom utility that optimizes the use of the http_find_values routine for data correlation in HTTP VU scripts and enhances the reporting of correlation errors.

Editor's note: This article is part of a library of custom VuC functions developed by performance test engineers to handle special situations encountered in scripting applications. For an overview of the library and links to other custom functions, see "A library of custom VuC functions."

The http_array_vals utility optimizes the use of the http_find_values routine for data correlation in HTTP VU scripts and enhances the reporting of correlation errors. In some circumstances the processing overhead for data correlation can account for the lion's share of CPU usage on the test platform, to the point that test results are impacted. The use of http_array_vals can significantly increase the scalability of test scripts and also reduce script development effort.

Purpose/use
When recording an HTTP VU script, IBM® Rational Robot® automatically generates code to populate "correlation variables" with values found in HTTP headers and HTML pages, and uses those variables as part of subsequent HTTP requests. For complex Web applications, the number of correlation variables generated may be such that the runtime overhead of parsing the response data to find the variables becomes excessive.

As part of a recent testing exercise, a test script generated with the default option of finding all possible correlation variables was found to cause the test platform (a 2-GHz processor) to run at 100% CPU utilization and report greatly extended response times when attempting a 10-user playback. (As a rule of thumb, the CPU utilization on the test platform should ideally remain below around 30% to 40% in order to avoid distorting the test results.)

The standard way to reduce the impact of searching for correlation variables is to analyze the script to work out which of the variables really need to be correlated and which can be left as constant values, and then to regenerate the script specifying only the required variables. This approach often eliminates 90% of the correlation variables. However, for a complex application it's not uncommon to generate hundreds of correlation variables per script. Working out which ones are really required and listing those variables by name to regenerate the script is a complex, time-consuming, and error-prone job. It's boring, too.

On a recent test for a customer this became even more of an issue. The application was complex, with hundreds of correlation variables per test script. A relatively high number of those actually needed to be correlated - probably too many to run the required virtual user load on the available test hardware. What made it worse in this case was that the application server dynamically generated the HTML form field names, which are used by Robot to locate the values for many of the correlation variables. Even minor changes of server configuration resulted in those dynamically generated names changing. This in turn meant that the test scripts broke and needed to be rerecorded. It simply wasn't viable to repeat the analysis of the test scripts and select which variables to correlate each time the scripts needed to be rerecorded.

The http_array_vals utility was developed to solve that problem in particular, but the result is applicable to any HTTP test and can significantly reduce test hardware requirements and possibly save substantial test script development effort. For scripts involving large numbers of correlation variables, the reduction in CPU usage can allow a major increase in the number of virtual users that can be emulated from a given test platform. In some cases an up to tenfold increase has been achieved.

The utility also includes extra options for the reporting of correlation errors, just like the enhanced version of the CHECK_FIND_RESULT macro. The utility consists of a Perl script to modify the generated VU script, a VU header file containing macros and routines required for error handling, and an external C routine for validating the array of results returned by http_find_values.

How the Perl script works
The runtime overhead of data correlation is attributable to the number of calls to the http_find_values routine in the generated VU script. The http_array_vals utility includes a Perl script that modifies the syntax of the VU script, reducing the number of calls to http_find_values required to locate all of the correlation variables. The script is designed such that the behavior of the modified VU script is functionally identical to the original, just quicker.

The reduction in the number of calls is achieved in two ways.

First, the scripts generated by Robot contain one call to http_find_values for each correlation variable, resulting in one search of the response data for each variable. However, the http_find_values call can search for up to 21 variables in a single pass. Testing shows that the CPU time required to search for 21 variables differs very little from the time required to search for one. Searching for multiple variables per call accounts for the bulk of the savings from this utility.

The second saving comes because in some cases the generated script contains a significant number of correlation variables that are never referenced after being parsed from the response page. Typically this seems to happen when the HTML response pages are broken up into "chunks" at recording time (that is, multiple http_nrecv calls). (This often happens with a slow network connection between the Web server and the browser, or sometimes when recoding via a proxy server.) In this situation, the generated script searches for each correlation variable seen in the first chunk immediately after the first http_nrecv. But after the second http_nrecv, it searches for all variables seen in all of the response received to date - in other words, it repeats the search for any variables seen in the first chunk as well as searching for any in the second chunk, even though the remainder of the script will typically only reference the final copy of each correlation variable. This utility eliminates any such unused variables.

The Perl script works on either a single VU script or a set of scripts generated from a single recording session. For a multiple-script session, the list of scripts must include all scripts generated from the recording session and these must be listed in the order of recording.

For each VU script, the utility finds the names of all SgenRes_nnn correlation variables, maps these variables onto arrays of up to 21 elements, and then changes all calls to http_find_values, CHECK_FIND_RESULT, and other references to use the new array elements. The script handles the persistent variables needed for split scripts automatically.

Enhanced error reporting
A generated VU script that hasn't been processed by the http_array_vals utility uses the CHECK_FIND_RESULT macro to validate that each correlation variable has been found successfully at runtime. There are two versions of this macro defined in the standard vu.h header file. By default, the macro logs an error to the virtual tester error file, sets a default value for the correlation variable, and continues script execution. Alternatively, the macro does nothing and the script aborts when the undefined correlation variable is accessed.

This utility provides additional options for dealing with correlation failures, just like the enhanced version of the CHECK_FIND_RESULT macro. A VU header file provided with the utility defines several versions of the macro CHECK_ARRAY_RESULT, which performs the equivalent check for a successful find. The different versions offer various error-reporting options. As with the enhanced version of CHECK_FIND_RESULT, the script uses tokens to control conditional compilation to determine which option is used.

The error-reporting options (conditional compilation tokens and actions on failure to find a value) are as follows:

• No conditional compilation token - Retains the default Robot behavior (that is, reports errors in the virtual user error file, sets the correlation variable to a default value, and continues execution of the script).
• ERROR_IF_FIND_VALUES_FAILS - Reports a failed test case (in red) in the test log, sets the correlation variable to a default value, and continues execution of the script.
• EXIT_IF_FIND_VALUES_FAILS - Exits the virtual user emulation with an abnormal exit status, reporting the error to the user error file.
• ABORT_IF_FIND_VALUES_FAILS - As with vu.h, does nothing, causing the script to abort when the undefined correlation variable is referenced. Refer to comments below for an advantage of using this option with Unix agents.

Probably the most useful option is the ERROR level of reporting. This highlights any correlation failures clearly through red Fail entries in the test log.

See the article on the enhanced version of the CHECK_FIND_RESULT for a discussion of possible reasons for correlation errors and how to address some common types of errors.

Validating the results array
If http_find_values fails to find a value for one of the search arguments, the corresponding element of the result array is left undefined. Any attempt to access such an undefined element results in a fatal runtime error for that virtual user. Unfortunately, the syntax of the VU language doesn't provide a way to detect which array elements are undefined.

To avoid fatal errors in this case, an external C routine is called to check for undefined elements of the result array. This requires a DLL (Windows) or shared library (Unix) to be present on the master or agent platform. The DLL is supplied with the utility, and the C source code is also supplied to enable a shared library to be built for Unix.

Creating the Unix shared library can be something of a voyage of discovery and isn't for the faint of heart, especially if relevant Unix and C development skills aren't readily available. Although the VU documentation does provide guidance, the precise instructions depend on the version of Unix and the version of the C compiler and linker, and there are an awful lot of options to choose from.

An alternative that avoids the need to build a shared library for Unix at all is to take the option to abort the virtual user session if http_find_values fails. If necessary, it's generally possible to resolve any generic scripting problems causing correlation issues by playing back the scripts at low user numbers on the master platform only, with more helpful logging enabled. Genuine, stress-related, errors during high-volume playback on the Unix agents would be highlighted by virtual user sessions aborting.

Possible runtime errors
The http_find_values routine supports the following search types: HTTP_FORM_DATA, HTTP_HREF_DATA, HTTP_COOKIE_DATA, HTTP_LOCATION_DATA, and HTTP_REDIRECT_PREFIX. Most of these search types have worked well with this script, but calls in which HTTP_REDIRECT_PREFIX search arguments are combined with other arguments seem to give rise to SIGSEGV-type errors at runtime.

The option to leave certain classes of search arguments as individual calls to http_find_values is supported in order to avoid this type of issue. By default, only HTTP_REDIRECT_PREFIX is excluded from the processing in this way, but any of the other classes of search argument can also be left as individual calls, in case similar issues are experienced with other classes as well.

Requirements
A Perl runtime environment is required in order to run the Perl script. The utility was developed using ActivePerl version 5.8.1, but it should work with any relatively recent release of Perl (version 5 or above). ActivePerl is available for free download from the ActiveState site. A Perl runtime is also installed automatically as part of various other product installs, including some of the Rational suites. Alternatively, Perl is generally available on any Unix or Linux platform.

The Perl script is designed to run against automatically generated VU scripts, before any manual changes have been made. However, any manual edits will be retained intact (although this has not been tested thoroughly, so keep the original version!). Any code manually inserted in the region of the http_find_values calls should be checked to ensure that the position still makes sense in the modified code, and any manually inserted references to the SgenRes variables should be checked to ensure that they've been updated to reference the correct new array values.

Utility code
The utility is supplied as a ZIP file that can be downloaded. It contains the following code:

• http_array_vals.pl - Perl script for modifying VU scripts
• http_array_vals.h - VU header file containing routines for error checking
• http_array_vals.dll - Windows DLL for checking returned results
• http_array_vals.c - C source to allow the equivalent Unix shared library to be built

Here's how to install the utility:

1. Place the Perl script in a directory that's on the path so that it can be used without specifying a path name (for example, create a directory called c:\perlscripts and add that to the path).

2. Place the VU header file in the following directory in the repository:

   ..\TestDatastore\DefaultTestScriptDatastore\TMS_Scripts\include

3. To use with error reporting other than ABORT_IF_FIND_VALUES_FAILS, the DLL or Unix shared library must be present in the correct location on the master and all agent machines. Place the DLL in the following directory in the repository on the master machine:

   ..\TestDatastore\DefaultTestScriptDatastore\TMS_Scripts\externC

   Place the DLL or shared library in a directory called externC, created under the "local directory" used by the test agent on each agent machine. The local directory defaults to the windows TEMP directory or the /tmp directory in Unix but can be overridden for each agent through TestManager with the menu option Suite > Edit Computer.

   Note that the directory location documented in the VU manual for the DLL or shared library appears to be wrong, at least for Windows and Linux. The externC directory, which is the location for the external C library, needs to be created as a subdirectory of the rtagent_O directory, under the agent local directory, as opposed to directly under the local directory.

4. Build the shared library for use on Unix from the C source file that's included with the utility. The precise instructions for this are platform dependent; however, the header of the C source file includes some hints for the Linux environment. Consult the VU language reference manual and appropriate operating system manuals for full details.

Command syntax
http_array_vals.pl [options] vu_script.s [vu_script.s...]

where vu_script.s is the name or names of all VU scripts generated in a single recording session. If multiple scripts are specified (that is, the split script option was used during recording), all scripts must be listed in the order in which they were created.

The original versions of the scripts are saved as <vu_script>_orig.s. Note that the _orig files won't be visible through Robot, since they won't have been registered in the repository, but will be present in the scripts directory.

The options are as follows:

• -help - Displays summary list of commands.

• -not_found <action> - Specifies what to do if value isn't found. The action can be WARN (default), ERROR, EXIT, or ABORT, and isn't case-sensitive. Each of these actions causes a token to be set to control conditional compilation of the script, as follows:

  WARN - none
  ERROR - ERROR_IF_FIND_VALUES_FAILS
EXIT - EXIT_IF_FIND_VALUES_FAILS
ABORT - ABORT_IF_FIND_VALUES_FAILS

  The value can be manually edited (in the script header) later to change the error-reporting level. There are no other differences in the generated script.

• -ignore <obj_types> - Don't include <obj_types> in the processing. <obj_types> is a pipe-delimited list of any of the following search types: HTTP_FORM_DATA, HTTP_HREF_DATA, HTTP_COOKIE_DATA, HTTP_LOCATION_DATA, HTTP_REDIRECT_PREFIX. The list must be enclosed in quotes and is not case-sensitive (for example: "XXX|YYY..."). The default is HTTP_REDIRECT_PREFIX, but anything specified here will override the default, so the string should include HTTP_REDIRECT_PREFIX as well as any other required classes.

• -keep_unused - Don't suppress unused variables. This option is provided in case there are any circumstances in which it's desirable to retain the unused correlation variables (though none have been identified to date).

Examples of use
As an example of using this utility, let's say a recording session is split into the following three scripts: Logon, New Customer, and Logoff. We would take the following steps to edit the scripts and enable reporting of correlation errors in the test log:

1. Ensure that the http_array_vals.pl Perl script is on the path and that the files http_array_vals.h and http_array_vals.dll are installed in the appropriate places in the repository.

2. Start a command prompt and change directory to the location of the VU scripts (either edit in place in the repository, or copy the scripts to a temporary location).

3. Enter the command

   http_array_vals.pl -not_found ERROR "Logon.s" "New Customer.s" "Logoff.s"

4. Ensure that the http_array_vals external C library is selected, either under the VU Compilation tab of Test Script Properties for each script, or as a project default on the VU Compilation tab of the Options window of TestManager. The Test Script Properties window is shown below.
   
   
5. Compile and run the scripts.

The following snippets of code illustrate the changes made by the script.

#include <VU.h> 
#define ERROR_IF_FIND_VALUES_FAILS 
#include <http_array_vals.h> 

… 

{ /* Persistent variables needed by other scripts */ 
persistent string SResArr_001[2]; 
persistent string SResArr_002[4]; 
persistent string SResArr_003[21]; 
PERSISTENT STRING SRESARR_004[21]; 
persistent string SResArr_005[8]; 

… 

/* PERSISTENT STRING SGENRES_001[]; REPLACED BY SRESARR_001[0] */ 
/* Introduced in NB 1S 3R Login.s */ 
/* Terminated in NB 1S 3R Logoff.s [by default] */ 
/* PERSISTENT STRING SGENRES_002[]; REPLACED BY SRESARR_001[1] */ 
/* Introduced in NB 1S 3R Login.s */ 
/* Terminated in NB 1S 3R Logoff.s [by default] */ 

… 

<After http_nrecv emulation command> 

… 

{ 
SResArr_005 = http_find_values("SunriseURL", HTTP_HREF_DATA, 1 
,"SessionId", HTTP_HREF_DATA, 1 
,"actionCategory", HTTP_HREF_DATA, 1 
,"MagicNo2016341774", HTTP_HREF_DATA, 1 
,"ProcessingMode", HTTP_HREF_DATA, 1 
,"mySystemLoginType", HTTP_HREF_DATA, 1 
,"mySystemLoginId", HTTP_HREF_DATA, 1 
,"mySystemUserId", HTTP_HREF_DATA, 1); 
check_array_vals(8, SResArr_005, HTTPChkArr); 
CHECK_ARRAY_RESULT(SResArr_005[0],HTTPChkArr[0],"SunriseURL", 
"http%3A%2F%2F10.64.118.31%3A8080%2Fte%2Fservlet%2FSRServlet") 
CHECK_ARRAY_RESULT(SResArr_005[1],HTTPChkArr[1],"SessionId", 
"0657b736a9ffb865002a97-TSTZUR1") 
CHECK_ARRAY_RESULT(SResArr_005[2],HTTPChkArr[2],"actionCategory", 
"rate") 
CHECK_ARRAY_RESULT(SResArr_005[3],HTTPChkArr[3],"MagicNo2016341774", 
"1068003998969") 
CHECK_ARRAY_RESULT(SResArr_005[4],HTTPChkArr[4],"ProcessingMode","M") 
CHECK_ARRAY_RESULT(SResArr_005[5],HTTPChkArr[5],"mySystemLoginType","I") 
CHECK_ARRAY_RESULT(SResArr_005[6],HTTPChkArr[6], 
"mySystemLoginId","TSTZUR1") 
CHECK_ARRAY_RESULT(SResArr_005[7],HTTPChkArr[7],"mySystemUserId","TEST") 
}

About the author

Comments

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• About the author
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

My developerWorks community

Dig deeper into Rational on developerWorks

Rate a product. Write a review.

Write a review for Rational products today!

Special offers