Use Rational Performance Tester to perform high-volume SAP tests

Configurations, best practices, and common pitfalls

IBM® Rational® Performance Tester Extension for SAP® Solutions extends the performance and scalability testing of Rational Performance Tester to SAP solutions. This article describes how to optimize IBM Rational Performance Tester SAP Protocol Extension and SAP R/3 servers to run high load tests. The article also provides best practices for SAP load testing using Rational Performance Tester and solutions for some of the common problems that performance testers face while running tests.

Share:

Arun N. Kutty (arun.kutty@in.ibm.com), Software Engineer, IBM

Arun N. Kutty is a level-3 support engineer for IBM Rational Performance Tester. His area of expertise isprotocol extensions including SAP, Citrix, Sockets, and TN3270. Additionally, he handles customer issues in the core product. He has 12 years of industry experience and has a Bachelor's degree in electronics and communication engineering and Post Graduate Diploma in enterprise management.



Nitish Thakur (nitish.thakur@in.ibm.com), Software Engineer, IBM

Nitish Thakur works as a senior developer for Rational Performance Tester. He is responsible for SAP, web reports, and license components in Rational Performance Tester. He has 8 years of industry experience and has a Masters degree in computer science.



10 December 2013

Also available in Chinese Russian

Introduction

Learn how to optimize the IBM® Rational® Performance Tester SAP® Protocol Extension to do performance and scalability testing on SAP solutions using Rational Performance Tester. This article describes how to:


Set configuration parameters for SAP R/3 servers

The default configuration in SAP R/3 servers does not support high user and transaction loads. SAP forums are filled with descriptions from SAP administrators who find it difficult to reach the optimum configuration advised for production systems.

Often, performance testers find bottlenecks in the application server in the pre-production environment while running load tests. These bottlenecks cause errors when you create new connections with the SAP R/3 application server at the same time you run load tests. Typical results include:

  • Connection exceptions
  • Rational Performance Tester user interface errors
  • Connection time-outs.

These results can be interpreted incorrectly as issues at the client side or with the performance testing tool. This section discusses some of the common configurations required on SAP R/3 server to avoid bottlenecks.

Maximum number of concurrent conversations (CPIC_MAX_CONV)

The SAP graphic user interface (GUI) clients connect to the SAP R/3 application server using Remote Function Call (RFC) over TCP/IP sockets using the X/Open Standard Common Programming Interface - Communications (CPIC) for communication. Usually the first bottleneck that occurs, even before the full load hits the application server, is in the CPIC layer. When a performance test hits this bottleneck, SAP connection errors start accumulating in the SAP Execution Event Console in Rational Performance Tester along with the exception Max no of conversations of 100 exceeded.

The default value of MAX_CPIC_CONV is set to 100. That value is not sufficient to handle high user and transaction loads. To solve this, increase the value according to the transaction complexity and the level of concurrency required. If in doubt, increase the value to 2,000 to start.

The MAX_CPIC_CONV parameter is set in the environment of the administrative user of the operating system where the SAP R/3 application runs. In UNIX® environments, MAX_CPIC_CONV can be set as an environment variable in the .sapenv_sappib.csh file, the .profile file, or in any other initialization shell script for the administrative user who has privileges to start SAP R/3 application server. In Windows, MAX_CPIC_CONV is set as an environment variable in the administrator's profile.

SAP gateway and profile parameters

To interact with external clients or to other application servers an SAP application server has an instance of a gateway server installed. Remote Procedure Call (RPC) requests over TCP/IP from client applications to the SAP application server are routed through an SAP gateway. The SAP gateway can be located along with the application server or can be used in a stand-alone mode for load balancing SAP R/3 servers.

The memory management and resource management parameters are important to enable high-volume processing.

The following table lists the common gateway memory and resource management parameters to be tweaked for high user load:

Table 1. SAP gateway parameters
Parameter name Significance Default value
gw/cpic_timeout Specifies the maximum wait time for a connection setup. This parameter should be increased if time-outs occur during connection establishment 20 seconds
gw/max_overflow_size This parameter specifies the size of the overflow area in bytes. CPIC requests are written to the overflow area when the limit gw/max_shm_req_per_conn for a CPIC connection is reached. 1 MB
gw/max_shm_req This parameter specifies the maximum number of CPIC requests that can be stored in the gateway in shared memory at the same time. 50
gw/max_conn Specifies the maximum number of connections that can be active at one time. 500
gw/max_sys Specifies the maximum number of clients connected at the same time. 300
rdisp/tm_max_no This parameter restricts the maximum number of users per instance. 200
rdisp/max_comm_entries This parameter specifies the maximum number of communication entries on an application server.
This parameter allows you to control the number of RFC/CPIC connections on an application server.
500

The optimal values for the parameters depend on a number of factors including the user load, the amount of transactions, and the nature of transactions. For a test with medium complexity, Rational Performance Tester test script running a load of 1000 virtual users, has completed the tests with the following values:

  • gw/cpic_timeout = 20
  • gw/max_conn = 5000
  • gw/max_sys = 5000
  • gw/max_overflow_size = 25000000
  • gw/max_shm_req = 50
  • rdisp/max_comm_entries = 5000
  • rdisp/tm_max_no = 2000

The SAP gateway and profile parameters described above are modified in the SAP GUI using the RZ10 transaction.

After connecting to the SAP R/3 server, invoke the RZ10 transaction. In the transaction screen, select the profile where the variable values are to be modified,then click on the Change button, as shown in Figure 1.

Figure 1. Modify SAP R/3 parameters on the RZ10 transaction screen
Edit profile using RZ10 SAP transaction code

Note:
The system architecture determines which profile to select. On an SAP R/3 system with a single application server, select instance profile and for a system with multiple application servers, select default profile.

The next screen in the transaction, as shown in Figure 2, displays the list of parameters that are already created in this profile. If the parameter to change is listed in this screen, double-click to reach the parameter modification screen. If the parameter is not listed, then use the Parameter button to create the parameter. In the parameter modification screen as shown in Figure 3, enter the value to set for the parameter, then click the Copy button to apply the changes to the profile. You do not have to restart the SAP instance after making these changes.

Figure 2. List of parameters and create new parameter
Maintain profile parameters for SAP R/3 server
Figure 3. Modify a SAP parameter
Modify the gw/max_overflow_size

Configure Rational Performance Tester for SAP load tests

To configure Rational Performance Tester for SAP load tests, use the recommendations in this section.

Deploy Rational Performance Tester high-volume SAP tests

Use the following guidelines to deploy Rational Performance Tester for high-volume tests:

  • Define at least one agent to drive virtual user load.
    Note:
    Do not use the workbench to generate virtual user load.
  • Install the SAP GUI software on the workbench to record the tests. Install the user interface on all the load-generating agent machines to drive the load.
  • Perform dry runs on all SAP performance tests, starting with a few virtual users. Dry runs help to debug tests to ensure that correlations and dynamic data substitutions will work for multiple users.
  • Note that each SAP virtual user spawns an SAP GUI session in the agent machine while a performance test is running. A single load-generating agent can handle approximately 25-30 virtual users on each agent for medium complexity SAP tests.
  • If a schedule has only low complexity SAP tests involved, an agent can handle approximately 50 parallel SAP sessions and therefore, 50 virtual users.
  • For a schedule that requires more virtual users, increase the number of agents linearly at the approximate rate of 30 virtual users per agent.
  • Augment the virtual user load using the SAP Java Connector Objects (JCO) library with the Rational Performance Tester batch test mode. Because SAP virtual users running JCO do not create SAP user interface sessions, they use fewer resources in the client side. It is recommended that you run a higher percentage of virtual users using the batch test. In most instances, you can run a performance test with 75% of the virtual user load run using SAP JCO and the remaining 25% of the virtual user load using SAP GUI based virtual users.

Set the schedule for an SAP performance test

Configure parameters relating to SAP performance test using the instructions in this section.

Select the statistics to be collected in a performance test run

In Rational Performance Tester, you can configure a schedule (as shown in Figure 4) to select what type of data has to be collected from the agents, how often the data should be collected (this is known as the sampling rate) and whether the data should be collected from all users or a representative sample. For a high-user load schedule, carefully consider which options are optimal for the performance requirements you need to validate.

Figure 4. Schedule the statistics collection
Set level and interval for statistics collection

For a high-user load, set the Statistics log level to Primary test actions. This setting limits the processing overheads at the agents and workbench for statistics collection. For an SAP test, the Primary Test Actions option includes all schedule actions and SAP screens.

When you run a schedule, the reports display information that is refreshed periodically during the course of the run. How often the information is refreshed depends on the Statistics sample interval setting, as shown in Figure 4. The default Statistics sample interval in Rational Performance Tester is 5 seconds. A value between 30 seconds to 120 seconds is recommended.

Select the Only store All Hosts statistics checkbox to reduce the amount of statistical data stored. This setting enables you to test a larger load over a longer period of time with significantly less memory usage. The drawback of this setting is that the statistical data from each agent that contributes to the total is not available. However, this data is not typically of interest in a large performance test run.

Determine the test log level and sample information

The test log contains elements that correspond to events occurring during a test run. The data helps a tester to debug tests during the initial phases of test development and to ensure there are no red flags that occur when high-user load tests are running.

A large virtual user load is typically run after all troubleshooting of individual and minimal user load tests are finished. It's best to set data collection at the Primary Test Actions level, as shown in Figure 5. At this level, in addition to the events pertaining to the schedule actions, the following are included:

  • Test verdict, test start, and test stop events
  • Loop iteration start and loop iteration stop events
  • Transaction start and stop events
  • SAP screen information such as SAP screen title verification points
Figure 5. Set test log level and sample information
Set schedule contents and element details

Click to see larger image

Figure 5. Set test log level and sample information

Set schedule contents and element details

To set a sampling rate, select the check box Only sample information from a subset of users then select one of the options. The number, or the percentage, that you specify is applied to each user group. If you are running user groups at remote locations, the number or percentage that you select is distributed evenly among the remote locations. For high-user loads, set a low value.

Hide the user interface GUI for all virtual users

For all SAP load tests make sure the SAP GUI is hidden for all virtual users, and for all tests in the schedule as shown in Figure 6.

Figure 6. Hide the SAP user interface during all tests GUI
Select the check box Hide GUI during execution

Enable long run mode

When tests exceed 24 hours, resource use issues can cause problems with the SAP clients. The long run mode increases the reliability of long duration tests for SAP protocol by running the tests in multiple processes.

Enable long run mode for an SAP schedule in the Performance Schedule Editor, as shown in Figure 7.

Figure 7. Enable long run mode
SAP Test option to enable long run mode

For SAP tests in the long run mode, a new SAP client process is started each time the number of test instances reaches the value provided. For example, for a load test that uses the setting as shown in Figure 7, a new process starts as soon as the number of tests run is larger than 100.


Employ best practices for SAP load tests

Before you start a load test, ensure that the test plays back reliably, with no errors, when run as a single test.

Improve the success rate of long duration tests

Use the following guidelines to improve the running of long duration tests:

  • Use agent computers with at least 2GB of RAM and 10GB of free disk space. The agents should run the same version of the IBM Rational Performance Tester and Rational Performance Tester Agent. Disable antivirus software, screen savers, and automatic updaters in the agent computer. Avoid using the agent computers for other purposes while a test is running.
  • Keep individual tests short by avoiding loops of more than 10 iterations within each test, and achieve the desired long run duration by looping within the schedule. Do not exceed 20 or 30 virtual testers per agent computer with a think time above several seconds.
  • Use tests with a minimal number of verification points.

Configure SAP client settings for load tests

Use the following recommendations to configure SAP client settings:

  • Do not use the SAP Signature Theme as shown in Figure 8.
  • Disable all animations by unchecking the check box Activate animated focus in the SAP GUI Options page, as shown in Figure 8.
Figure 8. SAP Signature Theme and animation setting
SAP GUI configuration preferences
  • During playback of SAP tests, avoid mouse clicks. Mouse clicks could be interpreted by a hidden SAP GUI window and thus cause unexpected behavior.
  • Design the test such that the Screen Throughput counter remains low. The recommended screen throughput is approximately 1 screen per second per agent.

Set the SAP batch input recording and playback

When running high-volume tests augment the virtual users driven through SAP client with virtual users by running an SAP batch input test. Batch input tests access the SAP R/3 server at a low level using the SAP JCO library. Using the JCO library, Rational Performance Tester bypasses the SAP GUI interface and is able to achieve loads with far less memory and CPU resource use. You can add normal SAP tests and SAP batch tests in the same test schedule. The main purpose of batch input tests in such schedules is to add extra load on the SAP R/3 server that is already loaded with normal tests. SAP batch input tests cannot contain verification points or SAP GUI elements. Therefore, for performance data, you have to rely on normal SAP performance test results.

You can record a batch test in the SAP GUI if you select the built-in recorder to capture the interactions of the SAP GUI with the SAP R/3 server. This is shown in Figure 9. The recorder has a provision to write the captured data into a text file.

Figure 9. Access built-in recorder in SAP GUI for batch input tests
Navigating to the recorder tool in SAP GUI

After recording the scenario to a batch input file in the SAP GUI, a new test should be created in Rational Performance Tester by importing the batch input file. A single batch input test can contain multiple transactions, as shown in Figure 10.

Figure 10. Create new SAP test from batch input recording text file
Test creation wizard where batch file is consumed

The JCO connection properties can be configured in Rational Performance Tester at the test level to control the parallel connections through the JCO library, as shown in Figure 11. The JCO connections library provides an option of re-using connections from a pool. Connection pooling in a load test helps to limit the overhead cost of establishing connections. It also helps to limit the number of open connections in the SAP R/3 server and client sides.

Figure 11. SAP JCO connection properties in Rational Performance Tester
SAP JCO Connection configurable parameters

Click to see larger image

Figure 11. SAP JCO connection properties in Rational Performance Tester

SAP JCO Connection configurable parameters

Avoid common pitfalls in SAP load tests

This section describes some of the common problems with SAP load tests and how to avoid them.

Going for too much too soon

Troubleshoot the recorded tests completely with a single virtual user to ensure that the workflow and data correlation is accurate. After that, play back the recorded tests in a minimal, multi-user scenario (fewer than 5 virtual users) to ensure that the user-specific dynamic data gets substituted and the workflow is successful for all users. After the test is proven for the minimal load, gradually increase the schedule load to reach the required levels. The availability of incremental measurements as the load is increased helps to identify where load bottlenecks are hit and gives useful pointers that help locate where the bottlenecks are, whether they are at the Rational Performance Tester configuration or related to the SAP R/3 server.

Error: “Invalid Control ID” while playing back SAP tests

It is common to observe the error Invalid Control ID while playing back SAP tests. This error is generated by the SAP GUI when a method is invoked on one of the GUI elements that is not present. This situation can happen if the element has not been rendered to the SAP GUI yet. This issue can be avoided by increasing the think time associated with the SAP screen or by inserting adequate delays in the tests to allow for proper rendering of screens. Ensure an adequate ramp-up rate so that all the users do not login and start transactions with the SAP R/3 servers at the same time.

This error can also occur if the user selected an input record from a modal dialog window by clicking on any column other than the first one while recording. Figure 12 shows the correct way to select a record. Figure 13 shows the incorrect way.

Figure 12. Select a record using the first column (correct)
Correct way to pick up items on a GuiComboBox
Figure 13. Select a record by clicking on other columns (incorrect)
Incorrect way to pick up items on a GuiComboBox

Loading unnecessary modal dialogs in a performance test

In the SAP GUI it is common for input fields to use the SAP GUI element GuiComboBox (F4) to select a value from a predefined value set. You can click on the Help button along with the GuiComboBox to open a modal dialog window with the predefined values. Select an input value from that the predefined values. Avoid using the option shown in Figure 15. Instead enter the required value directly in the text box, as shown in Figure 14. This approach helps avoid resource-hungry, but unnecessary GUI pop-ups from being opened.

Figure 14. Direct input of required value (recommended)
Recommended way to enter values in a GUIComboBox
Figure 15. Select a value from the modal dialog (not recommended)
Using F1 help to enter values which is discouraged

Summary

This article explains how to optimize SAP R/3 servers and Rational Performance Tester to run high user load tests. In addition, this article covers some of the recommended best practices and common usage pitfalls that happen when using Rational Performance Tester to performance test SAP R/3 servers.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=955902
ArticleTitle=Use Rational Performance Tester to perform high-volume SAP tests
publish-date=12102013