Use Rational Performance Tester to perform high-volume SAP tests
Configurations, best practices, and common pitfalls
Learn how to optimize the IBM® Rational® Performance Tester SAP® Protocol Extension to do performance and scalability testing on SAP solutions using Rational Performance Tester. This article describes how to:
- Set configuration parameters for SAP R/3 servers
- Configure Rational Performance Tester for SAP load tests
- Employ best practices for SAP load tests
- Avoid common pitfalls in SAP load tests
Set configuration parameters for SAP R/3 servers
The default configuration in SAP R/3 servers does not support high user and transaction loads. SAP forums are filled with descriptions from SAP administrators who find it difficult to reach the optimum configuration advised for production systems.
Often, performance testers find bottlenecks in the application server in the pre-production environment while running load tests. These bottlenecks cause errors when you create new connections with the SAP R/3 application server at the same time you run load tests. Typical results include:
- Connection exceptions
- Rational Performance Tester user interface errors
- Connection time-outs.
These results can be interpreted incorrectly as issues at the client side or with the performance testing tool. This section discusses some of the common configurations required on SAP R/3 server to avoid bottlenecks.
Maximum number of concurrent conversations (CPIC_MAX_CONV)
The SAP graphic user interface (GUI) clients connect to the SAP R/3 application server using Remote Function Call (RFC) over TCP/IP sockets using the X/Open Standard Common Programming Interface - Communications (CPIC) for communication. Usually the first bottleneck that occurs, even before the full load hits the application server, is in the CPIC layer. When a performance test hits this bottleneck, SAP connection errors start accumulating in the SAP Execution Event Console in Rational Performance Tester along with the exception
Max no of conversations of 100 exceeded.
The default value of
MAX_CPIC_CONV is set to 100. That value is not sufficient to handle high user and transaction loads. To solve this, increase the value according to the transaction complexity and the level of concurrency required. If in doubt, increase the value to 2,000 to start.
MAX_CPIC_CONV parameter is set in the environment of the administrative user of the operating system where the SAP R/3 application runs. In UNIX® environments,
MAX_CPIC_CONV can be set as an environment variable in the .sapenv_sappib.csh file, the .profile file, or in any other initialization shell script for the administrative user who has privileges to start SAP R/3 application server. In Windows,
MAX_CPIC_CONV is set as an environment variable in the administrator's profile.
SAP gateway and profile parameters
To interact with external clients or to other application servers an SAP application server has an instance of a gateway server installed. Remote Procedure Call (RPC) requests over TCP/IP from client applications to the SAP application server are routed through an SAP gateway. The SAP gateway can be located along with the application server or can be used in a stand-alone mode for load balancing SAP R/3 servers.
The memory management and resource management parameters are important to enable high-volume processing.
The following table lists the common gateway memory and resource management parameters to be tweaked for high user load:
Table 1. SAP gateway parameters
|Parameter name||Significance||Default value|
|Specifies the maximum wait time for a connection setup. This parameter should be increased if time-outs occur during connection establishment||20 seconds|
| This parameter specifies the size of the overflow area in bytes. CPIC requests are written to the overflow area when the limit ||1 MB|
|This parameter specifies the maximum number of CPIC requests that can be stored in the gateway in shared memory at the same time.||50|
|Specifies the maximum number of connections that can be active at one time.||500|
|Specifies the maximum number of clients connected at the same time.||300|
|This parameter restricts the maximum number of users per instance.||200|
| This parameter specifies the maximum number of communication entries on an application server.|
This parameter allows you to control the number of RFC/CPIC connections on an application server.
The optimal values for the parameters depend on a number of factors including the user load, the amount of transactions, and the nature of transactions. For a test with medium complexity, Rational Performance Tester test script running a load of 1000 virtual users, has completed the tests with the following values:
gw/cpic_timeout = 20
gw/max_conn = 5000
gw/max_sys = 5000
gw/max_overflow_size = 25000000
gw/max_shm_req = 50
rdisp/max_comm_entries = 5000
rdisp/tm_max_no = 2000
The SAP gateway and profile parameters described above are modified in the SAP GUI using the RZ10 transaction.
After connecting to the SAP R/3 server, invoke the RZ10 transaction. In the transaction screen, select the profile where the variable values are to be modified,then click on the Change button, as shown in Figure 1.
Figure 1. Modify SAP R/3 parameters on the RZ10 transaction screen
The system architecture determines which profile to select. On an SAP R/3 system with a single application server, select instance profile and for a system with multiple application servers, select default profile.
The next screen in the transaction, as shown in Figure 2, displays the list of parameters that are already created in this profile. If the parameter to change is listed in this screen, double-click to reach the parameter modification screen. If the parameter is not listed, then use the Parameter button to create the parameter. In the parameter modification screen as shown in Figure 3, enter the value to set for the parameter, then click the Copy button to apply the changes to the profile. You do not have to restart the SAP instance after making these changes.
Figure 2. List of parameters and create new parameter
Figure 3. Modify a SAP parameter
Configure Rational Performance Tester for SAP load tests
To configure Rational Performance Tester for SAP load tests, use the recommendations in this section.
Deploy Rational Performance Tester high-volume SAP tests
Use the following guidelines to deploy Rational Performance Tester for high-volume tests:
- Define at least one agent to drive virtual user load.
Do not use the workbench to generate virtual user load.
- Install the SAP GUI software on the workbench to record the tests. Install the user interface on all the load-generating agent machines to drive the load.
- Perform dry runs on all SAP performance tests, starting with a few virtual users. Dry runs help to debug tests to ensure that correlations and dynamic data substitutions will work for multiple users.
- Note that each SAP virtual user spawns an SAP GUI session in the agent machine while a performance test is running. A single load-generating agent can handle approximately 25-30 virtual users on each agent for medium complexity SAP tests.
- If a schedule has only low complexity SAP tests involved, an agent can handle approximately 50 parallel SAP sessions and therefore, 50 virtual users.
- For a schedule that requires more virtual users, increase the number of agents linearly at the approximate rate of 30 virtual users per agent.
- Augment the virtual user load using the SAP Java Connector Objects (JCO) library with the Rational Performance Tester batch test mode. Because SAP virtual users running JCO do not create SAP user interface sessions, they use fewer resources in the client side. It is recommended that you run a higher percentage of virtual users using the batch test. In most instances, you can run a performance test with 75% of the virtual user load run using SAP JCO and the remaining 25% of the virtual user load using SAP GUI based virtual users.
Set the schedule for an SAP performance test
Configure parameters relating to SAP performance test using the instructions in this section.
Select the statistics to be collected in a performance test run
In Rational Performance Tester, you can configure a schedule (as shown in Figure 4) to select what type of data has to be collected from the agents, how often the data should be collected (this is known as the sampling rate) and whether the data should be collected from all users or a representative sample. For a high-user load schedule, carefully consider which options are optimal for the performance requirements you need to validate.
Figure 4. Schedule the statistics collection
For a high-user load, set the Statistics log level to Primary test actions. This setting limits the processing overheads at the agents and workbench for statistics collection. For an SAP test, the Primary Test Actions option includes all schedule actions and SAP screens.
When you run a schedule, the reports display information that is refreshed periodically during the course of the run. How often the information is refreshed depends on the Statistics sample interval setting, as shown in Figure 4. The default Statistics sample interval in Rational Performance Tester is 5 seconds. A value between 30 seconds to 120 seconds is recommended.
Select the Only store All Hosts statistics checkbox to reduce the amount of statistical data stored. This setting enables you to test a larger load over a longer period of time with significantly less memory usage. The drawback of this setting is that the statistical data from each agent that contributes to the total is not available. However, this data is not typically of interest in a large performance test run.
Determine the test log level and sample information
The test log contains elements that correspond to events occurring during a test run. The data helps a tester to debug tests during the initial phases of test development and to ensure there are no red flags that occur when high-user load tests are running.
A large virtual user load is typically run after all troubleshooting of individual and minimal user load tests are finished. It's best to set data collection at the Primary Test Actions level, as shown in Figure 5. At this level, in addition to the events pertaining to the schedule actions, the following are included:
- Test verdict, test start, and test stop events
- Loop iteration start and loop iteration stop events
- Transaction start and stop events
- SAP screen information such as SAP screen title verification points
Figure 5. Set test log level and sample information
To set a sampling rate, select the check box Only sample information from a subset of users then select one of the options. The number, or the percentage, that you specify is applied to each user group. If you are running user groups at remote locations, the number or percentage that you select is distributed evenly among the remote locations. For high-user loads, set a low value.
Hide the user interface GUI for all virtual users
For all SAP load tests make sure the SAP GUI is hidden for all virtual users, and for all tests in the schedule as shown in Figure 6.
Figure 6. Hide the SAP user interface during all tests GUI
Enable long run mode
When tests exceed 24 hours, resource use issues can cause problems with the SAP clients. The long run mode increases the reliability of long duration tests for SAP protocol by running the tests in multiple processes.
Enable long run mode for an SAP schedule in the Performance Schedule Editor, as shown in Figure 7.
Figure 7. Enable long run mode
For SAP tests in the long run mode, a new SAP client process is started each time the number of test instances reaches the value provided. For example, for a load test that uses the setting as shown in Figure 7, a new process starts as soon as the number of tests run is larger than 100.
Employ best practices for SAP load tests
Before you start a load test, ensure that the test plays back reliably, with no errors, when run as a single test.
Improve the success rate of long duration tests
Use the following guidelines to improve the running of long duration tests:
- Use agent computers with at least 2GB of RAM and 10GB of free disk space. The agents should run the same version of the IBM Rational Performance Tester and Rational Performance Tester Agent. Disable antivirus software, screen savers, and automatic updaters in the agent computer. Avoid using the agent computers for other purposes while a test is running.
- Keep individual tests short by avoiding loops of more than 10 iterations within each test, and achieve the desired long run duration by looping within the schedule. Do not exceed 20 or 30 virtual testers per agent computer with a think time above several seconds.
- Use tests with a minimal number of verification points.
Configure SAP client settings for load tests
Use the following recommendations to configure SAP client settings:
- Do not use the SAP Signature Theme as shown in Figure 8.
- Disable all animations by unchecking the check box Activate animated focus in the SAP GUI Options page, as shown in Figure 8.
Figure 8. SAP Signature Theme and animation setting
- During playback of SAP tests, avoid mouse clicks. Mouse clicks could be interpreted by a hidden SAP GUI window and thus cause unexpected behavior.
- Design the test such that the Screen Throughput counter remains low. The recommended screen throughput is approximately 1 screen per second per agent.
Set the SAP batch input recording and playback
When running high-volume tests augment the virtual users driven through SAP client with virtual users by running an SAP batch input test. Batch input tests access the SAP R/3 server at a low level using the SAP JCO library. Using the JCO library, Rational Performance Tester bypasses the SAP GUI interface and is able to achieve loads with far less memory and CPU resource use. You can add normal SAP tests and SAP batch tests in the same test schedule. The main purpose of batch input tests in such schedules is to add extra load on the SAP R/3 server that is already loaded with normal tests. SAP batch input tests cannot contain verification points or SAP GUI elements. Therefore, for performance data, you have to rely on normal SAP performance test results.
You can record a batch test in the SAP GUI if you select the built-in recorder to capture the interactions of the SAP GUI with the SAP R/3 server. This is shown in Figure 9. The recorder has a provision to write the captured data into a text file.
Figure 9. Access built-in recorder in SAP GUI for batch input tests
After recording the scenario to a batch input file in the SAP GUI, a new test should be created in Rational Performance Tester by importing the batch input file. A single batch input test can contain multiple transactions, as shown in Figure 10.
Figure 10. Create new SAP test from batch input recording text file
The JCO connection properties can be configured in Rational Performance Tester at the test level to control the parallel connections through the JCO library, as shown in Figure 11. The JCO connections library provides an option of re-using connections from a pool. Connection pooling in a load test helps to limit the overhead cost of establishing connections. It also helps to limit the number of open connections in the SAP R/3 server and client sides.
Figure 11. SAP JCO connection properties in Rational Performance Tester
Avoid common pitfalls in SAP load tests
This section describes some of the common problems with SAP load tests and how to avoid them.
Going for too much too soon
Troubleshoot the recorded tests completely with a single virtual user to ensure that the workflow and data correlation is accurate. After that, play back the recorded tests in a minimal, multi-user scenario (fewer than 5 virtual users) to ensure that the user-specific dynamic data gets substituted and the workflow is successful for all users. After the test is proven for the minimal load, gradually increase the schedule load to reach the required levels. The availability of incremental measurements as the load is increased helps to identify where load bottlenecks are hit and gives useful pointers that help locate where the bottlenecks are, whether they are at the Rational Performance Tester configuration or related to the SAP R/3 server.
Error: “Invalid Control ID” while playing back SAP tests
It is common to observe the error
Invalid Control ID while playing back SAP tests. This error is generated by the SAP GUI when a method is invoked on one of the GUI elements that is not present. This situation can happen if the element has not been rendered to the SAP GUI yet. This issue can be avoided by increasing the think time associated with the SAP screen or by inserting adequate delays in the tests to allow for proper rendering of screens. Ensure an adequate ramp-up rate so that all the users do not login and start transactions with the SAP R/3 servers at the same time.
This error can also occur if the user selected an input record from a modal dialog window by clicking on any column other than the first one while recording. Figure 12 shows the correct way to select a record. Figure 13 shows the incorrect way.
Figure 12. Select a record using the first column (correct)
Figure 13. Select a record by clicking on other columns (incorrect)
Loading unnecessary modal dialogs in a performance test
In the SAP GUI it is common for input fields to use the SAP GUI element GuiComboBox (F4) to select a value from a predefined value set. You can click on the Help button along with the GuiComboBox to open a modal dialog window with the predefined values. Select an input value from that the predefined values. Avoid using the option shown in Figure 15. Instead enter the required value directly in the text box, as shown in Figure 14. This approach helps avoid resource-hungry, but unnecessary GUI pop-ups from being opened.
Figure 14. Direct input of required value (recommended)
Figure 15. Select a value from the modal dialog (not recommended)
This article explains how to optimize SAP R/3 servers and Rational Performance Tester to run high user load tests. In addition, this article covers some of the recommended best practices and common usage pitfalls that happen when using Rational Performance Tester to performance test SAP R/3 servers.
- For guidelines and best practices for load testing using Rational Performance Tester for HTTP tests, read the article Optimize load handling for high-volume tests with IBM Rational Performance Tester (developerWorks, May 2011).
- For more information on SAP gateway parameters, read the Parameterization of the SAP gateway guide.
- Memory management of SAP Gateway gives some useful information on how SAP R/3 handles CPIC requests and can help in troubleshooting issues.
- Learn more about connection pools in JCO in this SAP help article.
- Find out more on the Rational Performance Tester product overview page. Then explore the Rational Performance Tester page on IBM developerWorks for links to technical articles and browse the user assistance in the Rational Performance Tester Information Center for useful guidelines for SAP performance test.
- Stay current with developerWorks technical events and webcasts focused on a variety of IBM products and IT industry topics.
- Improve your skills. Check the Rational training and certification catalog, which includes many types of courses on a wide range of topics. You can take some of them anywhere, anytime, and many of the Getting Started ones are free.
- Download the trial version of IBM Rational Performance Tester.