WebSphere Commerce Performance Tests with Rational Performance Tester, Part 1
Designing tests with Rational Performance Tester
This content is part # of # in the series: WebSphere Commerce Performance Tests with Rational Performance Tester, Part 1
This content is part of the series:WebSphere Commerce Performance Tests with Rational Performance Tester, Part 1
Stay tuned for additional content in this series.
WebSphere Commerce is IBM's enterprise solution for e-commerce. An ongoing concern is website performance and reliability. To address this issue, performance testing and monitoring can be conducted through tools such as Rational Performance Tester. A WebSphere Commerce solution is composed of several software applications and integrations so performance issues can be complex and multifaceted. The deployed hardware resources are another factor because they can limit the scalability of the overall solution.
Performance testing begins with a test plan that defines objectives and criteria that are used to evaluate the results. The testing is iterative in nature because results can lead to application tuning, code modification, or resource reallocation. These changes must then be retested to observe improvement or regression.
Writing tests requires an understanding of what business transactions occur in production and how to realistically recreate those transactions. Transaction information can be found from web analytics tools or by examining application logs and database tables. The volume of transactions to simulate should be based on what is expected during normal and peak business activity. This can be simulated in Rational Performance Tester using properties of a schedule and random selectors.
Performance tests need to provide results that are meaningful and that can be interpreted by stakeholders. Some relevant and key performance indicators include:
- Order throughput
- Response time
- Page views
- CPU utilization
- Cache hit ratio
- Database SQL activity
Order throughput showcases the rate of revenue transactions that the solution can handle. The test workload can be increased to see whether a greater order throughput is possible without exceeding constraints, such as response time and CPU resources. Response time directly impacts customer satisfaction and has an inverse relationship to throughput. The number of page views is an indicator of how popular the website is, regardless of whether it is generating revenue or not. System details, such as CPU utilization, cache hit ratios, and database SQL activity are of interest to architects and developers who need to know what to modify to achieve better performance. Rational Performance Tester can interface with application metrics to provide these details.
This is Part 1 of a three-part article series. Part 1 addresses design and reuse in writing tests. Part 2 examines how to implement browsing in the Commerce catalog. Part 3 describes problem determination and provides tips on investigating test issues.
This article describes recommendations for designing Rational Performance Tester tests. It includes:
- Modularizing tests to reuse them
- Using local and global variables
- Clearing the cookie cache
- Setting verification points against responses
- Sequencing tasks into transactions in a schedule
- Configuring automatic HTTP redirection
The catalog that is used in all the examples is balanced and has three first-level categories. Each category has two subcategories, and each subcategory has three products, as shown in Figure 1.
Figure 1. Catalog structure
Modularization and reuse
In Rational Performance Tester, building a test usually starts from a recording. In this section, an end-to-end browsing session of a registered user was recorded and modified into reusable tests and then used to build a test schedule.
The recording went through the following path, as depicted in Figure 2: Homepage → Login → Browse to top-category #2 through Departments button → Browse to subcategory #4 → Display Product #10 → Logout
Figure 2. Test recording
The recording produced a set of pages, which can be divided into a set of four reusable tests:
Grouping a set of pages or tests under a transaction block allows Rational Performance Tester to report statistics on that transaction. By dividing the recording into small tests and using them inside transactions, you can create a browse transaction for guest and registered users, as depicted in Figure 4.
These tests pass some information to each other. The data is passed from one test to the next using global variables.
Most of these global variables are set from data pools. Some of them are set either by direct variable assignments or by custom code.
One of the best practices is to initialize all global variables from data pools in a separate test (with a name such as InitVariables) and put it at the beginning of the schedule. Doing so avoids modifying all tests whenever a data pool is changed.
Another test should be created to set the login ID for virtual users in each iteration. If user IDs go from sadmin1 to sadmin300, a custom code can randomly generate a login ID from this range. This custom code can be put in the same InitVariables test or in a new test. The second option was used and a new test, InitUser, was created. Rational Performance Tester also has a built-in random number generator in the test data sources when choosing what to substitute a value with.
A data pool with one row of values is used to initialize variables in all tests. Hostname, catalog ID, language ID, and store ID are needed for every test. The UsedIdPrefix and MaxRegisteredUsers values are used to generate the login ID. Rational Performance Tester also allows for a schedule to initialize global variables from a file and for a user group to have its own variable initialization table. The data pool approach was used in these examples.
Figure 3. Constants data pool
The performance schedule is similar to the one shown in Figure 4.
Figure 4. Test schedule
The Transaction blocks under each selector help to define the rules in case variables are not properly set or a verification point fails.
Notice that Login and Logout tests are not under a transaction but outside of it. In a realistic schedule, registered users not only do a browse transaction but also add a product to a cart, order a product, add a product to a wish list, or check orders. Each of these tests can be under its own transaction.
If the login and logout response time should be included as part of a transaction, then they should be moved inside the transaction.
The InitVariables test uses the Constants data pool. The label OUTPUT for the variable container is used to logically indicate that these variables are the output of this test; that is, they were set by this test and are used by others. The visibility of these global variables must be “All tests for this user”, as shown in Figure 6.
Figure 5. InitVariables test
Figure 6. Variable visibility
The tests that need to use the variables set in the InitVariables test should set up a variable container that is labeled INPUT and declare the same variables. The variables must have the same name and the visibility must be set to “All tests for this user”, which means that they are shared across tests.
In the Homepage test, for example, the global variable StoreId is set from the InitVariables test. It is used to substitute any hardcoded store ID in the Homepage.
Figure 7. StoreId variable and substitution
Controlling cookie cache
The first test in a schedule should clear the cookie cache because that simulates a new web browser session. If a loop is used in the schedule to repeat tests, then it is important to configure the behavior of the cookie cache. The cookie cache is cleared, as shown in Figure 8.
Figure 8. Clear cookie cache
To validate tests for correctness, verification points need to be created. They are used to check whether responses have problems. Performance results are invalid if the pages contain errors or unexpected content. There are three types of verification points: response size, response code, and response content. They can be created against a response through the Add context menu. They can also be created for the entire test by clicking the test name in the test tree and clicking Add. Response code verification points and response size verification points can also be turned on automatically in the test preferences. Figure 9 shows how to add a verification point.
Figure 9. Adding a verification point
As an example, Figure 10 shows the dialog for content verification point creation. You can define verification that is based on the strings present or missing in the response content.
Figure 10. Content verification point creation
After a string is entered, you can also set the action when the verification point fails, such as stopping the current transaction or exiting the current loop iteration, as shown in Figure 11.
Figure 11. Verification point action
You can monitor verification points using the verification point report during playback of a test or schedule. This provides real-time information about the correctness of a test run. Figure 12 shows a verification point report.
Figure 12. Verification point report
Using a test schedule
After modularizing a test, as described earlier, you can sequence them in a schedule under a transaction container. You can create different sequences or transactions depending on what scenario is being simulated. For example, the schedule in Figure 13 contains a transaction RegisteredShopper_CompleteOrder and contains the tests necessary to complete an order.
Figure 13. Schedule for registered shopper transaction
You can create a new schedule for guest shoppers based on the registered shoppers version, as shown in Figure 14. You define a new transaction GuestShopper_CompleteOrder and sequence the tests that are required to complete the transaction. Notice that tests that are common to both Registered and Guest shoppers are reused, while others are added or removed because they are specific to either transaction.
Figure 14. Schedule for guest shopper transaction
Virtual users in a Rational Performance Tester schedule are divided into user groups and the schedule provides random selectors for choosing different transactions. You can assign a percentage or absolute quantity of virtual users to a user group. In Figure 15, the schedule shows registered and guest shopper user groups, each with 50% of all virtual users. The guest shoppers can perform either complete order or browse transactions according to their respective weightings in the random selector. The transactions GuestShopper_CompleteOrder and GuestShopper_Browse both have a weight of 1, which means they are equally weighed so there is a 50% chance of either transaction.
Figure 15. Full schedule
Automatic HTTP redirection
Rational Performance Tester V8.3 and later can automatically follow HTTP redirections without having the redirected requests explicitly shown in the test. An HTTP redirection means that a request returns with a request for another URL. An example of HTTP redirection occurs with the Login request. The Login request returns with an HTTP code of 302, meaning it is a redirection and the new request is OrderItemMove. The OrderItemMove redirects to OrderCalculate. OrderCalculate redirects to AjaxLogonForm, which is the final request.
A Login test without automatic HTTP redirection must include these three requests to complete Login successfully. The redirection request chain is hardcoded. If Login has a new redirection sequence, it needs to be re-recorded.
Figure 16. Logon redirection
The more robust approach is to follow redirects as they occur. This is automatic HTTP redirection and is accomplished by changing the expected response status code of Login from [302 – Found] to [200 – OK]. This instructs Rational Performance Tester to complete all redirections until it ends up with [200 – OK], which in the Login test is the AjaxLogonForm page.
As shown in Figure 17 and Figure 18, the Login test is set to use automatic redirection by changing the expected response status code and disabling the redirected requests. It was played back and shows that all the redirection requests are properly made.
Figure 17. Logon auto redirection
Figure 18. Logon auto redirection playback
In this part, Part 1, of the series about designing tests with Rational performance Tester, some best practices are implemented for test design. In Part 2, a real example of browsing a WebSphere Commerce catalog is implemented in which some of these best practices are applied.
The authors thank and acknowledge the following people from Rational Performance Tester for their review and valuable input:
- Kent Siefkes - Lead Architect, Rational Performance Tester
- Dawn Peters - Software Designer, Rational Quality and Requirements Management
- Read about Rational Performance Tester in the Rational Performance Tester information center documentation.
- Read about WebSphere Commerce in the WebSphere Commerce information center documentation.
- Follow developerWorks Commerce communities.
- Download Rational Performance Tester.
- Download WebSphere Commerce.
- Download IBM WebSphere products.