Introduction to IBM Rational Performance Tester V7.0

This article will take a look at some features of Performance Tester V7. Along with an overview of what’s new, you'll record and execute a basic test to show off some of the features.

Michael Kelly (Mike@MichaelDKelly.com), Consultant, www.MichaelDKelly.com

Mike Kelly is currently a software testing consultant for the Computer Horizons Corporation in Indianapolis. He's had experience managing a software automation testing team and has been working with the Rational tools since 1999. His primary areas of interest are software development lifecycles, software test automation, and project management. Mike can be reached by e-mail.



30 January 2007

Also available in Chinese Russian

Testing Web applications

Most likely you're already familiar with IBM® Rational® Performance Tester, but if not here’s a quick overview. Rational Performance Tester is a tool designed for testing Web applications in order to capture and correct performance problems before deployment. Rational Performance Tester helps you pinpoint system bottlenecks before deployment by emulating the desired number of concurrent users, and generating reports that clearly highlight poorly performing Web pages, URLs, and transactions.

High-level features include detailed test scheduling at the activity and usage pattern level of each of the user groups. Rational Performance Tester also provides an automated "data pooling" capability that varies the test data set used by each simulated user. Using a browser-like window integrated with the test editor, you can review Web pages accessed during test recording. In addition, advanced testers have the option of inserting custom Java code into their performance tests to perform activities such as advanced data analysis and request parsing.

This article will take a look at some of the features in Rational Performance Tester V7.0. Along with an overview of what’s new, you'll also record and execute a basic test in order to show off some of the features.

Editor's Notes

This article was written using IBM Rational Performance Tester V7.0.0 running under the Microsoft® Windows® XP Pro SP2 operating system, and the BookPool.com Web site as it existed on November 11, 2006. Please note that as you walk through this article, the BookPool.com Web site may appear different because they are constantly updating content. In addition, this is a very basic walkthrough on creating a script with no modifications suitable for testing Web applications where data is only being read from the system. Tests that involve data input require additional steps, such as datapooling, not covered in this article.

Core features

The following is an overview of features available in IBM Rational Performance Tester V7.0. There are a lot; those IBMers have been busy. We'll look at some of these in more detail, others we will only cover at a high level here. There are detailed overviews of all of these features in Rational Performance Tester Help.

Citrix and SAP protocol testing

The Citrix Presentation Server extension enables you to load-test Windows applications running on Citrix Metaframe Presentation Servers. You can accomplish this using window creation and change events, along with optical image recognition techniques, to synchronize user input with server output. Before you record a session with a Citrix application, the behavior of that application must be perfectly reproducible. Specifically, the application must always create windows and GUI elements at the same locations and in the same sequence. Mouse or keyboard events must always produce the same output.

Because the Citrix performance tests interact with the Citrix Presentation Server client at a very low level (mouse movements and key presses), any changes that you make to the test after the recording (such as moving test elements, adding loops or conditions, or inserting new sequences) can alter the context of the emulated user actions and cause synchronization timeouts. It is essential to be aware of the context of user actions when you edit the test.

In addition to Citrix, there is also some support for SAP testing. You can use test elements such as loops, conditions, and transactions anywhere in a test. You can also insert a recording at a selected point in a test suite. Verification points for SAP are enhanced to capture any property of a SAP GUI object and optionally check it against an expected value. Furthermore, you can record and play back SAP applications accessible from a Web interface (by generating SAP Web constructs).

Problem analysis tools

The problem analysis tools do three things. They:

  • Collect response time breakdown data
  • Collect resource monitoring data
  • Provide views and tools for analyzing the collected data to find the causes of performance problems

You do this using the data collection agents in the data collection infrastructure for capturing trace, monitoring, and log data from production or development environments. You also use Eclipse-based tools for viewing and analyzing code and runtime data, and for visually correlating that data.

The performance and problem analysis tools help you find and fix coding problems that can cause distributed performance issues. For those of you familiar with the IBM® Performance Optimization Toolkit, it includes these same tools, just not packaged within Rational Performance Tester. Some of the following feature descriptions (resource monitoring and response time breakdown, for example) go into more detail on some of these tools.

Resource monitoring

Resource monitoring data consists of a sequence of observations collected at regular intervals. You can collect data in real time, or you can retrieve it from an IBM® Tivoli Enterprise™ Monitoring Server. In addition to response time breakdown data, resource monitoring data provides you a more complete view of a system to aid in problem determination. Following are some examples of data that you can collect and analyze:

  • CPU usage (total, for individual processors, or even for individual processes)
  • Available memory
  • Disk usage
  • TCP/IP and network throughput

This feature provides a more complete view of your system to help isolate problems. You can monitor the system under test (or the agents) using IBM® Tivoli® Monitoring agents, Windows Performance Monitor, or the UNIX® rstatd monitor. To view resource monitoring data you can use either the Eclipse Test & Performance Tools Platform (TPTP) viewer, or Rational Performance Tester performance reports.

Response time breakdown

Response time breakdown shows you how much time was spent in each part of the system under test as the system was exercised. The response time breakdown view is associated with a page element (URL) from a particular execution of a test or schedule. This lets you drill down into the response time statistics on any HTTP page element to see how much time was spent in each part of the system under test. You can use response time breakdown to do the following:

  • Identify code problems
  • See which application on which server is the performance bottleneck
  • Drill down further to determine exactly which package, class, or method is causing the problem

To capture response time breakdown data, you must enable it in a test or schedule, and configure the amount of data to be captured. The data collection infrastructure (something you'll see as you install the Rational Performance Tester tools) collects response time breakdown data. Each host on which the application runs, and from which you want to collect data, must have the data collection infrastructure installed and running. In addition, you must configure (or instrument) each application server to use the data collection infrastructure.

Note: Enabling response time breakdown collection for a whole performance schedule can require substantial amounts of memory, so be picky about where and when you enable it.


Long-awaited features

The next two features are things that I've been waiting for since the first release of Rational Performance Tester. The absence of IP aliasing and support for client-side digital certificates have kept me from using Rational Performance Tester for some of the types of applications I typically performance test. Now those two key features are included.

IP aliasing

Rational Performance Tester now has IP aliasing, a much-needed feature. By default, when you run a schedule, each virtual user has the same IP address. But that’s not what happens in real life when your application is under load. For certain types of applications, this can affect how the load gets distributed, and even impact detailed application functionality. With this feature, you can make each virtual user appear as though it is running on its own host.

To do this, you configure IP aliases on the host computer, and enable IP aliasing in the schedule. When you run the schedule, the network traffic will appear to be generated by multiple hosts. IP aliasing lets you configure an agent so that it appears as though the load is coming from different IP addresses during an HTTP test run.

Digital certificates

A digital certificate is a file that binds a public cryptographic key with an identity (a user or an organization). Trusted certificate authorities issue digital certificates, which are then used to authenticate users and organizations for access to Web sites, e-mail servers, and other secure systems. A certificate store is an archive file that contains almost any number of digital certificates, possibly certificates that are issued from different certificate authorities.

With Rational Performance Tester V7.0, you can do the following:

  • Create digital certificates
  • Access them with datapools
  • Associate these datapools with tests

This is done by creating a digital certificate store using a supplied KeyTool command-line program. Digital certificates let you record and run tests against servers using SSL over HTTP, for applications that require client-side digital certificates to authenticate users.


Other features

Finally, here are some additional features include in the V7.0 release:

  • When you record an HTTP application using the Firefox or Mozilla browser, you no longer have to configure the browser.
  • Content verification points now support expected and unexpected results.
  • How to replace a host name in a test is now documented in Help.
  • The test execution services documentation contains more extensive examples.
  • The Performance Testing SDK (software development kit) is available as an installable option.

Creating a performance test script

Let’s take a look at a simple example of a performance test. You'll run a small load (very small actually, you don't want to make anyone angry) against BookPool.com. As you work through the example, you'll look at some of the new features in detail. Others may just be pointed out so that you know where they are.

To create this test script, perform the following steps.

  1. Open the Create New Test From Recording wizard.
  2. Select HTTP Recording, as shown in Figure 1, and click Next.
    Figure 1. The Create New Test From Recording wizard
    selecting a recorder type
  3. Enter a name for the script, as shown in Figure 2. This test performs a basic search and adds a book to the shopping cart, so this example name (bookpool) reflects that basic flow.
    Figure 2. Entering a name for the test
    select project and file name for test
  4. Click Finish. This starts the Recorder, as shown in Figure 3. This operation may take a few minutes.
    Figure 3. Starting the Recorder
    connecting to agent controller
  5. If you switch back to Rational Performance Tester, as shown in Figure 4, you can see that the Recorder Control is logging the actions you perform while recording.
    Figure 4. Rational Performance Tester Recorder Control
    KBytes Recorded and Recorder Status
    After the Recorder is started, Rational Performance Tester opens the browser to a ReadMe page, as shown in Figure 5. This page outlines some common practices for performance testing. (Note: Depending on your configuration, it may also just load an about:blank page.)
    Figure 5. Rational Performance Tester ReadMe page in Firefox
    welcome to performance test recording
  6. This is the official launch pad for your test. In the Address bar, enter the URL www.BookPool.com and start recording tests.
  7. In the search dialog, enter Software Testing and click Search, as shown in Figure 6.
    Figure 6. Search dialog box
    green Search button
  8. When the results page loads, click the Add to Basket button for the first book returned. When I created this test, that book was How to Break Software: Functional and Security Testing of Web Applications and Web Services (which, by the way, is an excellent book). This is shown in Figure 7.
    Figure 7. Example of first book returned
    screenshot of book image and details
  9. After the Shopping Cart page loads, close the browser. Closing the browser signals to Rational Performance Tester to stop recording. Rational Performance Tester then generates your script and opens it in the test editor, as shown in Figure 8.
    Figure 8. Performance Test: the test editor
    Test Contents and Test Element Details sections
    The test editor lists a test's HTTP pages by title. When you expand them, you can see the request and response data in each page. You can use the editor to inspect or customize a test that was automatically generated from a recorded session or, if you are brave, you can code a test from scratch. Note that in V7.0 the Common Options and HTTP Options are both tabbed in the Test Element Details section, whereas they used to be on the same screen. You should also see the Enable response time breakdown checkbox at the bottom of the view.
  10. If you click over to the Common Options tab, as Shown in figure 9, you can see where to specify any digital certificates for your test.
    Figure 9. Digital Certificates on the Common Options tab
    top section is Datapools
    You won’t be doing anything with digital certificates in this article, but you will take a look at the response time breakdown.
  11. In order to see what difference it makes, do not select the Enable response time breakdown option on the AddToCart element. Instead, enable this option for the Shopping Basket page, as shown in Figure 10.
    Figure 10. Selecting Enable response time breakdown for the Shopping Basket
    option circled in red

Now you are ready to create a schedule so that you can run your test.


Creating a schedule

Schedules let you group tests, order tests, and run tests at a remote location. A schedule can be as simple as one virtual user running one test, or as complicated as hundreds of virtual users in different groups, each running different tests at different times. As you probably already know, with a schedule you can:

  • Group tests to emulate the actions of different users
  • Set the order in which tests run: sequentially, randomly, or in a weighted order
  • Set the number of times each test runs
  • Run tests at a certain rate
  • Run one test, or a number of tests, at a remote location

After you create a schedule that describes the behavior for your system, you can run this schedule either using successive builds of the application-under-test, or using an ever-increasing number of virtual users.

To create a schedule:

  1. Right-click your project, and then select New > Performance Schedule.
  2. In the Performance Schedule wizard, enter the name of the schedule, and then click Finish, as shown in Figure 11.
    Figure 11. Performance Schedule Wizard
    selecting the parent folder
  3. This creates a new schedule with one user group, as shown in Figure 12.
    Figure 12. New Performance Schedule
    User Group 1 (100%)
    User groups let you group tests in a logical order by using various characteristics representing various types of users on your system. Figure 12 shows one group that contains 100% of the users during execution. With what you have right now, therefore, this would result in 100% of the users searching, adding an item to their cart, changing their mind at the last minute, and leaving to check the price on another Web site. For a more realistic distribution of users, you might make groups for browsers, buyers, merchants, and users who are checking order status. You would then create scripts to represent each of these actions. Also, note that there are a bunch of new tabs in this view. You won't look at all of them in this article, but you will take a look at a few. First, add your test to your user group.
  4. Right-click the user group, and then click Add > Test.
  5. In the Select Performance Tests dialog box, select AddToCart and click OK, as shown in Figure 13.
    Figure 13. Test added to schedule
    selecting the test to add
    Before proceeding, take a quick look at where you can find IP aliasing.
  6. If you click the user group, you should see IP aliasing in the table under the Schedule Element Details section. When you configure the remote execution location, you set up its IP aliasing options. For this test, just leave the Run this group on the local computer option selected, as shown in Figure 14.
    Figure 14. Aliasing under the User Group 1 element details
    options circled in red
  7. Next, you need to alternate the start times for the users. Select ScheduleOne, the root node for the schedule.
  8. On the General tab in the Schedule Element Details section, select the Add a delay between starting each user box, as shown in Figure 15.
    Figure 15. Setting the number of users and the delay between starting each user
    schedule name, number of users, and delay
  9. In Delay, enter 100milliseconds, also shown in Figure 15. In addition, note that the Number of users is set to five. Important:Please don't set the number to more than five or ten users. I'm sure BookPool.com doesn't want a denial of service attack.
  10. Select the Think Time tab and clear the Limit think times to a maximum value checkbox, as shown in Figure 16. I'm not a fan of artificial limits on think times. Note that there are a handful of different options to choose from, including Vary the think time by a random percentage.
    Figure 16. Setting the think times
    modify the delay duration
  11. Finally, select the Response Time Breakdown tab and select both the Enable collection of response time data checkbox and the AddToCart test case.
  12. Once you have the tests selected, set the options. Set the Detail level to High (let’s see what this tool can do).
  13. Because the detail level is on high (which can slow you down), limit it to one user for this test. When you are done, your Response Time Breakdown tab should look like that shown in Figure 17.
    Figure 17. Setting the Response Time Breakdown options
    select tests for which to collect data

Your schedule is now ready to run.


Running your schedule

The hard part is over! To run the schedule, do the following.

  1. In the Test Navigator, select ScheduleOne.
  2. Right-click the schedule, and then select Run > Run Performance Schedule.

This launches your test and report generation. You'll see all the reports you saw in earlier versions, plus a few new options and features. For example, reports that include the average response time now also provide a standard deviation from that time. In addition, you can now create a report for a particular time range. This article won't look at all of the reports, but we do want to look at some of the cool new response time breakdowns.

  1. You can find many of the reports by right clicking a page in any of the default reports, as shown in Figure 18.
    Figure 18. Right click a page for report options
    page performance
    From there you have several options, including the following:
    • Display Response Time Breakdown Statistics
    • Display Host Response Time Breakdown
    • Display Page Element Responses
  2. If you select Display Page Element Responses, you get a display of the average time for each page element, as shown in Figure 19.
    Figure 19. Display Page Element Responses
    performance summary
  3. If you select Display Response Time Breakdown Statistics, you get the Selection Wizard, as shown in Figure 20. From there, you can select an individual element.
    Figure 20. The runtime breakdown Selection Wizard
    select a page element for tier analysis
    From here you can toggle through the various pages and look at each URL’s detailed response time.
  4. Select a URL and click Finish to see the detailed Methods for that page, as shown in Figure 21.
    Figure 21. Method-level response times
    response time breakdown statistics
  5. If you select Display Host Response Time Breakdown, you get a popup of different reporting options, as shown in Figure 22. It's worth noting that in an actual testing project (not just my article example), you would likely have gathered data from the various servers involved in your testing (web server, application server, database server, etc...). That way, the Response Time Breakdown data for each of the servers in the system under test could be displayed in this report.
    Figure 22. Host Response Time Breakdown options
    also base time, cumulative time, and calls
  6. Select Average Base Time (seconds). You should see a report similar to that in Figure 23
    Figure 23. Host breakdown for average base time
    host breakdown details

There are other reporting options, but trust me, if you play around with only these for about a week or so it would be time well spent. Start there. After that explore some of the other reporting options by looking at Help.


Next steps

This article presents a beginner's look at load testing with Rational Performance Tester V7.0. You can do other kinds of performance testing with the tool as well, but this should get you started. When you get comfortable with the basics of performance testing, you can start using some of the other highlighted features.

As a general rule, if you investigate one new feature every time you open the tool, it should never get too overwhelming. The Rational Performance Tester Help is very good, and there are also good resources related to performance testing on the IBM® developerWorks® Web site. I would recommend looking at some advanced articles or just browsing the Performance and VU Testing forum listed in the Resources section.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=192030
ArticleTitle=Introduction to IBM Rational Performance Tester V7.0
publish-date=01302007