Testing Web applications
Most likely you're already familiar with IBM® Rational® Performance Tester, but if not hereâs a quick overview. Rational Performance Tester is a tool designed for testing Web applications in order to capture and correct performance problems before deployment. Rational Performance Tester helps you pinpoint system bottlenecks before deployment by emulating the desired number of concurrent users, and generating reports that clearly highlight poorly performing Web pages, URLs, and transactions.
High-level features include detailed test scheduling at the activity and usage pattern level of each of the user groups. Rational Performance Tester also provides an automated "data pooling" capability that varies the test data set used by each simulated user. Using a browser-like window integrated with the test editor, you can review Web pages accessed during test recording. In addition, advanced testers have the option of inserting custom Java code into their performance tests to perform activities such as advanced data analysis and request parsing.
This article will take a look at some of the features in Rational Performance Tester V7.0. Along with an overview of whatâs new, you'll also record and execute a basic test in order to show off some of the features.
The following is an overview of features available in IBM Rational Performance Tester V7.0. There are a lot; those IBMers have been busy. We'll look at some of these in more detail, others we will only cover at a high level here. There are detailed overviews of all of these features in Rational Performance Tester Help.
Citrix and SAP protocol testing
The Citrix Presentation Server extension enables you to load-test Windows applications running on Citrix Metaframe Presentation Servers. You can accomplish this using window creation and change events, along with optical image recognition techniques, to synchronize user input with server output. Before you record a session with a Citrix application, the behavior of that application must be perfectly reproducible. Specifically, the application must always create windows and GUI elements at the same locations and in the same sequence. Mouse or keyboard events must always produce the same output.
Because the Citrix performance tests interact with the Citrix Presentation Server client at a very low level (mouse movements and key presses), any changes that you make to the test after the recording (such as moving test elements, adding loops or conditions, or inserting new sequences) can alter the context of the emulated user actions and cause synchronization timeouts. It is essential to be aware of the context of user actions when you edit the test.
In addition to Citrix, there is also some support for SAP testing. You can use test elements such as loops, conditions, and transactions anywhere in a test. You can also insert a recording at a selected point in a test suite. Verification points for SAP are enhanced to capture any property of a SAP GUI object and optionally check it against an expected value. Furthermore, you can record and play back SAP applications accessible from a Web interface (by generating SAP Web constructs).
Problem analysis tools
The problem analysis tools do three things. They:
- Collect response time breakdown data
- Collect resource monitoring data
- Provide views and tools for analyzing the collected data to find the causes of performance problems
You do this using the data collection agents in the data collection infrastructure for capturing trace, monitoring, and log data from production or development environments. You also use Eclipse-based tools for viewing and analyzing code and runtime data, and for visually correlating that data.
The performance and problem analysis tools help you find and fix coding problems that can cause distributed performance issues. For those of you familiar with the IBM® Performance Optimization Toolkit, it includes these same tools, just not packaged within Rational Performance Tester. Some of the following feature descriptions (resource monitoring and response time breakdown, for example) go into more detail on some of these tools.
Resource monitoring data consists of a sequence of observations collected at regular intervals. You can collect data in real time, or you can retrieve it from an IBM® Tivoli Enterprise™ Monitoring Server. In addition to response time breakdown data, resource monitoring data provides you a more complete view of a system to aid in problem determination. Following are some examples of data that you can collect and analyze:
- CPU usage (total, for individual processors, or even for individual processes)
- Available memory
- Disk usage
- TCP/IP and network throughput
This feature provides a more complete view of your system to help isolate problems. You can monitor the system under test (or the agents) using IBM® Tivoli® Monitoring agents, Windows Performance Monitor, or the UNIX® rstatd monitor. To view resource monitoring data you can use either the Eclipse Test & Performance Tools Platform (TPTP) viewer, or Rational Performance Tester performance reports.
Response time breakdown
Response time breakdown shows you how much time was spent in each part of the system under test as the system was exercised. The response time breakdown view is associated with a page element (URL) from a particular execution of a test or schedule. This lets you drill down into the response time statistics on any HTTP page element to see how much time was spent in each part of the system under test. You can use response time breakdown to do the following:
- Identify code problems
- See which application on which server is the performance bottleneck
- Drill down further to determine exactly which package, class, or method is causing the problem
To capture response time breakdown data, you must enable it in a test or schedule, and configure the amount of data to be captured. The data collection infrastructure (something you'll see as you install the Rational Performance Tester tools) collects response time breakdown data. Each host on which the application runs, and from which you want to collect data, must have the data collection infrastructure installed and running. In addition, you must configure (or instrument) each application server to use the data collection infrastructure.
Note: Enabling response time breakdown collection for a whole performance schedule can require substantial amounts of memory, so be picky about where and when you enable it.
The next two features are things that I've been waiting for since the first release of Rational Performance Tester. The absence of IP aliasing and support for client-side digital certificates have kept me from using Rational Performance Tester for some of the types of applications I typically performance test. Now those two key features are included.
Rational Performance Tester now has IP aliasing, a much-needed feature. By default, when you run a schedule, each virtual user has the same IP address. But thatâs not what happens in real life when your application is under load. For certain types of applications, this can affect how the load gets distributed, and even impact detailed application functionality. With this feature, you can make each virtual user appear as though it is running on its own host.
To do this, you configure IP aliases on the host computer, and enable IP aliasing in the schedule. When you run the schedule, the network traffic will appear to be generated by multiple hosts. IP aliasing lets you configure an agent so that it appears as though the load is coming from different IP addresses during an HTTP test run.
A digital certificate is a file that binds a public cryptographic key with an identity (a user or an organization). Trusted certificate authorities issue digital certificates, which are then used to authenticate users and organizations for access to Web sites, e-mail servers, and other secure systems. A certificate store is an archive file that contains almost any number of digital certificates, possibly certificates that are issued from different certificate authorities.
With Rational Performance Tester V7.0, you can do the following:
- Create digital certificates
- Access them with datapools
- Associate these datapools with tests
This is done by creating a digital certificate store using a supplied KeyTool command-line program. Digital certificates let you record and run tests against servers using SSL over HTTP, for applications that require client-side digital certificates to authenticate users.
Finally, here are some additional features include in the V7.0 release:
- When you record an HTTP application using the Firefox or Mozilla browser, you no longer have to configure the browser.
- Content verification points now support expected and unexpected results.
- How to replace a host name in a test is now documented in Help.
- The test execution services documentation contains more extensive examples.
- The Performance Testing SDK (software development kit) is available as an installable option.
Creating a performance test script
Letâs take a look at a simple example of a performance test. You'll run a small load (very small actually, you don't want to make anyone angry) against BookPool.com. As you work through the example, you'll look at some of the new features in detail. Others may just be pointed out so that you know where they are.
To create this test script, perform the following steps.
- Open the Create New Test From Recording wizard.
- Select HTTP Recording, as shown in Figure 1, and click Next.
Figure 1. The Create New Test From Recording wizard
- Enter a name for the script, as shown in Figure 2. This test performs a basic search and adds a book to the shopping cart, so this example name (
bookpool) reflects that basic flow.
Figure 2. Entering a name for the test
- Click Finish.
This starts the Recorder, as shown in Figure 3. This operation may take a few minutes.
Figure 3. Starting the Recorder
- If you switch back to Rational Performance Tester, as shown in Figure 4, you can see that the Recorder Control is logging the actions you perform while recording.
Figure 4. Rational Performance Tester Recorder Control
After the Recorder is started, Rational Performance Tester opens the browser to a ReadMe page, as shown in Figure 5. This page outlines some common practices for performance testing. (Note: Depending on your configuration, it may also just load an about:blank page.)
Figure 5. Rational Performance Tester ReadMe page in Firefox
- This is the official launch pad for your test. In the Address bar, enter the URL www.BookPool.com and start recording tests.
- In the search dialog, enter
Software Testingand click Search, as shown in Figure 6.
Figure 6. Search dialog box
- When the results page loads, click the Add to Basket button for the first book returned. When I created this test, that book was How to Break Software: Functional and Security Testing of Web Applications and Web Services (which, by the way, is an excellent book). This is shown in Figure 7.
Figure 7. Example of first book returned
- After the Shopping Cart page loads, close the browser. Closing the browser signals to Rational Performance Tester to stop recording. Rational Performance Tester then generates your script and opens it in the test editor, as shown in Figure 8.
Figure 8. Performance Test: the test editor
The test editor lists a test's HTTP pages by title. When you expand them, you can see the request and response data in each page. You can use the editor to inspect or customize a test that was automatically generated from a recorded session or, if you are brave, you can code a test from scratch. Note that in V7.0 the Common Options and HTTP Options are both tabbed in the Test Element Details section, whereas they used to be on the same screen. You should also see the Enable response time breakdown checkbox at the bottom of the view.
- If you click over to the Common Options tab, as Shown in figure 9, you can see where to specify any digital certificates for your test.
Figure 9. Digital Certificates on the Common Options tab
You wonât be doing anything with digital certificates in this article, but you will take a look at the response time breakdown.
- In order to see what difference it makes, do not select the Enable response time breakdown option on the
AddToCartelement. Instead, enable this option for the Shopping Basket page, as shown in Figure 10.
Figure 10. Selecting Enable response time breakdown for the Shopping Basket
Now you are ready to create a schedule so that you can run your test.
Creating a schedule
Schedules let you group tests, order tests, and run tests at a remote location. A schedule can be as simple as one virtual user running one test, or as complicated as hundreds of virtual users in different groups, each running different tests at different times. As you probably already know, with a schedule you can:
- Group tests to emulate the actions of different users
- Set the order in which tests run: sequentially, randomly, or in a weighted order
- Set the number of times each test runs
- Run tests at a certain rate
- Run one test, or a number of tests, at a remote location
After you create a schedule that describes the behavior for your system, you can run this schedule either using successive builds of the application-under-test, or using an ever-increasing number of virtual users.
To create a schedule:
- Right-click your project, and then select New > Performance Schedule.
- In the Performance Schedule wizard, enter the name of the schedule, and then click Finish, as shown in Figure 11.
Figure 11. Performance Schedule Wizard
- This creates a new schedule with one user group, as shown in Figure 12.
Figure 12. New Performance Schedule
User groups let you group tests in a logical order by using various characteristics representing various types of users on your system. Figure 12 shows one group that contains 100% of the users during execution. With what you have right now, therefore, this would result in 100% of the users searching, adding an item to their cart, changing their mind at the last minute, and leaving to check the price on another Web site. For a more realistic distribution of users, you might make groups for browsers, buyers, merchants, and users who are checking order status. You would then create scripts to represent each of these actions. Also, note that there are a bunch of new tabs in this view. You won't look at all of them in this article, but you will take a look at a few. First, add your test to your user group.
- Right-click the user group, and then click Add > Test.
- In the Select Performance Tests dialog box, select AddToCart and click OK, as shown in Figure 13.
Figure 13. Test added to schedule
Before proceeding, take a quick look at where you can find IP aliasing.
- If you click the user group, you should see IP aliasing in the table under the Schedule Element Details section. When you configure the remote execution location, you set up its IP aliasing options. For this test, just leave the Run this group on the local computer option selected, as shown in Figure 14.
Figure 14. Aliasing under the User Group 1 element details
- Next, you need to alternate the start times for the users. Select ScheduleOne, the root node for the schedule.
- On the General tab in the Schedule Element Details section, select the Add a delay between starting each user box, as shown in Figure 15.
Figure 15. Setting the number of users and the delay between starting each user
- In Delay, enter
100milliseconds, also shown in Figure 15. In addition, note that the Number of users is set to five. Important:Please don't set the number to more than five or ten users. I'm sure BookPool.com doesn't want a denial of service attack.
- Select the Think Time tab and clear the Limit think times to a maximum value checkbox, as shown in Figure 16. I'm not a fan of artificial limits on think times. Note that there are a handful of different options to choose from, including Vary the think time by a random percentage.
Figure 16. Setting the think times
- Finally, select the Response Time Breakdown tab and select both the Enable collection of response time data checkbox and the AddToCart test case.
- Once you have the tests selected, set the options. Set the Detail level to High (letâs see what this tool can do).
- Because the detail level is on high (which can slow you down), limit it to one user for this test.
When you are done, your Response Time Breakdown tab should look like that shown in Figure 17.
Figure 17. Setting the Response Time Breakdown options
Your schedule is now ready to run.
Running your schedule
The hard part is over! To run the schedule, do the following.
- In the Test Navigator, select ScheduleOne.
- Right-click the schedule, and then select Run > Run Performance Schedule.
This launches your test and report generation. You'll see all the reports you saw in earlier versions, plus a few new options and features. For example, reports that include the average response time now also provide a standard deviation from that time. In addition, you can now create a report for a particular time range. This article won't look at all of the reports, but we do want to look at some of the cool new response time breakdowns.
- You can find many of the reports by right clicking a page in any of the default reports, as shown in Figure 18.
Figure 18. Right click a page for report options
From there you have several options, including the following:
- Display Response Time Breakdown Statistics
- Display Host Response Time Breakdown
- Display Page Element Responses
- If you select Display Page Element Responses, you get a display of the average time for each page element, as shown in Figure 19.
Figure 19. Display Page Element Responses
- If you select Display Response Time Breakdown Statistics, you get the Selection Wizard, as shown in Figure 20. From there, you can select an individual element.
Figure 20. The runtime breakdown Selection Wizard
From here you can toggle through the various pages and look at each URLâs detailed response time.
- Select a URL and click Finish to see the detailed Methods for that page, as shown in Figure 21.
Figure 21. Method-level response times
- If you select Display Host Response Time Breakdown, you get a popup of different reporting options, as shown in Figure 22. It's worth noting that in an actual testing project (not just my article example), you would likely have gathered data from the various servers involved in your testing (web server, application server, database server, etc...). That way, the Response Time Breakdown data for each of the servers in the system under test could be displayed in this report.
Figure 22. Host Response Time Breakdown options
- Select Average Base Time (seconds). You should see a report similar to that in Figure 23
Figure 23. Host breakdown for average base time
There are other reporting options, but trust me, if you play around with only these for about a week or so it would be time well spent. Start there. After that explore some of the other reporting options by looking at Help.
This article presents a beginner's look at load testing with Rational Performance Tester V7.0. You can do other kinds of performance testing with the tool as well, but this should get you started. When you get comfortable with the basics of performance testing, you can start using some of the other highlighted features.
As a general rule, if you investigate one new feature every time you open the tool, it should never get too overwhelming. The Rational Performance Tester Help is very good, and there are also good resources related to performance testing on the IBM® developerWorks® Web site. I would recommend looking at some advanced articles or just browsing the Performance and VU Testing forum listed in the Resources section.
- Visit the Performance Tester resource page on developerWorks for training, technical articles, free tutorials, and more.
- Learn about "IBM Rational's software quality offering".
- Browse the technology bookstore for books on these and other technical topics.
Get products and technologies
- Get a trial download of Performance Tester V7.