![Close [x]](http://dw1.s81c.com/i/c.gif){kind=small}
This article provides five tips that a performance test engineer can use to gain deeper insight into performance results:
- Handling spikes in results
- Generating and analyzing percentile reports
- Creating and using business transactions
- Determining volumes during a subset of a run
- Correlating client-side results with server activities
1. Handling spikes in performance test results
In an IBM® Rational® Performance Tester report, the Response vs Time Detail tab plots response time for each page. Extraordinarily high response times, or spikes (as shown in Figure 1), can be a concern if they occur frequently. Analyzing the raw response time data can give useful information about application performance.
Figure 1. Spikes in page response times
To determine the percentage of such outliers, you need the raw data for the test run that Rational Performance Tester uses to plot the performance report graphs. These are the steps to get this data:
- Right-click any tab of the report, and select Export Report Counters to CSV.
- Browse to a location to save the output, and name the file.
- For Export Type, select Full to get all counters.
- Check Export overall results and Split output if columns exceed
250(250 is the maximum column limit for a Microsoft Excel file). - In the location that you just specified, multiple .csv files will get created. Browse the files for the one with this column name: All_Hosts Pages Response Time Page_Name Average interval.
Note:Page_Name here will be replaced by the page titles in your test schedule, and there will be a separate column for each page in the test schedule.
Look for data on pages that show spikes in the graph. This data can now be used to determine the percentage of outliers for each page and troubleshoot potential bottlenecks.
2. Generating and analyzing percentile reports
Percentile response times are often better indicators of the application performance than average response times. Understanding how to generate and analyze performance reports will help you understand system behavior more accurately.
After a test is finished and the performance report has been generated, you can generate a percentile report.
- From the Performance Test Runs window, under the report, right-click All Hosts.
- Select HTTP Reports > Percentile Report, as shown in the drop-down menu selections in Figure 2, to open the percentile report with default analysis targets.
Figure 2. Generating percentile report
By default, a percentile report shows the 85th, 90th, and 95th percentiles. You can define a new value or modify the default percentile analysis target.
- Select Window > Preferences.
- Expand Test and click Percentile Analysis Targets.
- Either click Add and enter a required percentile value, as shown in Figure 3, or select an existing target and click Edit.
- Click OK, and then click Apply to implement your changes.
Figure 3. Custom percentile analysis targets
- Regenerate the percentile report to see your custom percentile target values.
Theoretically, if the nth percentile of a set equals x, it means that n% values in the given set are below the value of x. So, applying that to percentile response time reports in Rational Performance Tester, if the 85th percentile response time for a page is 5 seconds, it means that 85% of virtual users got a response time of 5 seconds or better.
Percentile response times are more relevant than the average times when it is important that each user of the system gets acceptable response times consistently. The more stringent a customer is on each user's experience, the higher the percentile target to be considered. As the number of spikes in the response time set increases, the probability of getting a high percentile response time value for that page also increases.
Also see the Handling spikes in performance test results section.
3. Creating and using business transactions
Customers often characterize non-functional requirements in terms of business transactions. A business transaction typically spans multiple pages; whereas, by default, Rational Performance Tester reports metrics on a page basis. Generating performance results represented in terms of business transactions makes it easier for business users to benefit from what the reports reveal.
For business transaction-based metric reports, define the business transactions first, and then define the required performance metrics first. Then you can generate the transaction reports.
A typical business transaction is representative of a use case consisting of several pages (navigating through required pages to raise a service request, for example). After you have identified a business transaction, you can define it by following these steps:
- Record the scenario in Rational Performance Tester.
- Select all sequential pages in the script that form the transaction.
- Right-click and select Insert > Transaction.
- In response to the question in the pop-up window that asks if you would like to move the selected objects into a new transaction, click Yes.
- Give the transaction a meaningful name, and then Save the test.
Enable performance requirements
A performance requirement specifies acceptable thresholds of performance and helps validate service level agreements. You can define performance requirements on several test elements.
Follow these steps to enable performance requirements on business transactions:
- Select the transaction in the test contents.
- Under Test Element Details, click the Advanced tab.
- Click the check box for Enable Performance Requirements.
- Now define all of the transaction performance requirements that will determine the transaction status (Pass or Fail) of your test. For example, if it is a requirement that the transaction must complete in not more than 120 seconds:
- Select the Maximum Transaction Execution Time [ms] performance requirement.
- Choose the <= operator (less than or equal to operator).
- Enter the value in milliseconds. In this case, enter
120000, as shown in Figure 4.
- Ensure that the page think times and pace times in this transaction align with these requirements. Thus, the sum of all think times, pace times, and expected page response times should fit within this limit.
Figure 4. Business transaction performance requirements
Generate transaction performance reports
If you have transactions defined in your test suite, then the Summary tab of the performance report details a section on transaction performance metrics, such as average execution time, maximum or minimum execution time, and so on, as Figure 5 shows.
Figure 5. Business Transaction Summary report
To get a detailed report for each business transaction, generate the Transaction Report and the Transaction Percentile Report:
- In the Performance Test Runs window, right-click the test run.
- Select either Display Transaction Report or Display Transaction Percentile Report.
- The Transaction Report gives the transaction throughput, transaction count for the run, and a plot of transactions at run time. The Transaction Percentile Report shows execution (run) times by transaction at the defined percentile analysis targets. See the Generating and analyzing percentile reports section for more details.
4. Determining volumes for a specified subset duration
Some performance tests have non-functional requirements for volumes of business transactions achieved over a specified duration in a steady state. By default, a Rational Performance Tester report gives the total number of transactions achieved across the total duration of the run. These results include statistics gathered during the ramp-up phase, where the page hits are much lower. Getting performance metrics during the steady state gives you a more realistic measure of how a system will perform.
You can use the steady state as the time range for your report. Alternatively, you can also define your own time range in the test run by specifying start and end times. By doing this, you will then see metrics in the report that pertain only to that time range.
Follow these steps to define your own time range:
- Click the Server Health Detail tab on the report. The Hits -- Count [for run] column lists volumes against each page or transaction.
- Right-click the report and select Change Time Range.
- In the Select Time Range view (Figure 6), click the New Time Range button to define a new range.
- You then have two choices:
- Either click Set to Steady State
- Or enter values of your choice for Start Time and End Time from within the test run duration
Figure 6. Changing the time range for the report
- Finish and wait for the report to generate with the new time range.
The report will then show statistics only from samples in the range that you specified.
(Also see the Creating and using business transactions section.)
5. Correlating reports with server activities and logs
When monitoring or analyzing performance reports, it is very useful to correlate the reported performance with activities on the server side (through logs, traces, for instance). By default, the Rational Performance Tester performance report shows time across all tabs in seconds (starting at zero). Matching the time scale in the report with that of the server helps correlate performance results with server events and aids troubleshooting.
It is easier to correlate client-side performance metrics from Rational Performance Tester with the server-side log activities if the test report shows absolute times rather than relative times starting from zero.
To customize the time, open a performance report and complete these tasks:
- Go to the Response Vs Time Detail tab on the report.
- Right-click and select Customize.
- For the X Axis field, click the arrow to open the drop-down menu, and select Date & Time (Absolute) to change the template, as shown in Figure 7.
- Click OK and let the report refresh to reflect the change.
Figure 7. Customizing time on reports
Important:
To ensure precise correlation, synchronize the system time on the Rational Performance Tester host controller, agent, and all servers under test.
Learn
- Download and read the PDF titled Using Rational Performance Tester Version 7 (IBM® Redbooks®, 2008).
- Find out more on the Rational Performance Tester product overview page. Then explore the Rational Performance Tester page on IBM® developerWorks® for links to technical articles and browse the user assistance in the Rational Performance Tester Information Center.
- Visit the Rational software area on developerWorksfor technical resources and best practices for Rational Software Delivery Platform products.
- Stay current with developerWorks technical events and webcasts focused on a variety of IBM products and IT industry topics.
- Attend a free developerWorks Live! briefing to get up-to-speed quickly on IBM products and tools, as well as IT industry trends.
- Watch developerWorks on-demand demos, ranging from product installation and setup demos for beginners to advanced functionality for experienced developers.
- Improve your skills. Check the Rational training and certification catalog, which includes many types of courses on a wide range of topics. You can take some of them anywhere, any time, and many of the “Getting Started” ones are free.
Get products and technologies
- Download the trial version of IBM Rational Performance Tester.
- Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.
Discuss
- Join the Performance Testing forum, where you can share you questions and knowledge about IBM performance testing products.
- Share your knowledge and help others who use Rational software by writing a developerWorks article. You'll get worldwide exposure, RSS syndication, a byline and a bio, and the benefit of professional editing and production on the developerWorks Rational website.
- Follow Rational software on Facebook and Twitter (@ibmrational), and add your comments and requests.
- Get involved in the Rational forums, groups, and Rational wikis.
- Connect with others who share your interests by joining the developerWorks community and responding to the developer-driven blogs.
Jigar Kapasi is a software performance architect in the IBM India Software Lab, working with the HiPODS team. He works with large companies to lead critical performance engineering projects, which include architecture and design recommendations, performance testing, infrastructure and middleware analysis and tuning. He has been with IBM for 12 years and holds a bachelor’s degree in Information Technology from Bangalore University.
Priyanka Arora is a software performance analyst with the High-Performance On Demand Solutions team in India. She has more than three years of experience with strong technical analysis, tuning, troubleshooting, tooling, and load test execution of end-to-end performance engagements for several IBM telecom and public sector customers. She has an MSc Tech in Information Systems from Birla Institute of Technology and Science in India and is a certified WebSphere Application Server and Portal Server Administrator.
