 | Additional analysis reports
This section will explain performance reports (although many of them are self
explanatory) by focusing on and explaining the content of each report. In order to
pinpoint performance bottlenecks, analyzing the right reports is essential. You will be
viewing the default report presentations in this section; the customizations of these
reports are discussed in a later section.
Percentile Report
The Percentile Report - Summary graphs show response time versus page counter for
response time distribution for all pages in three percentiles: 85th, 90th, and 95th. You
can apply filters to the graph. The 85th percentile bar indicates that 85% of all users
achieved the response time or better. The 90th percentile bar indicates that 90% of all
users achieved the indicated response time or better. Likewise, for the 95 percentile bar,
95% of all users achieved the indicated response time or better. The following bullets
describe the Performance Summary table shown in Figure 1:
- The first left column displays the page name.
-
Response Time - Minimum for Run. This is the fastest response time for that
page.
-
Response Time - Average for Run. This adds up all of the average response times
for that page.
-
Response Time - Maximum for Run. This is the slowest response time for that
page.
-
Response Time - Standard Deviation for Run. This is the deviation from the mean.
A bigger deviation number means there was less consistency in the response times, and a
smaller deviation number means more consistency.
-
Response Time - 85th Percentile. 85% of that page response time was equal or
faster than the time shown.
-
Response Time - 90th Percentile. 90% of that page response time was equal or
faster than the time shown.
-
Response Time - 95th Percentile. 95% of that page response time was equal or
faster than the time shown.
-
Attempts - Rate for Run. In one second, how many times that page was sent to the
server.
Figure 1. Percentile Report:
Summary
Verification Point
Report
Verification points ensure that the expected result can be delivered for a particular
page or page element. Unexpected returns during a test run will be reported as a failed
verification point. There are four types of verification points:
-
Page title. Page title is a case-sensitive comparison that ignores spaces between
words. This verification works for the primary page.
- To enable the page title verification point, highlight the primary page and
select the Enable Verification Point box at the bottom right corner.
Alternatively, right-click the primary page to select Verification Points
> Enable Page Title VPs.
- To turn on page title verification points for all pages, select and enable them
from the highest hierarchy in the test contents.
-
Content. Content verification applies string matching to ensure that an expected
string is found. A failure is defined as at least one of the search strings is
found or none of the search strings are found. You can enable content
verification points for all of the elements by selecting a test element in a higher
hierarchy, or simply select an individual element to be enabled. In other words,
enabling at the test, page, and page element level is supported.
-
Response Code. Response code verification point sets the passing criteria using
the returned response code (set at the test, page, or page element level). You can set
the response code matching method to either Relaxed or Exact. A relaxed
match is a match that gives a pass for response codes that fall in the same category,
while an exact match will give a fail if the response code does not exactly match the
specified code.
-
Size. A failed response size verification point is defined as the verification
point that fails the size matching test. Again, you can enable the verification point at
the test, primary page, and page element level. For the primary page, the default
matching method is range (bytes and %), while page element uses exact
matching method as the default. Other matching methods include At Least and At
Most.
The Verification Point - Summary report, shown in Figure 2, gives you a summary of
the aforementioned verification points. It is a line graph representing the summary of
percent page verification points passed for all pages, with each point referring to an
interval. The table gives information such as the sum total page verification points
attempted, passed, and failed.
Figure 2. Verification Point Report:
Summary
The Page Verification Points report, shown in Figure 3, is a table showing information
for page verification points' pass, fail, and percent pass on a per-page basis. Note that
only primary pages that are enabled with verification points will be shown here.
Figure 3. Verification Point Report: Page
Verification Points
The Page Element Verification Points report, shown in Figure 4, displays the pass count,
fail count, and percent pass.
Figure 4. Verification Point Report: Page
Element Verification Points
Transaction Report
A transaction is a collection of elements (both primary pages and page elements) that can
be gathered for better performance analysis. To select elements that constitute a
transaction, perform a multiple select under the Test Contents panel, as shown in Figure
5. The transaction elements may not be (and often are not) sequential in the order listed
in the Test Contents panel. In other words, you can pick any element to make up a
transaction. Usually, you gather a few test elements whose performance you are interested
in to make up a transaction.
Figure 5. Adding a transaction
The Transaction Report consists of three default reports: Overall transaction rate,
Transaction Duration vs. Time, and Transaction Throughput.
The Overall Transaction report, shown in Figure 6, presents the average execution time
for all transactions, with each point representing an interval. The table shows the
execution time standard deviation, minimum time, and maximum time for all
transactions.
Figure 6. Transaction Report:
Overall
The Duration vs. Time Transaction Report, shown in Figure 7, may consist of an individual
page or page element within a test. This report shows the average execution time, with
each point representing an interval. Each transaction is shown as an individual line in
the graph. The table provides the following information related to each transaction:
-
Minimum Execution Time. This is the minimum execution time for the run.
-
Maximum Execution Time. This is the maximum execution time for the run.
-
Average Execution Time. This is the average execution time for the run.
-
Execution Time Standard Deviation. This is the deviation from the mean for the
run.
-
Rate of Completion. This is the transaction completion rate measured in
seconds.
-
Total Attempts. This is the total attempts made by the transaction for the
run.
Figure 7. Transaction Report: Duration vs.
Time
The Transaction Throughput report, shown in Figure 8, includes line graphs and two
tables. The left graph shows the transaction start and complete rate measured in seconds,
with each point representing an interval. The left table provides the transaction
completion rate measured in seconds, and also the completion count for the run. The right
graph shows the user load being added (active users), and the number of users who
completed the run (completed users). The table on the right gives the count for active,
completed, and total users.
Figure 8. Transaction Report: Transaction
Throughput
| |
