This article was orignally published in December, 2001 and refers to Rational Suite TestStudio

This is the fifth article in the "User Experience, Not Metrics" series, which focuses on correlating customer satisfaction with your Web site application's performance as experienced by users. The impetus behind the series is my desire to promote the transition in thought that's taking place as more and more managers and stakeholders are demanding validation of a positive user experience under load rather than the more traditional component-based performance metrics. Here's what the series has covered so far:

• Part 1: Introduction
• Part 2: Modeling Individual User Delays
• Part 3: Modeling Individual User Patterns
• Part 4: Modeling Groups of Users

The previous three articles were collectively about modeling real users. If you've read these articles, you understand that a test can only be as accurate as its model, and you know how to determine and model realistic usage patterns. We now move into the next group of three articles, which will focus on capturing and interpreting response times during a load test. In the third collection of three articles, we'll discuss how to organize these time measurements and their interpretations into easy-to-understand reports for project stakeholders. The concepts and methods discussed in this series are based on Noblestar's years of experience and have been derived through dozens of performance engineering projects that all included actual performance tuning of systems.

This article discusses and demonstrates how to use IBM® Rational Suite® TestStudio® to collect accurate and meaningful user-experience measurements through the use of timers. Before reading this article, you should understand the concepts presented in Part 2, since this article is in many ways a continuation of that one. The article is intended for all levels of TestStudio users but will be most useful to intermediate tool users and above.

What should you time?

Collecting meaningful time measurements is a critical part of performance testing. If you collect the wrong measurements, or the right measurements at the wrong time, you may miss bottlenecks and get misleading results, and your measurements may not correlate with actual user experience. Basically, you have a choice of timing three different things:

• user experience (for example, load times for a complete page)
• individual transactions (for example, load times for an individual graphic)
• component performance (for example, database search time)

To help you understand this, let's consider the online bookstore example from our previous articles. At some point, a customer who has decided to order a book or CD starts the process by entering billing information. Because we don't know how well this function within the application will perform, we begin by collecting a user-experience measurement. This measurement captures the time elapsed between the moment the user clicks the Submit button and the moment the "Your transaction has been approved" message appears on the next page.

If we find that the response time of this entire page load, or user experience, is unacceptably long, we can focus on the performance of the components involved in completing the operation. To get some insight into what part of the page load is slow, we can look at the individual transactions that make up that page load, such as individual graphics, text, or database requests. If we notice that the individual transaction containing the database request is slow, and thus that the database component may be the bottleneck, it may be useful for us to measure the time between when the application server sends the update statement to the database and when the database acknowledges the update. In this scenario, all three types of time measurements are used to detect a bottleneck. All of these measurements can be captured using TestStudio.

In my opinion, user-experience measurements rather than component-based metrics should be the end result in performance testing. It doesn't matter how well each component is performing independently if the overall system performs poorly. Component-based testing and tuning has been touted over the years as being synergistic, implying that if each component is tuned to its best possible performance, the system as a whole will perform acceptably. This simply isn't a valid assumption. Until a system is loaded with a realistic user load and observed under this load over time, we can't predict what an actual user's experience will be. It's often valid to assume that if each component is tuned to its best possible performance, the system as a whole can't be further tuned without re-architecting at least some part of the system. But even in this case, stakeholders won't know what the performance limitations of the system are, just that the system is "the best it can be." Stakeholders need to know how many users the system can support before performance becomes unacceptable to be able to make sound business decisions about the system or about the development project as a whole.

This doesn't mean that you shouldn't do component-based testing, only that you shouldn't do it exclusively. Testing components individually is a necessary part of the tuning process. In fact, component-based measurements complement user-experience measurements to create a complete picture of system performance as a whole, as we saw in the bookstore scenario above. If the actual performance of a system or application is unknown, or if performance is known but no bottleneck has been determined, the place to start testing is with user-experience measurements, an approach to performance measurement collection known as top-down. Once a specific bottleneck has been identified, we can take component-based measurements, in an approach known as bottom-up.

In summary, my answer to the question "What should you time?" is that every performance test should include user-experience measurements, and that individual transactions or component-based transactions should only be timed when necessary to find or resolve a bottleneck.

Let's look further at the top-down and bottom-up approaches, and at timing individual transactions.

Timing user experience first: the top-down approach

Simply put, a top-down approach to measuring performance begins by determining actual user experience and only moves into further analysis when poor performance is detected. As we saw in the bookstore example, user-experience measurements capture the time that passes from when a user takes a particular action to when the user sees the results of that action. This applies to all types of applications, whether Web, client-server, embedded software, or whatever. As another example, if you were to time how long it took for Part 2 of this series to fully load when you clicked on the link at the beginning of this article, that would be a user-experience measurement.

Generally, when you conduct a performance test, very clear performance goals have been established. The goals should be focused on an acceptable user experience and may look something like this:

"All static pages on the Web site will display in 8 seconds or less on a 56.6 kbs modem 95% of the time while 500 users are accessing the site according to the documented user community model."

It's easy to see that page generation, communications between the client and the server, and client-side processing are what need to be timed in the case of this requirement. For a Web-based application, this will be the most common starting measurement. These measurements are collected with Rational TestStudio using timers that are placed as described later in this article. When these timers reveal measurements that don't meet the performance goals, the next level of measurements is needed -- individual transaction times, discussed below. Luckily, TestStudio collects this next level of detail automatically.

Timing component performance first: the bottom-up approach

If previous testing has determined that only user transactions requiring a database search yield unacceptable performance, TestStudio can be used to record and play back database searches directly against the database. This can be done without going through the client computer, the Web server, or the application server, and will simulate the application server communicating with the database server. In this way only the actual time of the database search will be captured and not any of the additional time taken by the application server to process the data, the Web server to present the data, or the client to display the data. These measurements -- called database response times -- are one type of individual transaction measurement.

At the same time that you're executing this test, you can use TestStudio to collect component-based measurements, such as CPU utilization and memory usage. To do this, you must have TestStudio agent software installed and configured on the database server. Alternatively, those measurements can be collected manually by using a third-party performance-monitoring tool such as PerfMon (a component of Windows operating systems) or PerfMeter (a component of the Solaris operating system).

The kind of load generated for this type of test won't be modeled to match real user patterns but rather to pinpoint bottlenecks on the database server. In this way, the database can be tuned for the best possible performance while being searched. At the conclusion of component tuning -- in this case, of the database component -- user-experience measurements must be collected to determine if the overall performance goals have been met.

Occasionally, a system or application will have specific component load or performance requirements. For example, a database may be required to sustain 30 transactions per second without queuing. This type of requirement often comes into play when an individual component of the system is outsourced. In this case, the provider must be held to very specific performance requirements. This is usually done by establishing specific service-level agreements and by periodically testing the system to validate compliance. Here's an example of a performance requirement of a service-level agreement for an outsourced database:

"All database transactions will be processed in two seconds or less 95% of the time while the database is receiving five or fewer requests per second."

It's well beyond the scope of this article to discuss how to configure TestStudio to do the kind of testing I just described, or how to tune our theoretical database. If you find yourself in a situation where you've pinpointed a bottleneck and created a repeatable test to exploit that bottleneck but don't know how to fix it, you can always do what I do -- call the smartest person you know and offer to buy her or him lunch for some insight!

If these types of measurements aren't granular enough to detect or tune a particular bottleneck, you may need to look to another tool, such as a code profiler, to collect what are known as "white box" measurements. TestStudio can only capture hardware statistics and times for transactions that occur over a network of some sort. If the bottleneck is actually inside a piece of code, that's where profilers can be used. Rational Software has several tools available that may be appropriate for this level of analysis, such as Purify®, Quantify®, and PureCoverage®. These tools are part of the Rational TestStudio package or are available separately.

Timing individual transactions

Timing individual transactions is a part of every performance-testing approach. Whether the object of testing is a Web site, a client-server application, or the database component of a multi-tier application, every communication that occurs between the computers involved is made up of individual transactions. A Web page may be generated in a single transaction, but most Web pages are actually painted as a result of many (I've seen as many as 46) individual HTTP requests, or transactions. These transactions represent each individual object that must be retrieved from the server to then be displayed to the user. In contrast, a client-server application may handle everything except database transactions on the client side, so there may only be one measurable transaction per user interaction with the system.

This is why when you insert timers into your script while recording, you often see many commands in the script between the start and stop times. If you refer to the script we created in Part 2 ( RDN_on_Noblestar_Modified.txt ) -- you'll notice that Page 1 generated three individual transactions, or http_request commands, and Page 2 generated twelve individual transactions. You can see that these individual transactions yield a user-experience measurement when viewed collectively but represent just one part of the user experience when viewed separately. It's valuable to see these individual transactions because they can often show you, for example, whether the slow component of a Web page is a graphic or the database search.

Back to top

Where should you put the timers?

Top-down or bottom-up, we eventually want to collect user-experience measurements. In this section we'll explore where to place timers while recording or editing VU scripts. I'm assuming you're familiar with inserting block timers and timers into a script while recording. If not, you'll find information about how to do this in the documentation that came with your software.

Determining what user activity goes inside and outside the timers is, in most cases, a fairly simple task. Whenever a user clicks a button or a link that causes communication back to the server, the response needs to be captured and timed. On a static Web site, each timer starts immediately before performing an activity that leads to a new page being generated and stops as soon as that page is completely painted.

The question of where to place timers gets a little more complicated when there are active components on the client side of the application. It's often unclear to the tester whether processes such as data validation upon submittal or active menu bars that change based on mouse position invoke client-side or server-side processing. VU scripts don't actually measure client-side processing time during playback but do indirectly collect that time during recording. A developer can tell you what invokes server-side processing and what doesn't, but if you don't have access to one you'll just have to experiment.

The easiest way to do this is to block everything with block timers or timers while recording and then manually delete timers in your script that have no activity between the start and stop commands. As an example, let's return to the Noblestar home page. There we notice that as we hover over the menu selections in the top menu bar, the bottom menu bar changes dynamically based on which main heading is currently highlighted. Initially I thought this was all client-side processing, but when I recorded a script as follows, I found that I was wrong.

1. Start a timer, or block timer, named "Home Page" and launch the browser going directly to the Noblestar home page.
2. Stop the timer as soon as the page is loaded.
3. Start a new timer named "Menus."
4. Move your mouse around the screen, ensuring that you highlight several menu options, but don't click on any.
5. Stop the timer.
6. Start a new timer called "Page1."
7. Move directly to a menu option or link, and click it.
8. Stop the timer as soon as the next page is loaded.
9. End the recording.

In the resulting script ( Noblestar_menu_example.txt ) a lot of requests and receives were recorded inside the "Menus" timer. This indicates that hovering over the menu selections actually generates requests to the server for information, and the changes to the bottom menu bar aren't dynamically generated on the client side. If they were generated client-side, no communications would be captured and there would be no commands between the start and stop timers. What this means to us is that hovering over the menu selections is a user experience separate from the initial page load time, and this experience could be timed separately if it's of interest to us or our stakeholders.

So where should I put my timers in this example? The "Home Page" timer was captured correctly. While I could enter timers that block hovering over each menu selection, that would really be overkill. In truth, the transactions that produce the dynamic menu are very few, very fast, and cached the first time. If I were to use block timers (discussed in detail below) rather than timers, I could stop the "Home Page" block timer, move the mouse to highlight the link that I want to navigate to next, then start the "Page1" block timer. That's what I've done in the Noblestar_block_example.txt script. In the next section I'll show you how the results are organized when this script is executed.

The key to recording such a script is to ensure that the link you want to click is visible on the screen before starting the timer, and ensuring that you don't hover over a menu option that will change what's visible on the screen before clicking the link. This method logically separates the script into two distinct page load times and an identifiable set of individual transaction times that can be analyzed for the menu transactions. Listing 1, below, is a section of the script that shows just the block timers and a small selection of command IDs.

/*
->-> Session File Information <-<-
*/

#include <VU.h>
{
push Http_control = HTTP_PARTIAL_OK | HTTP_CACHE_OK | HTTP_REDIRECT_OK;
push Timeout_scale = 200; /* Set timeouts to 200% of maximum response time */
push Think_def = "LR";
Min_tmout = 120000; /* Set minimum Timeout_val to 2 minutes */
push Timeout_val = Min_tmout;
/* Start_Block "Home Page" */

push Think_avg = 0;

www_noblestar_com_2 = http_request \["Home Pa001"\] "www.noblestar.com:80",
HTTP_CONN_DIRECT,
"GET / HTTP/1.1\r\n"
"Accept: */*\r\n"
"Accept-Language: en-us\r\n"
"Accept-Encoding: gzip, deflate\r\n"
"User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)\r\n"
"Host: www.noblestar.com\r\n"
"Connection: Keep-Alive\r\n"
"\r\n";

{ string SgenURI_001; }
SgenURI_001 = _reference_URI; /* Save "Referer:" string */

start_time \["Home Page"\] _fc_ts;

set Server_connection = www_noblestar_com_2;

http_header_recv \["Home Pa002"\] 200; /* OK */

http_nrecv \["Home Pa003"\] 100 %% ; /* 1178 bytes */
?
stop_time \["Home Page"\]; /* Stop_Block */

set Think_avg = 20119;

set Server_connection = www_noblestar_com_1;

/* Keep-Alive request over connection www_noblestar_com_1 */
http_request \["Noblest~001"\] 
"GET /images/spacer.gif HTTP/1.1\r\n"
"Accept: */*\r\n"
"Referer: " + SgenURI_003 + "\r\n"
/* "Referer: http://www.noblestar.com/global/top.html" */
"Accept-Language: en-us\r\n"
"Accept-Encoding: gzip, deflate\r\n"
"If-Modified-Since: Fri, 29 Dec 2000 18:58:43 GMT\r\n"
"If-None-Match: \"457fe811-1-55-3a4cdee3\"\r\n"
"User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)\r\n"
"Host: www.noblestar.com\r\n"
"Connection: Keep-Alive\r\n"
"Cookie: NSES40Session=94f%253A3ca25b7e%253A53c78693dbcf7e9d\r\n"
"\r\n";

http_header_recv \["Noblest~002"\] 304; /* Not Modified */

http_nrecv \["Noblest~003"\] 100 %% ; /* 85 bytes - From Cache */
?
http_request \["Noblest~109"\] 
"GET /images/enterprise_on.gif HTTP/1.1\r\n"
"Accept: */*\r\n"
"Referer: " + SgenURI_003 + "\r\n"
/* "Referer: http://www.noblestar.com/global/top.html" */
"Accept-Language: en-us\r\n"
"Accept-Encoding: gzip, deflate\r\n"
"If-Modified-Since: Thu, 31 Jan 2002 22:30:19 GMT\r\n"
"If-None-Match: \"654a4790-1-12e-3c59c57b\"\r\n"
"User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)\r\n"
"Host: www.noblestar.com\r\n"
"Connection: Keep-Alive\r\n"
"Cookie: NSES40Session=94f%253A3ca25b7e%253A53c78693dbcf7e9d\r\n"
"\r\n";

http_header_recv \["Noblest~110"\] 304; /* Not Modified */

http_nrecv \["Noblest~111"\] 100 %% ; /* 302 bytes - From Cache */
/* Start_Block "Page1" */

set Think_avg = 9273;

set Server_connection = www_noblestar_com_1;

/* Keep-Alive request over connection www_noblestar_com_1 */
http_request \["Page1001"\] 
"GET /images/custom_off.gif HTTP/1.1\r\n"
"Accept: */*\r\n"
"Referer: " + SgenURI_003 + "\r\n"
/* "Referer: http://www.noblestar.com/global/top.html" */
"Accept-Language: en-us\r\n"
"Accept-Encoding: gzip, deflate\r\n"
"If-Modified-Since: Thu, 31 Jan 2002 22:30:18 GMT\r\n"
"If-None-Match: \"f5065728-1-136-3c59c57a\"\r\n"
"User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)\r\n"
"Host: www.noblestar.com\r\n"
"Connection: Keep-Alive\r\n"
"Cookie: NSES40Session=94f%253A3ca25b7e%253A53c78693dbcf7e9d\r\n"
"\r\n";
start_time \["Page1"\] _fs_ts;


http_header_recv \["Page1002"\] 304; /* Not Modified */

http_nrecv \["Page1003"\] 100 %% ; /* 310 bytes - From Cache */
?
stop_time \["Page1"\]; /* Stop_Block */

}

Back to top

Which should you use -- timers or block timers?

Fundamentally, timers and block timers perform the same overall function -- they both time the stuff that happens between the start and stop commands in the script. There are a few differences that are worth discussing, though. To illustrate these, let's take a look at comparable segments of the Noblestar_block_example script (Listing 2) and the Noblestar_menu_example script (Listing 3). Though the start_time commands in Listings 2 and 3 are in different places, they both time the same thing. In effect, these are two different ways of scripting exactly the same time measurements.

/* Start_Block "Home Page" */

push Think_avg = 0;

www_noblestar_com_2 = http_request \["Home Pa001"\] "www.noblestar.com:80",
HTTP_CONN_DIRECT,
"GET / HTTP/1.1\r\n"
"Accept: */*\r\n"
"Accept-Language: en-us\r\n"
"Accept-Encoding: gzip, deflate\r\n"
"User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)\r\n"
"Host: www.noblestar.com\r\n"
"Connection: Keep-Alive\r\n"
"\r\n";

{ string SgenURI_001; }
SgenURI_001 = _reference_URI; /* Save "Referer:" string */

start_time \["Home Page"\] _fc_ts;

set Server_connection = www_noblestar_com_2;

http_header_recv \["Home Pa002"\] 200; /* OK */

http_nrecv \["Home Pa003"\] 100 %% ; /* 1178 bytes */

set Think_avg = 1602;
?
stop_time \["Home Page"\]; /* Stop_Block */

start_time \["Home Page"\];

push Think_avg = 0;

www_noblestar_com_2 = http_request \["Noblest~001"\] "www.noblestar.com:80",
HTTP_CONN_DIRECT,
"GET / HTTP/1.1\r\n"
"Accept: */*\r\n"
"Accept-Language: en-us\r\n"
"Accept-Encoding: gzip, deflate\r\n"
"User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)\r\n"
"Host: www.noblestar.com\r\n"
"Connection: Keep-Alive\r\n"
"\r\n";

{ string SgenURI_001; }
SgenURI_001 = _reference_URI; /* Save "Referer:" string */

set Server_connection = www_noblestar_com_2;

http_header_recv \["Noblest~002"\] 200; /* OK */

http_nrecv \["Noblest~003"\] 100 %% ; /* 1178 bytes */

set Think_avg = 11582;
?
stop_time \["Home Page"\];

If you look closely at these two script segments, you'll notice three differences:

1. The two start_time commands are in different places.
2. The start_time command in Listing 2 has an extension of _fc_ts, while the start_time command in Listing 3 has no extension.
3. Listing 2 shows command IDs inside the timer as \["Home Pa001"\], and so on, while Listing 3 shows them as \["Noblest~001"\], etc.

The third difference, in how individual transactions are labeled, is the one that generally determines which method testers choose. Before I show you why, I'll say a little more about the other two differences.

The first two differences we noticed are actually related. The start_time commands are in different places because of the extension. The start_time command supports six extensions, which specify the point during a transmission when the timing begins. The extensions and what they stand for are as follows:

_fc_ts -- First Connect Timestamp
_fs_ts -- First Send Timestamp
_fr_ts -- First Receive Timestamp
_lc_ts -- Last Connect Timestamp
_ls_ts -- Last Send Timestamp
_lr_ts -- Last Receive Timestamp

When no extension is specified, the script defaults to _fs_ts and, thus, when the two script segments above are executed, they both time the same thing. It's beyond the scope of this article to say more about the extensions and their uses, because it's a very complex topic, but at least now you know they exist.

Now let's return to the topic of how command IDs are labeled and why testers care. When a timer is used, the command ID label is always the first eight characters of the name of the script (or the first seven followed by a ~ if the name of the script is longer than eight characters), followed by a sequence number. When a block timer is used, the command ID labels inside the block are the first eight characters of the name of the timer block followed by a sequence number. (Command ID labels outside the block follow the convention for timers, though.) This difference may not seem significant at first glance, but when you start interpreting times (as described in the next article in this series), you'll find that it's much easier to track down bottlenecks when you can see which timer each individual transaction time belongs to.

I'll show you what I mean. First, I executed the Noblestar_menu_example script for one user over ten iterations and got the results in TestManager shown in Figure 1.

Figure 1: Partial results from executing the Noblestar_menu_example script

(click here to enlarge)

Then I executed the Noblestar_block_example script for one user over ten iterations and got the results in TestManager shown in Figure 2.

Figure 2: Partial results from executing the Noblestar_block_example script

(click here to enlarge)

With the results shown in Figure 1, the only way to associate a command ID with a specific timer is to return to the script, find the command in question, then search the script to see what timer it comes immediately after. With the results shown in Figure 2, on the other hand, it's easy to see which timer a command ID relates to -- a good reason to use block timers instead of timers.

Before we leave the topic of using block timers versus timers, let me add a note about inserting delay times in each case. In Part 2 of this series we discussed how to insert delay times when using timers. In the case of block timers, all of the same theories apply, but the delays need to be placed immediately after the previous stop_time command rather than immediately before a start_time command. In Listing 2, for example, we would add the line:

delay(uniform(2000,8000));

immediately after the line:

stop_time \["Home Page"\]; /* Stop_Block */

If there's no previous stop_time command, the delay needs to be manually placed immediately above the preceding http_request block.

One more postscript: To date, we haven't discussed Think_avg settings in a script in any detail. If you look at the Think_avg times set in Listings 2 and 3, you'll notice that in Listing 2 it's 1602 milliseconds (or 1.6 seconds) and in Listing 3 it's 11582 milliseconds (or 11.6 seconds), and in both cases they're inside the timer. Also in both cases, it should be obvious that 1.6 or 11.6 seconds of client-side processing time is much longer than what you would actually observe while recording the home page transaction. That means that these Think_avg settings represent user think time, not client-side processing time. In Part 2, we discussed substituting delays for Think_avg settings to simulate user think times. Once you've done that, it's important to search through the rest of your script and adjust the remaining Think_avg settings to ensure they reflect client-side processing time, not user think time inside the timers.

There are literally dozens of settings in TestStudio related to Think_avg times. I can't address them all here but do want to mention one. If you're following the approach outlined in these articles, I recommend setting the CPU/User threshold time to 250 milliseconds. Experience has shown that it's very rare for client-side processing to exceed 250 milliseconds; in situations where client-side processing does exceed that amount, significant additional considerations apply that are beyond the scope of this article.

Back to top

Nesting timers

From the very first time I learned about using Rational tools for performance testing, I was told never to overlap timers -- that is, to have two timers running at once. Through a significant amount of research and testing, I've found an exception to the "don't overlap timers" rule that I'd like to share with you.

Generally, there's no need to try to overlap timers if you're measuring page-loading times for Web pages. But I've found that it sometimes adds value to put timers around individual frames in Web pages as well as around the entire page. This is known as nesting timers. Now, I admit, to do that you have to be able to determine manually which command IDs are part of which frame, but that's beside the point.

If you do decide to nest timers, be sure to get the order of the starts and stops right. For nested timers to truly be nested and to return the correct values, it's important to organize them so that one timer is completely enclosed within the other. For example, these timers are nested properly:

start_time \["Outer"\];
start_time \["Inner1"\];
stop_time \["Inner1"\];
start_time \["Inner2"\];
stop_time \["Inner2"\];
stop_time \["Outer"\];

These timers are staggered, not nested, and will yield incorrect results:

start_time \["Outer"\];
start_time \["Inner1"\];
start_time \["Inner2"\];
stop_time \["Outer"\];
stop_time \["Inner1"\];
stop_time \["Inner2"\];

If you want to test this yourself, create a script with the timers listed above, put varying delays between each set of timer commands, and compare the results with the known delay times in IBM® Rational Suite® TestManager®.

Back to top

Now you try it

Rather than writing separate exercises to demonstrate the use of block timers and timers, I recommend that you follow the steps I outlined under "Where Should You Put the Timers?" to replicate the scripts I created for this article. You're also welcome to download the scripts linked to this article and play them back yourself (although I request that you play them back with ten or fewer users so my Webmaster doesn't have a heart attack).

If you've been using TestStudio for some time, you probably already have a preference for block timers or timers, but I suggest that you take the time to experiment with the use of each. I'd been a "timer guy" for many years before writing this article, but while I was creating the examples I found that block timers would have been quite valuable on some of my recent projects.

If you should decide to do research on overlapping or nesting timers, please share your findings with me. As I mentioned, this is an area I've been researching for some time.

Back to top

Summing it up

"Even with the response-time market in a continuing state of development, enterprises should deploy application response-time tools now. The measurements from these tools should be used to understand the response-time characteristics of the most critical applications, to help establish and meet service-level goals, and to correlate the performance experienced by external users with customer satisfaction." So wrote the authors of "End-to-End Application Response Times: Market Update" from Gartner Inc. in October 2000. Since then, I've been using this quotation shamelessly to promote user-experience measurements. In this article you learned a little more about why I consider such measurements essential.

Besides that, you learned that to collect and analyze user-experience measurements, you must place your timers correctly. Improperly placed timers will yield misleading or inaccurate results, while correctly placed timers will show you exactly what the current performance of a system or application is, and help you determine what, if anything, needs to be tuned for improved performance.

About the author

Comments (Undergoing maintenance)

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• What should you time?
• Where should you put the timers?
• Which should you use -- timers or block timers?
• Nesting timers
• Now you try it
• Summing it up
• About the author
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

My developerWorks community

Dig deeper into Rational on developerWorks

Try Rational System Architect

See how this fully integrated collection of models and documents can help you better align your IT systems and infrastructure to simplify business processes

Special offers