This article was orignally published in December, 2001 and refers to Rational Suite TestStudio

By this time you're pretty confident in your ability to test a Web-based application. You've got some successful testing projects behind you and you're feeling good. Now your boss comes to you and says, "Great job on those performance tests! I've got another one for you -- a custom application that needs to be performance tested. I'm sure you can handle it. Oh, by the way, it doesn't use HTTP. Good luck!" What do you do now?

Thus far in the "User experience, not metrics" series, all of the articles have focused on scripts that used the Hypertext Transfer Protocol (HTTP). But the fact is that not all Web-based applications use this communication protocol (and other types of applications that you might be testing don't at all). The TestStudio system testing tool does support scripts generated from the capture of traffic that uses a few other protocols, but what if the application you're supposed to test uses an unsupported protocol? That's the challenge we'll be exploring in this article, which is the final installment of the "User experience, not metrics" series.

Here's what the series, which focuses on correlating customer satisfaction with your application's performance, has covered so far:

• Part 1: Introduction
• Part 2: Modeling individual user delays
• Part 3: Modeling individual user patterns
• Part 4: Modeling groups of users
• Part 5: Using timers
• Part 6: Working with outliers
• Part 7: Consolidating test results
• Part 8: Choosing tests and reporting results to meet stakeholder needs
• Part 9: Summarizing results across multiple tests
• Part 10: Creating a degradation curve
• Part 11: Handling authentication and session tracking
• Part 12: Scripting conditional user path navigation

This article is intended for intermediate to advanced TestStudio users. A general knowledge of TCP/IP communication protocols will aid you in applying the concepts discussed.

A Primer on protocols

What do protocols have to do with performance testing? TestStudio VU scripts capture traffic at the communication protocol level, as do all of the top load-generation tools. This means that you need to be aware of the protocol used by the application under test and make adjustments to the recording settings and test scripts accordingly. Those adjustments will be explained in detail later in this article, but first let's make sure you understand the ABCs of protocols. If you already do, you can skip this section.

Basically, a communication protocol is a way for one computer to communicate with another across a network. It's the way Web servers communicate with Web browsers. A protocol isn't a language but rather a standard format for requesting and receiving data. Furthermore, there are high-level and low-level protocols. To understand the relationship between the two, it might help to draw a comparison to written mail. The letter would be the file being requested or received, the high-level protocol would be the rules dictating how the envelope is addressed, and the low-level protocol would be the postal service responsible for picking up and delivering the letter.

HTTP is the predominant high-level communication protocol and TCP/IP is the most common low-level communication protocol in use today. HTTP as well as a large number of other high-level protocols can be thought of as subprotocols to TCP/IP. Conceptually, all high-level communication protocols work the same way -- it's just the rules and formats that change. Some protocols transmit data in formats that are relatively easy for human eyes to read (like HTTP), while others transmit data in a compressed or encrypted format that's essentially unreadable to human eyes (like HTTPS).

Let's take a closer look at HTTP as an example of a high-level protocol. HTTP essentially consists of a format to request files, a format for responses to those requests, and a format for header files. HTTP requests (which look like Listing 1) typically ask that a file (of type HTML, GIF, or JPEG, for example) be sent over the Internet to the program requesting it, usually a Web browser.

"GET /solutions/papers/whitepapers.jsp HTTP/1.1\r\n"
"Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, applicat"
"ion/vnd.ms-powerpoint, application/vnd.ms-excel, application/msword, */"
"*\r\n"
"Referer: http://www.noblestar.com/" 
"Accept-Language: en-us\r\n"
"Accept-Encoding: gzip, deflate\r\n"
"User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)\r\n"
"Host: www.wmata.com\r\n"
"Connection: Keep-Alive\r\n"
"\r\n";

For each request, the server may send back one or more responses. The response includes a text header (as exemplified in Listing 2) and the requested data (which might look something like Listing 3).

"HTTP/1.1 200 OK\r\n"
"Server: Netscape-Enterprise/4.1\r\n"
"Date: Sat, 18 Jan 2003 21:03:51 GMT\r\n"
"Content-type: text/html; charset=Cp1252\r\n"
"Set-cookie: NSES40Session=e0f%253A3e29bba2%253A224c54f45191a4ac;path=/;"
"expires=Sat, 18-Jan-2003 21:33:51 GMT\r\n"
"Transfer-Encoding: chunked\r\n"
"\r\n"

"<HTML>\r\n"
"<HEAD>\r\n"
"<TITLE>Solutions - White Papers</TITLE>\r\n"
"<LINK REL=STYLESHEET TYPE=\"text/css\" href=\"/css/main.css\">\r\n"
"<SCRIPT LANGUAGE=\"JavaScript\" src=\"/js/menu.js\"></SCRIPT>\r\n"
"<SCRIPT LANGUAGE=\"JavaScript\" src=\"/js/topmenu.js\"></SCRIPT>\r\n"
"</HEAD>\r\n"
"<BODY BACKGROUND=\"/images/back_pat.gif\" TOPMARGIN=0 LEFTMARGIN=0 RIGH"
"TMARGIN=0 MARGINHEIGHT=0 MARGINWIDTH=0>\r\n"
"<TABLE border="0" CELLSPACING=0 CELLPADDING=0>\r\n"
" <TR BGCOLOR=\"#000066\" VALIGN=TOP>\r\n" 
"  <TD COLSPAN=7><A href=\"/index.jsp\"\r\n"
"  onMouseOver=\"clearMain()\" onMouseOut=\"reset()\" onClick=\"set(''"
"  ,0)\"><img\r\n"
. . .

Back to top

Supported and unsupported protocols

So you may be wondering, if TestStudio VU scripts capture traffic at the communication protocol level, what makes a particular protocol either "supported" or "unsupported"? The support that's being referred to is script generation support for playback. Just capturing the traffic isn't enough to be able to play back that traffic in a way that simulates real users.

If the application you're testing uses one of the supported protocols, TestStudio will generate scripts that can be played back more or less as is. (I say "more or less" because recorded scripts often do require some editing prior to multiuser playback, as discussed in previous articles in this series, but this editing has nothing to do with whether the protocol is supported by the tool.) On the other hand, if a protocol is unsupported it doesn't mean that TestStudio VU scripts can't be used to test the performance of an application using the protocol. It just means that additional considerations apply, which we'll be exploring in a moment.

Here are the communication protocols that TestStudio officially supports:

• TCP/IP Socket
• HTTP / HTTPS
• DBLIB
• DCOM
• IIOP
• Jolt
• ODBC
• Oracle
• SQL Server
• Sybase
• Tuxedo

TCP/IP Socket is, in a sense, a "superprotocol" to the others in the list and can be used to capture scripts that with some modification will handle unsupported protocols, as discussed in the next sections.

Every communication protocol that's not on the preceding list is considered to be unsupported by TestStudio. Some common examples of unsupported protocols are as follows:

• FTP (File Transfer Protocol)
• SNMP (Simple Network Messaging Protocol)
• SMTP (Simple Mail Transfer Protocol)
• POP3 (Post Office Protocol 3)
• WAP (Wireless Application Protocol)
• VoIP (Voice over Internet Protocol)

All of these protocols are TCP/IP protocols, as HTTP is. Any TCP/IP-type communication can be captured in TestStudio, provided the correct session recording options are set. I'll show you how to do that and then explain the modifications you'll need to make to scripts captured in an unsupported protocol format.

Back to top

Configuring the recording options

You'll generally want to configure the session recording options to capture TCP/IP Socket traffic, for one of two reasons:

• to determine what protocol a particular application is using and whether it's supported (or whether you own the proper license for the protocol)
• to capture traffic from unsupported protocols

You'll see later that scripts recorded using the TCP/IP Socket recording options will look different from HTTP scripts. But remember -- all of these protocols are just different ways to format requests and responses based on the applications and types of data being sent.

To configure the session recording options to capture TCP/IP Socket traffic, in Rational Robot choose Tools > Session Record Options from the menu bar. Select "Network recorder" on the Method tab (as shown in Figure 1) and "Manual Filtering" on the Generator Filtering tab (as shown in Figure 2).

Figure 1: "Network recorder" selected

Figure 2: "Manual Filtering" selected

While on the Generator Filtering tab, click the Advanced button and ensure that both "Well-known protocols" and "Unrecognized protocols" are checked under "Socket" (see Figure 3).

Figure 3: "Well-known protocols" and "Unrecognized protocols" selected

If you know the name or IP of the server that you'll be recording against and you're certain that all of the traffic that you're interested in will be from that server, you can specify that server to ensure that "extra" network traffic isn't unintentionally captured. Go to the Method: Network tab (Figure 4), click the Manage Computers button, and fill in the appropriate information. Then on the Method: Network tab select the server that you've added.

Figure 4: Server selected on the Method: Network tab

Now when you record a script you'll be presented with a window that shows you what protocols have been captured (see Figure 5), and you'll be able to decide which ones you want included in your script.

Figure 5: Manual Filtering protocol selection window

In Figure 5 you see several options. In this case I was recording against an instant messenger program that uses a combination of HTTP and an unsupported protocol (shown as "Socket"). If the recorded traffic uses any of the supported protocols, they'll show up in this window as the HTTP traffic did. "Socket" will always be an option, since the supported protocols are subsets of TCP/IP. When you see only "Socket," you know that you're working with an unsupported protocol. (You'll often also see "Reserved" or "Null." This is almost always traffic related to your network and not what you're looking for.) All you have to do here is click the red "P" next to the name of a protocol and select either "Include" or "Exclude" in the "Filtering" box to indicate which protocols you want captured in your script.

Back to top

Modifying scripts that use unsupported protocols

Once you've configured the session recording settings and recorded your first script, you'll see that the communication you've captured falls into one of two categories: it uses either "clear text" or "encoded" protocols. (These terms are my own and I put them in quotes because they're not always technically correct, but they're descriptively accurate.) The way you'll work with your script depends on which category it falls into, so your first task is to determine this.

In a script that uses a "clear text" protocol, you can essentially read what you recorded. For instance, Listing 4 shows a segment of SMTP traffic captured in a script.

#include <VU.h> 
int n;
{
Dmail_yahoo_com = sock_connect("SMTP001", "mail.yahoo.com:25");

set Server_connection = Dmail_yahoo_com;
n = sock_isinput();
sock_nrecv \\["SMTP002"\\] n;

sock_send "helo john.doe.com\r\n";
n = sock_isinput();
sock_nrecv \\["SMTP003"\\] n;
sock_send "mail from: sbarber@noblestar.com\r\n";
n = sock_isinput();
sock_nrecv \\["SMTP006"\\] n;
sock_send "rcpt to: cwalters@noblestar.com\r\n";
n = sock_isinput();
sock_nrecv \\["SMTP008"\\] n;
. . .

You can easily see that the mail message recorded in this script segment was sent from "sbarber" (me) to "cwalters" (my officemate). While you may not know all of the details of the SMTP protocol, you can still recognize much of the text.

Listing 5 shows a segment of a script that uses an "encoded" protocol. This script segment was recorded against a client/server application that was transmitting a short statement in a comment field. You'll see that this script segment is completely unreadable by human eyes.

sock_send

"`14030000010116030000`8~Pv\b`d3`pB`80aa31c3865e8ea9bf`(7`c4f0d0`&MCl`8787"
   "fb56f0a9ae36f632d9de548ae7dd29ed72ff55e5`;@\tDu`952bacc8170300017de03dde"
   "173a9620fb`;C`d941e07011`i1`a2dbb05db693dcb9f3df35fbe82f141c9ca65c90cc7e"
"08a3a7509024f775842089b3`Ul`c2`%lO`0e152d886de3d028d609`~wx`d7222e04e689"

"bfa64ce897`+.`eb5d17`&c`e379db44da`!G\fvc\\]`d0240cc5f2c520fdef`F4`0753b0aa"
   "a81a96dac3d49764fbe47d7f4202`_#`19cd7683146af4ec`c{`00a2fa`AJ`5c31b61d18"
   "34ebbe0f`_b`c49a7a10f3600297a59a7eebbe389bbad9a88fb137c9e9ecea`_z`01db11"
"ab3ceb89`p^h\f`d79085d8740de756052293a2fbdb7afcb8a43381dd3193b8`K3`e414`u"
   "Q`f0ea`=_`8c`k?\\[,`dc`t.`048d2fc9cf741a7f08a6bfd129bff74f0dc9efe6ab6bcd24"
   "d3c4e42d007f`S,f$,;`c5e0b2ae0ef01ae13b1673b70ccc`*5`aa96b73ccaf8`a;`d9`s"
"H`8245c5002cb4859396ad10a8de09cde283160a0c4b86fbf3a194024d8a`d/`0bb63bf9"
"`#oB`9a1bf78adf`c*`d7b6eccae9d3c1c411f0b077841fe5cbd9`L69`9ad497`WiW`b49d"
   "4d870d0cee618d`o";

sock_nrecv \\["test006"\\] 520;

Regardless of which type of protocol the script you've captured uses, you'll need to modify only the part of the script that's between the quotes ("") in the sock_send block to represent variations between users. It's generally safe to leave everything else unchanged, since TestStudio handles the protocol headers (and therefore socket connections) automatically regardless of the data that's being transmitted. In Listing 6, for instance, the commands in bold (which are similar to the HTTP commands you're used to except with sock substituted for http) create the connection to the server, set the connection, transmit the data, receive the response, and then release the connection. These commands would be left unchanged, and modifications would only be made to the section shown as "somekindofcontent". The types of modifications that you may want to make to the sock_send block are discussed next.

							hostname_1 = sock_connect("test004", "hostname:port");
set Server_connection = hostname_1;
sock_send
"somekindofcontent";
sock_nrecv \\["test005"\\] 146;
sock_disconnect(hostname_1);

Tweaking scripts that use "Clear Text" protocols

Making the necessary modifications to scripts that use unsupported "clear text" protocols is really pretty straightforward. Because TestStudio will handle the manipulation of the socket connections for you, all you have to do is manipulate the parameters you may want to vary (or datapool). These will be the same types of parameters that you're used to working with for HTTP traffic -- user names, account numbers, addresses, and such. For example, Listing 7 shows a version of the script segment from Listing 4 that's been modified to send the mail message to a variety of recipients as entered into a datapool, instead of always sending it to "cwalters."

#include <VU.h>
int n;
{

DP1 = datapool_open("smtp_datapool");
datapool_fetch(DP1);

Dmail_yahoo_com = sock_connect("SMTP001", "mail.yahoo.com:25");

set Server_connection = Dmail_yahoo_com;
n = sock_isinput();
sock_nrecv \\["SMTP002"\\] n;

sock_send "helo john.doe.com\r\n";
n = sock_isinput();
sock_nrecv \\["SMTP003"\\] n;
sock_send "mail from: sbarber@noblestar.com\r\n";
n = sock_isinput();
sock_nrecv \\["SMTP006"\\] n;
sock_send "rcpt to: "
+ http_url_encode(datapool_value(DP1, "ename"))+
"@noblestar.com\r\n"; 
n = sock_isinput();
sock_nrecv \\["SMTP008"\\] n;
. . .

DATAPOOL_CONFIG " smtp_datapool " OVERRIDE DP_SEQUENTIAL DP_SHARED DP_NOWRAP
{
   INCLUDE, "ename", "string", "cwalters";
}
						

It's also possible to build custom functions for relatively simple protocols like this one, but detailing that process is beyond the scope of this article. For an example of custom SMTP functions that allow you to build SMTP scripts without even recording, see "Supporting Unsupported Protocols: SMTP, POP3, VoIP," the PowerPoint slide show that Chris Walters and I presented at RUC 2002.

Tweaking scripts that use "encoded" protocols

Modifying the sock_send block in scripts that use unsupported "encoded" protocols is more complicated. The key to working with them is identifying two things:

• what changes between users
• what series of characters represents the changes between users

For instance, say you record a script that looks like Listing 5 above. Further say that you determine by recording it several times, each time logging in as a different user, and comparing the resulting scripts that the section shown in bold in Listing 8 changes every time and represents the user name in hexadecimal format. What you'll need to do is create a datapool with a listing of all of the user names you want to use, in hexadecimal format, and then substitute a datapool value for that section of the sock_send block in the same way as we discussed for "clear text" protocols.

sock_send
"`14030000010116030000`8~Pv\b`d3`pB`80aa31c3865e8ea9bf`(7`c4f0d0`&MCl`8787"
"fb56f0a9ae36f632d9de548ae7dd29ed72ff55e5`;@\tDu`952bacc8170300017de03dde"
"173a9620fb`;C`d941e07011`i1`a2dbb05db693dcb9f3df35fbe82f141c9ca65c90cc7e"
"08a3a7509024f775842089b3`Ul`c2`%lO`0e152d886de3d028d609`~wx`d7222e04e689"
. . .
"4d870d0cee618d`o";
sock_nrecv \\["test006"\\] 520;

It's rarely an easy process to find all the things that change from one recording to the next, but I've found that the following way of detecting these differences and handling them programmatically works very well:

1. Record the script that you'll ultimately want to play back (script A).
2. Record that script again, exactly the same way you did before (script B).
3. Record that identical script yet again (script C). (This step is often unnecessary, but there's no way to know until after you determine the pattern of differences among scripts. It's always safer to compare three scripts initially to be sure that you've identified the entire pattern.)
4. Compare scripts A, B, and C to identify all of the differences in between the quotes ("") in the sock_send block.
5. Parameterize those differences from the _response file. (See Part 11 and Part 12 for more information on capturing return data from the _response file using regular expressions.)
6. Record the script again (script D), this time changing any transactional data that will differ between users (user name, password, account number, and the like).
7. Compare scripts A and D and identify all of the differences in between the quotes ("") in the sock_send block that weren't identified in Step 4.
8. Add datapools to handle the varied data identified in Step 7.

Since it may not be easy to determine what "encoded" characters represent different user names, passwords, and so on, you may need to record several scripts, changing a single parameter each time, to determine the encryption or simply capture all of the different options you want to datapool. This can be a very tedious process, particularly if you're comparing the scripts manually.

One thing that makes this comparison process easier for a known, or documented, protocol is referring to the Request For Comment (RFC) for that protocol. Any protocol that isn't completely custom or proprietary has an RFC on the Internet Engineering Task Force (IETF) home page. Refer to this RFC for the exact specification describing what different characters and positions in the requests and responses represent. I've also found it helpful to use two quality tools that automatically identify differences between two similar files: CSDiff and ExamDiff. I actually recommend using both, because their strengths are complementary. Figures 6 and 7 shows screens from both comparing the same section of code.

Figure 6: A screen from CSDiff

Figure 7: A screen from ExamDiff

Back to top

Now you try it

Every protocol varies so drastically that there's really no way to create a representative example or exercise. On top of that, I don't have access to any application that uses an unsupported protocol -- either "clear text" or "encoded" -- that you would also have access to. So the best ways to practice what we've discussed in this article are the following:

• For "clear text" unsupported protocols, open an Internet mail account like Hotmail or Yahoo and then record and edit scripts captured while sending mail to yourself. Ensure that you don't simulate more than a single user during playback.
• For "encoded" unsupported protocols, record, edit, and play back scripts against any SSL-enabled Web site (that is, one with a URL that begins with "https" instead of "http"). If you use the settings detailed earlier in this article for recording, SSL scripts will look and act very much like they've been captured in an unsupported "encoded" protocol. Just make sure that if HTTP is one of your manual filtering options, it's excluded upon script generation.

Back to top

Wrapping it up and looking ahead

In this, the final installment of the "User experience, not metrics" series, you've learned that just because an application uses a protocol that's unsupported doesn't mean that IBM® Rational Suite® TestStudio® can't be used for performance testing. While scripting unsupported protocols will likely take a little more time, it can be done if you properly configure recording settings and modify recorded scripts. When you find the first performance-related defect, you'll surely recognize that spending the extra time and effort is justified.

Over the course of thirteen articles, we've been exploring together some of the issues related to predicting and optimizing the most important performance feature of an application, user experience. Along the way, we've talked about user community modeling, capturing meaningful times, creating valuable reports, and even some advanced scripting topics. As you probably realize, there's a lot left to say under the broad heading of performance engineering.

Many readers have contacted me directly over the past year to discuss the articles as well as other related topics. I've run into others on the forums at the Rational Developer Network as well as on QAForums.com. I've enjoyed talking with all of you, and assure you that I've learned a tremendous amount from those conversations. Writing this series and building relationships with its readers has been personally rewarding to me.

Because there's so much more to be said and I've really enjoyed writing this series, and due also to the positive feedback that both Rational and I receive about the series, we've just finalized plans for a follow-on series called "Beyond performance testing." This fourteen-part series will focus mainly on the analysis that occurs after your initial scripts have been recorded, edited, and executed. I'm looking forward to writing this series as well as participating in continued conversations with all of you. See ya' in Part 1 of Beyond performance testing!

Back to top

References

• "Introduction to TCP/IP" (Yale University Web site)
• "Supporting Unsupported Protocols: SMTP, POP3, VoIP," a PowerPoint slide show presented by Scott Barber and Chris Walters at RUC 2002
• Internet Engineering Task Force (IETF) home page, the site to visit in order to link to RFCs for any protocol that isn't custom or proprietary
• CSDiff, a file comparison tool for Windows that you can download free (ComponentSoftware Inc. Web site)
• ExamDiff, a visual file comparison tool for Windows that you can download free (PrestoSoft Web site)

About the author

Comments

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• A Primer on protocols
• Supported and unsupported protocols
• Configuring the recording options
• Modifying scripts that use unsupported protocols
• Now you try it
• Wrapping it up and looking ahead
• References
• About the author
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

My developerWorks community

Dig deeper into Rational on developerWorks

Rate a product. Write a review.

Write a review for Rational products today!

Special offers