Our mission

We were charged with profiling and improving the performance of IBM WebSphere Transcoding Publisher, which interacts with HTTP clients and HTTP servers as a standard HTTP proxy server. Instead of simply proxying the information, Transcoding Publisher tailors the content for the requesting device so that the data that it passes along suits the preferences of the requesting browser. The HTTP response is tailored according to a set of preferences based on the user, the device, and on the type of network that the request came from. Transcoding Publisher is a powerful product and requires several sets of performance data.

We started with an initial matrix of three network types and five device types supported by Transcoding Publisher, which we sketched out as:

For fun, we threw in another set of variables: Transcoding Publisher ships on six different platforms and runs with multiprocessor SMP support. You could sketch that out as:

We found ourselves with a three-dimensional matrix before we even took into account that Transcoding Publisher runs in two major configurations: it can run as a proxy server and as a content filtering servlet within IBM WebSphere Application Server.

In addition, we had to consider that there were a variety of different transcoding engines that might or might not be active. For example, if the user does not have any i-mode telephones as clients, there is no reason to install and configure the HTML-to-i-mode transcoder. We needed to test a variety of configurations.

To come up with a good initial combination, we chose to run sets of tests against two Windows NT machines, one with a uniprocessor and one with a two-way multiprocessor. For our UNIX testing, we chose to use two RS/6000 machines, one with a uniprocessor and one with a four-way multiprocessor.

We came up with several sets of devices and a standard set of Web pages to retrieve. We stored the Web pages on a local HTTP server, and isolated our test machines on a private LAN. This prevented any extraneous traffic from affecting our numbers and kept our LAN administrators from sending nasty e-mail to our manager. However, it also challenged us -- how would we remotely administer the test machines? We needed a way to get the machines to configure themselves.

Back to top

Our solution

Building a set of client/server tools that cooperated to stop, configure, and restart our Transcoding Publisher servers and our HTTP clients offered us the most labor-saving solution.

We used Mindcraft's WebStone software (see Resources for download information) to drive HTTP GET requests to simulate real-world server traffic. WebStone is an open-source benchmarking tool, which we modified to obtain real Web pages. The standard WebStone test obtains Web pages that are gibberish, but Transcoding Publisher needs real pages to do useful work.

In this article, we present the server side of our automation suite and include a skeleton of the client side (see Resources).

The Transcoding Publisher server is written in Java and runs on Windows NT, Windows 2000, AS/400, AIX, Solaris, and Linux. It runs as a Windows Service on the Windows platforms, and is invoked with shell scripts on the UNIX platforms. The logic that we used to stop and start the services on the various platforms is applicable to any similarly invoked application. We provide sample code in this article to show how our methodology can be applied to other test environments.

So, what did we write?

• A Perl server program that stops the Transcoding Publisher server, reconfigures it, and restarts it
• A Perl client test driver that sends commands to the Perl server program, and locally invokes the WebStone master

You can download the code for the server and for the client.

Figure 1 shows the test configuration and the interaction among the Webstone master and slaves, the Transcoding Publisher server, the Perl server, and the Web servers.

1. The WebStone master determines what test it is going to run based on user input. It sets up the files that WebStone uses for the test case. It then sends a message to the Perl server program indicating the plug-ins that need to be enabled. The sample Perl client script included with this article does not show all of this code (see Resources).
2. The Perl server stops Transcoding Publisher and writes the Transcoding Publisher files to enable the specified plug-ins. It restarts Transcoding Publisher using the new configuration.
3. The Perl client invokes the WebStone master. The WebStone master reads the files that instruct it what to do. The WebStone master distributes that test case information to the WebStone slaves.
4. The WebStone slaves obtain the specified pages through the Transcoding Publisher proxy.
5. The Transcoding Publisher proxy obtains the pages from the web servers.
6. The Web servers return the pages to the Transcoding Publisher proxy.
7. The pages are transcoded (for example, changed from HTML to WML).
8. The transformed pages are returned to the WebStone slaves. The process from step 4 to 8 is repeated for a period of time. Performance statistics are gathered during this processing. At the end of the period of time, the statistics are gathered by the WebStone master and put in a report.

Let's focus on the Perl server program. Everything in the UNIX version is standard, while the Windows version uses the Win32::Service module to interact with the Windows services directly.

Figure 2 shows the basic tasks that the Perl server needs to accomplish.

Open a socket and wait for connections
First we had to set up the link that we would use to communicate to the outside world. Here's the code we used:

   $PORT = 9000;                  # pick something not in use

   $server = IO::Socket::INET->new( Proto     => 'tcp',
                                    LocalPort => $PORT,
                                    Listen    => SOMAXCONN,
                                    Reuse     => 1);

   die "can't setup server" unless $server;
   print "[Server $0 accepting clients on port $PORT]\n";
   while ($client = $server->accept()) {
        

This opens a reusable TCP/IP socket on port 9000. The while loop accepts connections as long as the server is running, but only one at a time.

Once a socket connection has been made, listen for commands
The command is broken into a directive and an option.

while ($client = $server->accept()) {
   printf "[Connect from %s]\n", $hostinfo->name || $client->peerhost;
   while ( <$client> ) {
     print $_;
     next unless /\S/;       # skip blank lines

     chomp();
     #split $_ on whitespace into 2 pieces
     ($directive,$service)=split(/\s+/, $_, 2); 
     print "Directive ->$directive\n";
     print "Service   ->$service\n";        
        

Now we're getting somewhere. We've gotten a connection, and by using while ( ), we read input from that socket one line at a time, just as if it were a standard Perl file handle. Because we are expecting the input received to be commands that might have an argument attached, we split the input line into a directive and the rest. In this version, the argument will always be the shortname of a Windows NT service. The shortname can have spaces in it, so we don't break the line up further.

On Windows NT we can use our server to stop and restart any registered Windows service. Perl's Win32::Service module has many useful features for manipulating services. To stop a service, you have to know its shortname. These shortnames can have whitespace in them, so it is important to leave the arguments sent to the server intact.

If a recognized command is received, handle that command. Otherwise, skip this line.
We make our logic decisions with rudimentary regular expressions. Because we will be writing all the clients that will interact with this server, we can make it perform any way that we want. But, for the sake of intuition and uniformity across platforms, we have chosen to keep things simple.

     if    (/quit|exit/i) {    # if the client sends "quit" or "exit" close
           last;               # the socket and wait for more connections
     }
     elsif (/stop/)         {
        $success = &stopService("",$service);
        print "$service successfully STOPPED\n" if ($success);
     }
     elsif (/start/) {
        $success = &startService("",$service);
        print "$service successfully STARTED.\n" if $success != 0;
     }

     elsif (/proxybackbone/) {  # backbone is a WTP configuration that has
                                # no text or image transcoding active
        &proxy("","backbone");  # &proxy is discussed below 
     }
     elsif (/proxytext/) {      # act as a proxy with text transformations
        &proxy("","text");
     }
     elsif (/proxyboth/) {      # act as a proxy with text and image
        &proxy("","both");      # transformations
     }
     elsif (/proxyimage/) {     # only image transformations
        &proxy("","image"); 
     }
   
     elsif (/servletbackbone/) {   # run under websphere but no transforms
        &servlet("","backbone");   # $servlet is discussed below 
    }
     elsif (/servlettext/) {       # under WebSphere with text transforms
        &servlet("","text");
     }
     elsif (/servletboth/) {       # under WebSphere with text and image 
        &servlet("","both");       # transforms
     }
     elsif (/servletimage/) {      # under WebSphere with only image
        &servlet("","image");      # transforms
     }
   
   } continue {
   }
        

If exit or quit are received, we fall out of the while loop, close the socket, and wait for connections again.

Sending stop or start will call stopService or startService respectively. Both of these functions expect an argument that is the shortname of the Windows NT service that they are supposed to stop or start. This could be extended to work on other platforms, but the stopService and startService functions would need additional logic added. For our Unix versions of this server, we took this basic code and changed the lines using Win32::Service::StopService to search for and kill the process IDs of the services we needed to stop.

The Transcoding Publisher server can be configured to run as a proxy server or as a servlet running under WebSphere Application Server. In each of these configurations, we ran four sets of tests with different combinations of plug-ins enabled. We had commands for each of the eight major configurations; each command called the appropriate function to start the proxy or the servlet with the appropriate plug-ins enabled. This could easily be extended to work with other servers.

When we received the command to start Transcoding Publisher with a certain combination of plug-ins enabled, we called a method similar to the following:

sub proxy {
   my($hostname,$plugins) = @_;
   
   # stop the service if it is running as a proxy
   print "Stopping $transpubServiceName\n";
   $success = &stopService("",$transpubServiceName);
   print "$transpubServiceName successfully STOPPED\n\n" if ($success);
   
   # stop websphere to also stop the servlet version 
   print "Stopping $websphereServiceName\n";
   $success = &stopService("",$websphereServiceName);
   print "$websphereServiceName successfully STOPPED\n\n" if ($success);

   # Make certain we're in proxy model 
   modelSwap("proxy");

   # Set the requested plugins, this is a wtp configuration file
   editRegistry("proxy",$plugins);

   # Start the proxy
   print "Starting $transpubServiceName\n";
   $success = &startService("",$transpubServiceName);
   print "$transpubServiceName successfully STARTED.\n" if $success != 0;
}        
        

This method has code to stop both the Transcoding Publisher service and the WebSphere Application Server. Though they aren't likely to be running at the same time, we had to make sure that we stopped Transcoding Publisher regardless of which model it was running under.

After the servers are stopped, we call two methods that reconfigure the Transcoding Publisher server. This is where the code can be modified to suit your software's particular needs.

Once the configuration files are updated, we can restart the server according to which function was called.

Back to top

The full implementation

We have shown the functions required for starting, stopping, and reconfiguring the Transcoding Publisher proxy.

In our test lab setup, we have three WebStone servers, one of which is designated as the master. The master has another Perl program, which controls all the tests that we want to run. Because we run tests in batches, the Perl program ensures that the Transcoding Publisher server is configured correctly for each test. To do this, the Perl program running on the WebStone master sends commands to the Perl reconfiguration program described in this paper. After the reconfiguration program has finished, the Perl program running on the WebStone master instructs WebStone to start the first set of tests.

After the first set of tests is complete, the Perl program running on the WebStone master runs through the same set of steps and runs the next set of tests. When the entire set of tests is complete, the Perl program running on the WebStone master gathers all the data from the runs and collates it into a spreadsheet format.

This all occurs without any human intervention. You get results quickly because there is no down time between tests. (Of course, there is no guarantee that you'll meet your performance goals.) Plus, the complete end-to-end automation gives the testers a chance to chill out while the computers do the hard work.

Resources

• Download the sample Perl server script and sample Perl client script.
  
  
• Mindcraft's WebStone software is an open source benchmarking tool.
  
  
• ActiveState ActivePerl can be used to run Perl programs on the Windows platforms.
  
  
• Learn more about WebSphere Transcoding Publisher at IBM Pervasive Computing.
  
  
• Browse the WebSphere Transcoding Publisher library for a collection of information and whitepapers about WebSphere Transcoding Publisher.
  
  

About the authors

Rate this article

Comments

Back to top

Table of contents

• Our mission
• Our solution
• The full implementation
• Resources
• About the authors
• Comments

Next steps from IBM

Rational products offer a complete collaborative lifecycle management set of tools for the IBM i platform for existing and new developers.

---

• Try: Download a free trial of Rational Developer for Power Systems, which provides an integrated development environment across the major platforms.
• Buy: Rational Developer for Power Systems