Level: Intermediate Brian Goodman (bgoodman@us.ibm.com), Advisory Software Engineer, Certified IT Architect, IBM
James Kebinger (kebinger@us.ibm.com), Staff Software Engineer, IBM
19 Oct 2004 High volume Web sites often require asynchronous or threaded operations to achieve target performance criteria. While threads in Web containers are considered bad practice, the alternative is for developers to make blocking calls to code they cannot control. It becomes important that dependencies of this nature fail-fast. Goodman and Kebinger present a homegrown short-circuit pattern that ensures threaded execution and completion of a process in a fixed window of time.
Of the many aspects to performance, perhaps the most obvious to the user is the time it takes for an action to generate a response. In general terms, responses that take less than 250 milliseconds are perceived as instantaneous. In local fat-client applications, tasks in the several-second range can manage the delay with a busy cursor -- for anything longer than that, use a progress bar.
Web applications do not have the luxury of being able to handle delays in the user interface. They generally must complete all server-side processing before displaying content to the user; then there's the overhead from transferring and rendering the response. These constraints make it important that the back-end processing be fast and predictably so, because a slow Web page might as well be a dead page to the user.
This article focuses on the short-circuit pattern, a technique to uniformly encapsulate potentially long-running tasks. While it can work anywhere, it was developed for use in a Web application.
When to use the short-circuit pattern
The key to applying patterns successfully is to identify a situation that requires the reuse of a known solution. In the case of the short-circuit pattern, the most obvious scenarios to leverage this solution are I/O-related transactions.
Moreover, a situation in which the task has a known pattern of behavior is a pre-requisite (for example, all transactions terminate in less than one second). This pattern will not make tasks faster, but it will help to handle the edge cases. Tasks with longer lives are inappropriate for this pattern -- you run the risk of supporting the resource consumption of threads waiting to terminate.
Prime candidates for this type of time-boxed execution include:
- Socket or URL connection activity
- Database interactions
- Cumbersome file I/O operations
These I/O operations traditionally complete within a short amount of time; however, as with high volume Web sites, the need to govern this threshold tightly outweighs the need to complete the given task. Using the short-circuit pattern, you can govern these issues appropriately and repeatedly. In the case where the task times out, the application can either display an error message or show cached results.
Dissect the short-circuit pattern
Next, we'll provide a roadmap to the short-circuit pattern.
Define tasks
The pattern supplies a framework in which the application can execute specific processing tasks in a time-bounded way, abstracting away the complications of threads and synchronization from the developer. To use the framework, the developer extends an abstract class and provides implementations of two abstract methods:
- The first is the
doTask() method, the implementation of the task to be executed.
- The second is the
getData() method, the way the results of the task are retrieved.
- An optional third method,
cleanup(), allows for control over how a task's resources are disposed of.
The doTask() method is similar to the run() method on any Runnable in the Java environment. The program does its work as it would in any other thread. Just as in a thread, the code frequently checks to see if it should continue to run. The isTaskComplete() method is provided for this purpose. This prevents running threads from accumulating on the server, computing results that no one is waiting to view.
The getData() method is the way results are returned to the caller. It should be a non-blocking call implemented to return an object (or null) that represents the current data generated by the task. The data might potentially be a partial result without risk because the higher level getResult() method provides a status code with the data, guiding consumers as to whether or not the result is likely to be valid.
Execute tasks
Now that the task is plugged into the framework, you can execute it in a couple of ways. The first and easiest approach is to create an instance of your new class and call the inherited method execute() in blocking mode. The calling thread blocks until the task completes or the timeout value provided as an argument has lapsed. The return value is an instance of TaskResult containing a status code indicating if the task has executed to completion, along with the result of the operation.
The other way to use the pattern is to call execute() in non-blocking mode. The method call then returns null instantly, leaving the calling program to do other work. The results are retrieved either by calling getResult(), which blocks until the task either is completed or expired or by registering a listener to be notified when the task has completed or expired.
A simple example: Scrape the Web
In this example, a task that retrieves the contents of Web sites is defined. Because the time this operation takes may be unpredictable, it is executing in the short-circuit framework. First the task operation itself is defined (see Listing 1).
Listing 1. Scrape the Web doTask() method
public void doTask() {
BufferedReader br = null;
try {
URLConnection conn = new URL(url).openConnection();
br = new BufferedReader( new InputStreamReader(conn.getInputStream()));
String line;
/* very inefficient way to read HTML from a URL! */
while ((line = br.readLine()) != null && !isTaskComplete()) {
// check every loop to see if the deadline has elapsed.
buff.append(line);
try {
Thread.sleep(DELAY); // fake a delay
} catch (InterruptedException e) {
// do nothing
}
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (br != null)
br.close();
} catch (IOException e) {
//Do nothing
}
}
}
|
In Listing 2, we define a simple way to retrieve the data.
Listing 2. Scrape the Web getData() method
public Object getData() {
if (buff == null) {
return ("");
} else {
return buff.toString();
}
}
|
This class, URLTask, is then used as in Listing 3. (See Resources to download the full listing.)
Listing 3. Scrape the Web using the code
public static void main(String[] args) {
long ttl = 2000; // adjust up and down depending on results
System.out.println("Getting data...");
URLTask task = new URLTask("http://www.ibm.com");
TaskResult result = task.execute(ttl, true);
if (result.getState() == TaskState.TASK_SUCCESS) {
System.out.println("Got latest data"); // do processing
} else {
System.out.println("Return cached data or error message!");
}
}
|
An example scenario
Let's paint a scenario of the task. The Widgets International company just added a third-party expense report solution to its portal. Currently, employees log in to the portal to obtain a personalized portal experience and access to critical business applications. Because this is the primary Web presence in the company, most employees only have to log in once to access all the features of the portal.
The new expense account software is a Java applet that requires token-based authentication -- a system different from the one the company is already using. To mitigate increased cost and custom development, Widgets International decided to leverage the third-party authentication servers that generate the tokens based on the common employee ID and password. These tokens are set as part of a session cookie and when passed to the applet as part of the portal experience, allow employees authenticated access to the third-party system.
Figure 1 shows the basic components for Widgets International's deployment of its third-party expense account applet.
Figure 1. Widgets International's high level expense account architecture
The initial transaction with the browser and portal server requires authentication. The authentication is based on the employee's common ID and password. You might simplify this step by using the third-party authentication server as the corporate authentication system, but in the real world that introduces a dependency on third-party code, as well as on code changes to existing systems.
To minimize change, a second authentication is made to the third-party authentication system that is simply a protected servlet that generates tokens using an employee's ID and password or verifies that an ID passed in through CGI arguments is equal to the ID signed in the token. The portal returns HTML with the authentication token as part of the initialization parameters for the Java applet. The Java applet passes the token and the ID of the user to the third-party authentication server and verifies the ID.
Next, we will apply the short-circuit pattern to the connection from the portal server to the third-party authentication server.
Widgets International short-circuited
Widgets International's current architecture requires that an HTTPS connection has to occur securely to retrieve a token generated by a third-party authentication server. The third-party authentication server is a servlet accepting an employee's ID and password, returning the token as a successful response. The time-boxed task is this URL connection.
At this point, most readers ask "Why not use the timeout settings implemented by most HTTP connections?" When creating a socket connection, initial handshakes and lower level blocking often make it impossible to have control to abort the transaction. Under heavy load, a servlet using basic timeout settings will find inconsistent results on thread-termination response.
The main portal server creates the token-generation task. The token generation task POSTs the employee's ID and password to the third-party authentication server to retrieve the token for the user. The portal creates a TimeLimitedTask object, calls doTask() to set the expire time and then calls getData(). The task executes and when it completes or runs longer than the specified time-box, the code returns to the portal's execution. In the case of Widgets International, failure to get the token simply forces the employee to log in again if they use the expense account applet. Success allows the portlet to write out the HTML surrounding the applet, enabling seamless single sign-on.
Framework implementation
The framework consists of four classes and an interface, as you can see in the following diagram.
Figure 2. Basic class diagram of short-circuit framework and pattern
The brains of the operation lie in the TimeLimitedTask object. It uses threads to run the tasks, so a producer-consumer mechanism is used to communicate between threads. An easy way to think of it is that two producers, the task and a timeout, race to provide results to the consumer.
When a task is executed a sequence of operations happens, with the execute() method shown in Listing 4. First a timer is created and set to timeout with the passed timeout interval. Next a thread is started to run the task in the background. Then the method returns as null if not blocking, otherwise it calls the blocking method getResult.
Listing 4. The execute() method
/**
* Kicks off the task
* @param block true if caller wants to block on the completion
* or expiration of a task.
* @return null if block is false, the Result of the task if block is true.
*/
public final TaskResult execute(long timeLimit, boolean block)
{
Thread theTaskThread = new Thread() {
public void run() {
doTaskWrapper();
}
};
resetState();
expirer = new TaskExpirer();
getTimer().schedule(expirer,timeLimit);
theTaskThread.start();
if (block){
return getResult();
}
else {
return null;
}
}
|
The task now does one of two things. Either it runs on time and returns, or the timer fires before the task is completed and incomplete results are returned.
The code is similar for both code paths. When the task takes too long, the timer is executed; this sets the state to "expired" and calls a method taskDone(). When a task completes on time, it sets the state to "success" and calls the same taskDone() method. As shown in Listing 5, the taskDone() method wakes up a thread that might wait on an internal synchronization object, notifies any registered listeners, then executes the cleanUp() method.
Listing 5. The taskDone() method: Cleaning up
/**
* Cleans up after a complete or expired task
*
*/
private void taskDone()
{
final boolean taskWasAlreadyComplete = getAndSetTaskComplete(true);
if (!taskWasAlreadyComplete)
{
synchronized (blockObject){
blockObject.notifyAll();
}
notifyListeners();
cleanUp();
}
}
|
The results of the task are paired up with the final status of the task and returned to the user (as in Listing 6) after passing by the blockOnTask() method that blocks callers until the task has completed by waiting on a synchronization object.
Listing 6. blockOnTask() and getResult()
/**
* blocks until a task is complete/expired
*
*/
private void blockOnTask(){
synchronized (blockObject) {
while (!isTaskComplete()){
try {
blockObject.wait(); // releases lock on blockObject
} catch (InterruptedException e) {
//do nothing
}
}
}
}
/**
* gets the result of the task. if the task has not completed or expired,
* will block until that happens
* @return
*/
public final TaskResult getResult()
{
blockOnTask();
return new TaskResult(getState(),getData());
}
|
The other classes in the framework are simple:
-
TaskState implements an enumeration of states a task can be in.
-
TaskResult is a pair consisting of returned data and the state that the task was in when the data was returned.
-
TaskEventListener is an interface that can be implemented for consumers to register listeners.
Use the code with its limitations
While risky, controlled threaded execution with immediate response short-circuiting provides a palatable, manageable method of ensuring availability regardless of external dependency availability. The growing need to stay up and running regardless of adverse conditions is a strong incentive to manage resources more aggressively and respond to users with speed and grace.
This pattern only works if all tasks will terminate within a reasonable amount of time. Although individual users of the time-limited task are insulated from this, the greater system will soon run out of resources if the Threads executing the task do not die in a timely fashion.
Also, concrete instances of TimeLimitedTask all share the same timer, which is important because all timer notifications run in the same thread. Further timer notifications are delayed until the currently triggered code has returned. This means that in this pattern, the cleanUp() method should run very quickly.
This pattern won't cover all areas -- you must evaluate the needs of your system, weigh the pros and cons of the system versus alternative ways to manage workload. First identify places where your application has some operation that takes an unpredictable amount of time that potentially affects your service level. Next, evaluate what your application can do to proceed with missing data; you may be able to display to the user a cached value, an error message, or fall back on another mechanism.
You might wish to develop some heuristic that manages excessive task cancellation (time overrun) by extending the time, not bothering to even try and run the task, and/or set off alarms for administrators to look at the problem.
Download | Name | Size | Download method |
|---|
| wa-shortcir_code.zip | | HTTP |
Resources
About the authors  | | | Brian D. Goodman is part of the Workplace Technology team in the CIO Office. His primary focus is IBM's On Demand Workplace, specializing in enterprise Web and client solutions. |
| | | James Kebinger works on the IBM Workplace Client Technology team helping develop IBM's next-generation client computing platform. |
Rate this page
| 
