Background

Think of messaging solutions, and you probably envision a system that integrates two different applications by a remote message invocation mechanism. Traditionally, this kind of coupling has been reserved for distributed entities that communicate infrequently and in ways requiring a fairly modest transfer of data. Classic examples are homogeneous interfaces to heterogeneous backends and portals that delegate backend processing of user requests and then reformat them for end-user presentation.

The common thread in messaging methods is the assumption that the messaging solution, while providing robust, highly available communication between systems, is fundamentally inefficient and should be used only as a last resort when communication to an outside system cannot be avoided. This view of messaging has prevailed from the onset of remote method calls (RMC) to more modern messaging solutions like CORBA and DCOM, and it has limited the type of problem that messaging solutions are usually applied to.

Back to top

The goal

Over the last decade there has been an increased understanding of the needs of distributed systems. Emerging technologies like Java and .NET have included code distribution as part of their fundamental programming model. In doing so, these technologies have incorporated high availability and fault tolerance into messaging, encouraging solution providers to deliver systems with performance characteristics applicable to a wider variety of problems.

Recently our company was asked to implement a file distribution and replication solution that previously would have required a custom system integrating secure FTP, database replication, and other one-off solutions. Rather than head down the custom development pathways, we explored the possibility of applying state-of-the-art messaging solutions to the problem. We found that not only did JMS provide the necessary infrastructure for transferring information, but it also handled all of the infrastructure issues related to quality of service, security, reliability, and performance that our customer required. This article describes the challenges our team faced, and how JMS (in the form of MQSeries) let us meet and exceed our customer requirements.

Back to top

The problem

Our customer had a significant distributed data challenge, with a number of call centers around the country where operators record interactions with customers. The recordings had to be quickly and reliably indexed and archived in remote data centers. The storage procedure could not affect the ability of the operator's system to record and store ongoing customer interactions. The customer had an existing system that included a combination of custom code, VPN, and other technologies. But, the existing solution was falling far short of the performance and reliability goals, and was a cobbling of technology that was hard to understand and expensive to maintain.

While developing a replacement system, we considered JMS and a variety of non-JMS solutions, particularly those based on FTP and secure copy (SCP). However, the non-JMS solutions had two fundamental drawbacks:

• They were riddled with security flaws. (The security holes in FTP are well known and widely documented. For a sampling, see Resources.)
• They provided the infrastructure for only the actual transfer of data, requiring custom development to handle issues with reliability, fault tolerance, security, platform independence, and performance optimization.

Our team concluded that the development effort to add these additional characteristics would be prohibitive, and therefore settled on the JMS solution, which provided them right out of the box.

Back to top

The solution

We developed a JMS-based system that

• Provided reliable archiving for recorded multimedia files
• Would allow extendability to let multiple data centers receive the files
• Would allow additional data types to be archived.

The files in question were larger (50K - 500K) than data we had transferred in previous projects that involved messaging solutions. Our first task was to ensure that the data sizes would not preclude a JMS solution. We evaluated a number of JMS solutions, including IBM MQSeries, by testing the performance of the system's delivery message payloads of various sizes. The results showed that, with appropriate configuration, messages of up to 1 Meg did not have a noticeable effect on overall system performance. Because conventional wisdom was that messaging solutions were only appropriate for periodic, small payloads, our result was a significant finding. We proceeded to lay out the system architecture, outlined in Figure 1, that would provide the security, high availability, and reliability the customer needed.

The existing infrastructure had a system on each client machine that created multimedia files in response to interactions between operators and users. These files needed to be archived. Our system starts a process running on each machine and looks for these files in known directories. When new files are detected, they are packaged into a JMS payload and sent to a JMS server in one of the data centers for delivery. Once the JMS server acknowledges receipt, the files are removed from the sender. The JMS server transfers the data to one of the available handlers in the data centers for archiving.

Back to top

Key concepts

JMS is a Java-specific implementation of messaging and queuing. There are two fundamental ideas in messaging and queuing:

• Systems communicate using discrete packets of data that have a payload, or the information to be conveyed, and attributes, or the characteristics of that information and how it should be communicated. The packets are called messages.
• Messages are not sent to systems, but to a separate holding area. There can be as many holding areas as you want, and they can be identified and located by unique names. Each of the holding areas can receive messages and, depending upon configuration, with each message will either deliver it to all interested systems (publish-subscribe) or to the first interested system (point-to-point). The holding areas are called destinations.

The system we built relies on point-to-point destinations, called queues in JMS. Queuing is an important aspect of the system design shown in Figure 1. While the picture shows messages being delivered from the JMS Broker directly to the receiver client, this is not quite accurate. The messages are actually delivered to a queue and the receiver client retrieves them from the queue. This distinction will become important later when we delve into the implementation details, because it lets the system process the incoming messages in parallel.

Back to top

Cross-platform and cross-vendor

It was important to our client to limit vendor lock-in, meaning we needed to design our code to minimize the impact of changing JMS vendors. A key advantage to JMS was the broad industry support and open standard it was based on, so that with properly designed code we could make the system work with any JMS system. (There was an immediate improvement over the existing system, which was specifically designed to work on a certain set of hardware and in conjunction with specific vendor solutions.)

The platform independence was easily accomplished by encapsulating all vendor-specific calls inside of classes we called JMSProviders. These providers handled such vendor-specific issues as factory lookup, error handling, connection creation, and message property setting. See the code in Listing 1 below for an example.

public QueueConnection createConnection() throws JMSException {
     return getConnectionFactory().createQueueConnection(getUserName(), 
          getPassword());
}

By leveraging the Java Naming and Directory Interface (JNDI) we stored the vendor-specific settings in a repository (for example, an LDAP store) so the actual code requires even fewer vendor-specific references. There is a small amount of vendor-specific code needed to handle some idiosyncrasies, but it can be limited to some "adaptor" classes, keeping it out of the application code. See the code in Listing 2 below for an example. Because JMS is designed to work easily with JNDI, it was another immediate advantage over other solutions -- a centralized location for configuration information that not only could hold text based information, but also could store configured objects.

public final static String 
 CONNECTION_FACTORY_LOOKUP_NAME_KEY = "CONNECTION_FACTORY_LOOKUP_NAME";
public final static 
 String FILE_TRANSFER_QUEUE_LOOKUP_NAME_KEY = 
  "FILE_TRANSFER_QUEUE_LOOKUP_NAME";
public final static String 
 JMS_PROVIDER_CLASS_KEY = "JMS_PROVIDER_CLASS";

public void init() throws NamingException {
	InitialContext jndi = createInitialContext();
	initConnectionFactory(jndi);
	initFileTransferQueue(jndi);
}

public QueueConnection createConnection() throws JMSException {return 
     getConnectionFactory().createQueueConnection(getUserName(), 
	     getPassword());
}

public void initConnectionFactory(InitialContext jndi) throws 
     NamingException {
          setConnectionFactory((QueueConnectionFactory)jndi.lookup
               (getProperties().getProperty(CONNECTION_FACTORY_LOOKUP_NAME_KEY)));
}

public void initFileTransferQueue(InitialContext jndi) throws 
     NamingException {
          setFileTransferQueue((Queue) jndi.lookup
               (getProperties().getProperty(FILE_TRANSFER_QUEUE_LOOKUP_NAME_KEY)));
}

Out of the box, JMS solutions allow messages to be sent with a guarantee that once a message is acknowledged as delivered to the JMS server, it will be delivered to the destination (queue) to which it was addressed. MQSeries is no exception. Once the code to send the message to the server is successfully executed, the client can be assured that the destination will eventually receive the message even if the server in question has a failure during processing (if the destination is temporarily unavailable, or the JMS server dies, and so on). See the code in Listing 3 below for an example. The class in the code below is responsible for actually executing the sending of data once it has been determined that it's necessary to send the file.

By configuring the message as persistent we can guarantee that once the message is received by the destination (queue), it will remain there until it is retrieved from the queue -- even across system failures. Hence, once the message is safely delivered to the local JMS server, it can be deleted. The value of overcoming system failures cannot be overestimated; handling of periodic system outages and failures is one of the biggest problems with developing distributed archiving solutions. The customer's existing system had complicated and brittle code to handle failure scenarios, and failures were costly in terms of processing and maintenance. JMS allowed us to solve all these problems by delegating them to a robust, battle-tested, commercial solution.

public void sendMessage(byte[] payload, boolean persistent) throws 
     SendFailedException {
     QueueSender sender = null;
     try {
     Message message = createMessage(payload);
     sender = createSender
     (persistent ? DeliveryMode.PERSISTENT : DeliveryMode.NON_PERSISTENT);
     sender.send(message);
	     getClient().getLogService().logInfo(getName() + 
	     " sent message " + message.getJMSMessageID() + ".");
     } catch (JMSException exception) {
	     getClient().getLogService().logError
	     ("JMS exception processing " +  getName(),exception);
	     stop();
	     throw new SendFailedException("JMS Message Send Failed");
	}
     try {
	     sender.close();
     } catch (JMSException ignore) {
	     getClient().getLogService().logInfo(getName() + " failed to 
	     close sender.  Processing will continue.");
	}
}

The key to this solution is configuring the JMS messages and server to provide both adequate performance and quality of service. The configuration options are defined by the JMS specification and are implemented by all commercial solutions. However, the exact method of configuration varies from vendor to vendor.

Back to top

The setup

The architecture and system we created is general and powerful. However, there are a number of moving parts that must be configured and hooked up in just the right way. Below is an overview, some potential pitfalls, and practical instructions to successfully set up MQSeries as a JMS server.

With MQSeries, first set up a JNDI server to retrieve the implementation-specific settings, which in this case is the JMS Connection Factory. There are many different ways to do this, but a good all-purpose choice is a Lightweight Directory Access Protocol (LDAP) server. We chose to use Qualcomm SLAPD. Once the server is installed and running, the MQSeries admin tools (JMSAdmin.bat) can be set up to use it as the repository for MQ object information. See Resources for a link to a useful book about this procedure. Also, during setup it's important to pay close attention to the IBM documentation for setting up JMS on top of IBM MQSeries. The process involves creating queues and other objects that are specific for JMS usage and not part of the standard MQSeries installation.

com.ibm.mq.MQEnvironment.classJmsProviderMQSeriesProvider

newQueueConnection = 
getConnectionFactory().createQueueConnection(getUserName(),getPassword));
   

as in Listing 1, we must call

newQueueConnection = getConnectionFactory().createQueueConnection();
   

Finally, you need to supply the JMS-specific elements to the clients, such as the queues, queue managers, queue factories, and so on. Now, the reason for using LDAP and JNDI becomes apparent: We use the LDAP server to store these elements and use external files to hold the keys to those LDAP objects. The LDAP server can act like a JNDI server and respond to name lookups by returning the objects we stored. This is what allows the code in Listing 2 to work. The name for a JMS element is obtained from a class static variable (for the default name) or an external file (to use something other than the default). In short, the LDAP server is asked for the object that is stored at the key in question and an object, in this case the JMS object that we are interested in, is returned.

Our JMS-based solution facilitated a uniform, cross-platform, and cross-vendor configuration environment using off-the-shelf components. Now our code is as isolated as possible from platform-specific and vendor-specific settings.

Back to top

The application

There are two key components to the application: a sender and a receiver. The sender launches a background that polls a directory for files that need to be archived, while the receiver simply waits for JMS messages to be delivered then archives the files contained in the messages. The JMS API lets us define these components with almost no regard for the specific JMS implementation that we are using.

The sender consists of three main parts:

• A JMSProvider for creating connections
• A ConnectionPool for obtaining existing, idle connections (which we will call JMSConnections)
• A poller to watch for files that need to be transferred.

At startup, the JMSProvider is used to create some ready connections to the JMS server. The connections are placed in the pool and then the poller is started. When the poller detects a file that needs to be transferred, it creates a separate thread to process the file. (Describing the forking of the message creation and transferring operation as creating a separate thread is a simplification. In reality a combination of pooling and iteration is used to ensure that new threads are rarely created and are instead reused. But, that process is fairly complicated and would obscure the JMS focus of this article.)

In the separate thread, the poller then obtains a JMSConnection from the connection pool, uses it to create a BytesMessage, and places the binary contents of the file into that message. Finally, the message is addressed to the receiver, sent to the JMS Server, and then the JMSConnection is returned to the ConnectionPool. Part of this sending process is shown in Figure 2 below.

The receiver is a simpler component; it starts a number of FileListeners that wait for messages to be placed in the receiver queue. Listing 4 below shows the code for setting up the processing by the FileListeners. The class in Figure 6 is responsible for actually retrieving the messages from the queue and archiving them. JMS guarantees a queue will deliver each message no more than once, so we can safely start many different FileListener threads and know that each message (and therefore each file) will be processed only once. This guarantee is another important advantage to using a JMS-based solution. Developing such function in a home-grown solution, such as one based on FTP, is expensive and error-prone.

public void startOn(Queue queue) {
	setQueue(queue);
	createConnection();
	try {
		createSession();
		createReceiver();
		getConnection().start();  // this starts 
		the queue listener
	} catch (JMSException exception) {
		// Handle the exception
	}
}

public void createReceiver() throws javax.jms.JMSException {
	try {
		QueueReceiver receiver = getSession().
createReceiver(getQueue());
		receiver.setMessageListener(this);
	} catch (JMSException exception) {
		// Handle the exception
	}
}
public void createSession() throws JMSException {
	setSession(getConnection().
createQueueSession(false, Session.AUTO_ACKNOWLEDGE));
}

public void createConnection() {
	while (!hasConnection()) {
		try {
			setConnection(getClient().createConnection());
		} catch (JMSException exception) {
			// Connections drop periodically on the 
			internet, log and try again.
			try {
				Thread.sleep(2000);
			} catch 
			(java.lang.InterruptedException ignored) {
			}
		}
	}
}
   

The message handling code is written in a callback, a method that JMS automatically invokes when a message is delivered to the FileListener. The code for this message is shown in Listing 5 below.

public void onMessage(Message message) {
	BytesMessage byteMessage = ((BytesMessage) message);
	OutputStream stream = 
new BufferedOutputStream(
new FileOutputStream(getFilenameFor(message)));
	byte[] buffer = new byte[getFileBufferSize()];
	int length = 0;
	try {
		while ((length = byteMessage.readBytes(buffer)) != -1) {
			stream.write(buffer, 0, length); 
		}
		stream.close();
	} catch (JMSException exception) {
		// Handle the JMSException
	} catch (IOException exception) {
		// Handle the IOException
	}
}
   

A trick to remember when setting up the receiver is to ensure that the initial thread that launches all the FileListeners continues to run after all of the FileListeners have started. This is necessary because some JMS implementations start QueueListeners in daemon threads. So, the Java Virtual Machine (JVM) might exit unexpectedly early if the only threads that are running are daemon threads. Listing 6 below shows some simple code to prevent this from occurring.

public static void main(String[] args) {
   ReceiverClient newReceiverClient = new ReceiverClient();
   newReceiverClient.init();
   setSoleInstance(newReceiverClient);
   while(!finished) {	// This prevents the VM from exiting early
      try {
          Thread.sleep(1000);
      } catch (InterruptedException ex) {
      }
   }
}
   

Back to top

Conclusion

After our initial implementation of the project, we added features like message compression, auto-recovery when sites become unreachable, federated message brokers, security, robust logging, administration, and more. The elements were easy to add because of the open model JMS provides, as well as the robustness of our architecture. The entire system took six weeks to build and quickly replaced the existing, labor-intensive one the customer had been using. Within days the system had exceeded all benchmarks and had already corrected errors that the old system left behind. In addition to exceeding our customer's expectations, this project proved that JMS is a viable solution not only for small, message-oriented applications, but also for large-scale, mission-critical data transfer operations.

Resources

• Visit the JMS home page.
  
  
• Review the security issues with FTP.
  
  
• Consult the JMS setup book, "Professional JMS Programming", by Giotta, et al.
  
  
• Learn more about IBM MQSeries, or download a trial version.
  
  
• Read this related developerWorks article, "Java theory and practice: Should you use JMS in your next enterprise application?"
  
  
• Read Daniel's previous developerWorks article, "The profound impact of hardware, software, and networking decisions."
  
  

About the author

Comments (Undergoing maintenance)

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Background
• The goal
• The problem
• The solution
• Key concepts
• Cross-platform and cross-vendor
• The setup
• The application
• Conclusion
• Resources
• About the author
• Comments

Next steps from IBM

• Try: WebSphere MQ
• Webcast: A smarter way to successful SOA - Information as a trusted strategic asset
• Briefing: WebSphere flexible business processes for a smarter planet
• Buy: WebSphere MQ

My developerWorks community