IBM WebSphere Developer Technical Journal: Building an Enterprise Service Bus with WebSphere Application Server V6 -- Part 7

Switching between message protocols

Ideas from the first six articles in this series on using the new messaging engine in IBM® WebSphere® Application Server V6 to build an Enterprise Service Bus are brought together to show how you can use an ESB to actually switch between different message protocols.

Share:

Rachel Reinitz (rreinitz@us.ibm.com), Senior Consulting IT Specialist, IBM

Rachel ReinitzRachel Reinitz is an IBM Distinguished Engineer with IBM Software Services for WebSphere and a member of the IBM Academy of Technology. She has built much of IBM’s internal and external intellectual capital and education on SOA, Enterprise Service Bus, and Web services. Rachel focuses on how to get started with SOA and ESB best practices. She consults with clients and ISVs on how SOA and ESB can be used to achieve their business and technical objectives. She is a frequent conference presenter and written many developerWorks articles on ESB and Web services. Rachel lives in the Bay Area in California, and enjoys hiking, socializing, and international travel.



Andre Tost, Senior Technical Staff Member, IBM

Andre TostAndre Tost works as a Senior Technical Staff Member in the Software Group's Enterprise Integration Solutions organization, where he helps IBM's customers establishing Service-Oriented Architectures. His special focus is on Web services technology. Before his current assignment, he spent ten years in various partner enablement, development and architecture roles in IBM software development, most recently for the WebSphere Business Development group. Originally from Germany, he now lives and works in Rochester, Minnesota. In his spare time, he likes to spend time with his family and play and watch soccer whenever possible.



21 September 2005

Also available in Russian

From the IBM WebSphere Developer Technical Journal.

Introduction

The first six articles in this series have shown you how to set up your own bus (Part 2), configure it to leverage JMS (Part 3), Web services (Part 5), and WebSphere MQ (Part 6), and also showed you how to perform message transformations using mediations (Part 4). Now, in Part 7, we will bring many of these ideas together and show you how you can use Service Integration Bus (SIBus) support to connect an MQ message queue that interacts with client applications with a Web service that offers the actual business function. In other words, we will use the ESB to switch between different message protocols!

Our series to this point is made up of these articles:


Reviewing our bag of tricks

In this article, we will take advantage of many of the SIBus features that we previously described in concert, so let's start by revisiting them briefly before digging into the details:

  1. In our solution, we position the WebSphere Application Server V6 SIBus as an Enterprise Service Bus. One of the first things we did was to configure a bus instance and added some common features to it, such as listener applications that serve the various inbound protocols the bus supports (for example, a SOAP/HTTP "endpoint listener").

  2. We also installed the SDO Repository, which, among other things, helps the bus track the message structures of installed Web services.

  3. Finally, we created destinations on the bus, which represent the actual places that messages can be sent to.

  4. Mediations are associated with destinations and can operate on and manipulate messages that are sent to that destination. They can also change the routing that a message takes as it traverses the bus.

  5. Web services are plugged into the bus via outbound services. Special destinations are generated for each outbound service that knows how to exchange messages with the Web service.

  6. WebSphere MQ is connected to the bus using the MQLink feature, which enables communication between the bus and any existing WebSphere MQ queue manager.

If you have followed our series so far, then you know all of this already. If not, go back and review the previous articles before continuing. Shortly, we will look at the protocol switch that we will be covering in this article, but before we do so, let's take a quick look at the overall business scenario that requires us to use this function.


Business scenario

The services and functionality that we built previously were intended to connect various systems of the Posts-R-Us company to enable the company to track the delivery of packages. You may remember that in Part 5, we described a service called PackageTrackingService, which could deliver information about the status of a particular service.

One of the reasons for offering this service on the ESB was so that it could be invoked by clients via multiple protocols. Now, we are going to assume that there are clients that only support communication over WebSphere MQ, and do not have any SOAP functionality. For example, assume that Posts-R-Us uses a homegrown CRM system that runs on iSeries and is written in RPG. This system sends a request message into a particular MQ queue and expects the response to this request in a separate response queue. This enables the system to send requests about a number of packages all at once and check the response at a later time one by one. In other words, it uses the synchronous service in an asynchronous way, decoupling the sending of the request from the retrieval of the response. It is not aware of the fact that behind the scenes, it is invoking a Web service over SOAP/HTTP; the ESB takes care of that.


Solution architecture

Multiple components have to be integrated to make this work, so here is everything in one picture (Figure 1) that we will reference throughout the rest of the article:

Figure 1. Solution architecture
Figure 1. Solution architecture

Let us step through the message flow that occurs when the client sends a request; the step numbers correspond to those in Figure 1:

  1. The client sends a request message to a queue that is defined in WebSphere MQ. To the MQ system, it is not a local queue, but a remote definition queue. This means that MQ will route all messages going to that queue into the SIBus, and from there to the destination called PackageTrackingFromMqDestination (we created this remote definition and its associated SIBus destination in Part 6). The message is a text message containing XML.

  2. A mediation called PackageTrackingMQToSOAPMediation is associated with the destination and will process every message that arrives there. The mediation creates a SOAP message and inserts the content of the received XML message into this new SOAP message. It also changes the reverse routing path of every message, inserting the PackageTrackingResponseDestination. We will look closer at this mediation later.

  3. When the message has been processed by the mediation, it will be forwarded to the PackageTrackingOutboundService destination by configuring the forward routing path of the PackageTrackFromMqDestination destination.

  4. From the outbound service destination, the actual Web service is invoked (we created this outbound service in Part 6) through an additional destination, called PackageTrackingOutboundPort. Since this is generated when creating the outbound service, it is not shown separately in Figure 1.

  5. The response from the Web service invocation is routed to the next destination according to the reverse routing path of the original request message. Since that had "PackageTrackingResponseDestination" in it, that is destination where the response message now goes.

  6. The response destination has another mediation associated with it, called PackageTrackingSOAPToMQMediation, that converts the SOAP response message that came back from the Web service into a text message that only contains the actual payload of the response message.

  7. We create a new foreign destination on the SIBus that represents the response queue in WebSphere MQ. When the response message is converted from SOAP into a text message, we can route it to this foreign destination that will simply forward the message back to MQ. From there, the client destination can pick it up using the usual MQ mechanisms.

We will now take you through all the configuration and code you will need to set this up on your own machine. You will want to refer back to Figure 1 to see how all the pieces work together.


Mediation: From text to SOAP

This mediation handler is for demonstration purposes only, and was developed with simplicity and readability in mind. It lacks error handling and good object-oriented structure that you would expect to see in production-level code. The code shown here also has much hardcoded information that would normally be contained in parameters. You can define context properties (basically key-value pairs) for a destination via the admin console, and then easily retrieve them from within a mediation handler using the MessageContext.getProperty() method.

As mentioned earlier, the message that comes into the bus from a client is an XML text message. We need to convert this into a SOAP message before we can route it to the Web service. This is done through a mediation that we need to develop. You can get the code for this mediation (and all other code we introduce here) in the download file included with this article.

PackageTrackingMQToSOAPMediation has one mediation handler in it, in a class with the same name.

Each mediation handler implements a method called handle(), which receives an object of type com.ibm.websphere.sib.mediation.messagecontext.SIMessageContext. Among other things, this parameter provides access to the actual message, which itself is stored in an SDO DataGraph. When retrieving this DataGraph, we must indicate what type of message we want to look at; in our case, this is a JMS text message. The code to get to the message body looks like this:

String payload;

// retrieve the message from context
SIMessage message = ((SIMessageContext)context).getSIMessage();
		
// get payload - assume text message
DataGraph graph = 
message.getNewDataGraph(SIApiConstants.JMS_FORMAT_TEXT);
DataObject body = graph.getRootObject();
if (body.isSet("data")) {
	// Grab the message content as a String
	payload = body.getString("data/value");
} else {
	// Error...
}

Each DataGraph contains a root DataObject. In the case of a SIBus text message, this root object has a property called "data" defined on it, which in turn has a property called "value". The code above copies the contents of this property (that is, the actual message payload) into the "payload" variable.

Next, we create the new message. In our example code, we have taken the easy route by hardcoding the SOAP envelope that must be sent to the Web service. This is not very flexible, of course. A real life solution would build the new SOAP message in a more generic fashion, reading the values for things like target namespace, operation, port name, and so on, from context properties. See the section in the WebSphere Application Server Information Center on message structures for more information and code snippets showing how to do that.

// insert the received message into the SOAP envelope
// this is pretty ugly, but it does the job for demo purposes
String soapEnvelope = "<soapenv:Envelope "  +
"xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" "  +
"xmlns:soapenc=\"http://schemas.xmlsoap.org/soap/encoding/\" "  +
"xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" "  +
"xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">"  +
"<soapenv:Header/><soapenv:Body>" + 
	payload +
"</soapenv:Body></soapenv:Envelope>";
			
body.setString("data/value", soapEnvelope);
			
// Replace the contents of the message with the new graph
message.setDataGraph(graph,SIApiConstants.JMS_FORMAT_TEXT);

As you can see, we simply insert the request XML message into a new SOAP envelope that can be processed by the target Web service. Before we can send it on to this Web service, however, we have to tell the system that this is indeed a SOAP message. So far, to the SIBus, it is nothing but text (notice that we inserted the new DataGraph into the message using the JMS_FORMAT_TEXT attribute).

SIBus defines the notion of a format string that contains information about the target Web service that the SOAP message will go to. The format string for an outbound SOAP service has the following structure:

SOAP:dest:[busname]:[service destination name],[service namespace],[service name],[port name]

In the case of our PackageTrackingService, this means the format string looks like this:

SOAP:dest:TheBus:PackageTrackingOutboundService,http://service.postrus,
PackageTrackingServiceService,PackageTrackingService

Following good mediation handler practices, we read this string from a context property, and use it to generate a new message:

// collect context property setting
String formatString = (String)context.getProperty("formatString");

DataGraph soap = message.getNewDataGraph(formatString);
message.setDataGraph(soap, formatString);

We are using a little trick here: we retrieve a new DataGraph from the existing message, with the defined SOAP format string, then set this new DataGraph back on the message, effectively replacing the text message that was there before. The result is a message that can be routed to the outbound service destination from where the Web service is invoked.


Mediation: From SOAP to text

The response message is a SOAP envelope containing XML information about the status of a particular package. We use PackageTrackingSOAPToMQMediation to convert the SOAP message into a text message that we can send back to our MQ client application. The approach is similar to what we saw above for the request message.

First, we retrieve the message payload from the incoming message. The difference here is that we read the payload into a byte array instead of a String, because that makes parsing it a little easier.

byte[] payload;

// retrieve the message from context
SIMessage message = ((SIMessageContext)context).getSIMessage();
		
// get payload - we retrieve it as a byte array, even though we know 
// it is really a SOAP message - the system will do 
// the conversion for us
DataGraph graph = 
message.getNewDataGraph(SIApiConstants.JMS_FORMAT_BYTES);
DataObject body = graph.getRootObject();
if (body.isSet("data")) {
	// Grab the message content as a byte array
	payload = body.getBytes("data/value");
} else {
	// Error...
}

In our example, we just want to extract the contents of the SOAP body element and send it back to our clients. To do so, we use a utility class that is included in the Apache Xerces XML parser package (included in WebSphere Application Server), called XMLSerializer:

// now we will parse the message and extract everything that is 
// inside the <Body>
Document doc = 
DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(
new ByteArrayInputStream(payload));
Element result = (Element)doc.getElementsByTagName("getReturnStatus").item(0);
			
// serialize the result into a String that we can send back to MQ
StringWriter sw = new StringWriter();
new XMLSerializer(sw, new OutputFormat()).serialize(result);

Now, we replace the actual message, similar to the way we did it before. The new message is a text message:

// Replace the contents of the message with the new graph		
DataGraph newGraph = 
message.getNewDataGraph(SIApiConstants.JMS_FORMAT_TEXT);
newGraph.getRootObject().set("data/value", sw.getBuffer().toString());			
message.setDataGraph(newGraph, SIApiConstants.JMS_FORMAT_TEXT);

As you can see, we use the same trick that we used before: retrieved a DataGraph instance from the message using a format string, then replaced the DataGraph on the message. The result of this is a message that can be read as a text message from any MQ client application.


Routing

Looking back at Figure 1, not only do we need to make sure that the message formats are converted properly with mediations, but we also have to worry about routing messages to the right place. There are various places at which we need to adjust the route a message takes:

  • Each message that arrives at PackageTrackingFromMqDestination must next be routed to the PackageTrackingOutboundService destination. We do that by configuring this destination as the default forward routing path for PackageTrackingFromMqDestination using the admin console (don't worry, we'll step through that in a minute).

  • Each response message must be processed by the PackageTrackingResponseDestination before it can be sent back to the client. To achieve this, we place PackageResponseDestination at the top of the reverse routing path for each request message in PackageTrackingMQToSOAPMediation, using the following code:

    // now let's insert the response destination into the reverse routing 
    // path, so that we can catch the response message
    // first build the replyAddress to be added
    SIDestinationAddress replyAddress;
    replyAddress = SIDestinationAddressFactory.getInstance()
    	.createSIDestinationAddress("PackageTrackingResponseDestination", 
    false);
    
    // Get a copy of the current reverse routing path
    List rrp = message.getReverseRoutingPath();
    
    // Add the response destination to the list (so that the
    // new destination will be the first destination that will be visited 
    // by the reply message)
    rrp.add(0, replyAddress);
    
    // Change the message's reverse routing path
    message.setReverseRoutingPath(rrp);

    (The name of the response destination is hardcoded here, but in an actual solution, this would be retrieved from a context property so that it could be changed at run time.)

  • From the response destination, each response message must be routed to the foreign destination called PackageTrackingToMQDestination, which serves as a proxy for the actual MQ queue on the client system. We do this again by setting the forward routing path of all messages coming through PackageTrackingResponseDestination to TheForeignBus:PackageTrackingToMQDestination@QM_atost. (This name could also have already been set by the client, as the reply-to field of the request message; we use the admin console again here to set this value.)


Install and configure

We can now install and configure all the necessary pieces. We will assume here that you have stepped through the configuration covered in our previous articles, and that you are familiar with basic SIBus configuration, since you've done most of this before.

  1. We will start with installing the mediation code. You can simply use the PackageTrackingMediation.ear file from the download file (which also contains all the the mediation handler source code for reference). Install the EAR file as a new enterprise application. After the application has been installed, your enterprise applications window will look something like Figure 2.

    Figure 2. Installed enterprise applications
    Figure 2. Installed enterprise applications
  2. Define the two new mediations we just installed. Navigate to Buses => TheBus => Mediations and configure two new mediations; make the Mediation name and the Handler list name the same:

    • PackageTrackingMQToSOAPMediation
    • PackageTrackingSOAPToMQMediation.

    When you are finished, you should have mediations defined as shown in Figure 3.

    Figure 3. Mediations
    Figure 3. Mediations
  3. Next, we define the additional destinations that we need for the solution. If you have followed the past articles in the series, you will already have defined PackageTrackingFromMqDestination, together with the necessary MQLink configuration. All we need in addition to that are the components responsible for the response path of our scenario. We need:

    • PackageTrackingResponseDestination: a plain queue destination.
    • PackageTrackingToMQDestination: a foreign destination that points to an MQ queue.
    • PackageTrackingToMQDestination: an MQ queue that you can define using the MQ Explorer tool.
    1. Define PackageTrackingResponseDestination. As shown in Figure 4, set the forward routing path in the admin console to:

      TheForeignBus:PackageTrackingToMQDestination@[your queue manager name].

      Figure 4. The forward routing path
      Figure 4. The forward routing path
    2. With the new destination created, define PackageTrackingSOAPToMQMediation as its mediation, using the Mediate button on the Mediations panel in the admin console.

    3. Create a new foreign destination called PackageTrackingToMQDestination@QM_nnn, where QM_nnn is the name of your MQ queue manager, and TheForeignBus as the foreign Bus value (Figure 5).

      Figure 5. A new foreign destination
      Figure 5. A new foreign destination
    4. Create the counterpart to the foreign destination in WebSphere MQ as a regular local queue, PackageTrackingToMQDestination. If you are running MQ in Windows, the MQ Explorer dialog to create this queue will look like Figure 6.

      Figure 6. Creating a new MQ queue
      Figure 6. Creating a new MQ queue
    5. Some additional configuration on the PackageTrackingFromMqDestination is all that is left. Set its forward routing path to "TheBus:PackageTrackingOutboundService". Then create a new context property called formatString of type String (Figure 7) and set it to:

      SOAP:dest:TheBus:PackageTrackingOutboundService,http://service.postrus,
      PackageTrackingServiceService,PackageTrackingService

      Remember that we use this context property in the PackageTrackingMQToSOAPMediation mediation.

      Figure 7. formatString context property
      Figure 7. formatString context property
    6. Finally, mediate the PackageTrackingFromMqDestination destination with PackageTrackingMQToSOAPMediation.

    After all these changes are made, you should see the list of destinations shown in Figure 8.

    Figure 8. The completed list of destinations
    Figure 8. The completed list of destinations

Save all the changes you have made and restart your server, if you have not already done so.


Test the solution

As in the previous article, we will use the MQ rfhutil utility to test our solution. Start it up and load the packagetracking.xml file using the Read File button. The content of the file looks like this:

<p843:getPackageStatus xmlns:p843="http://service.postrus"><trackingNumber>123
</trackingNumber></p843:getPackageStatus>

Send this message to the PackageTrackingRequestRemoteQueue queue. If all goes well, you should see some lines printed to the System.out file (stemming from the print statements in the mediation handler code), and the response message sent back to the PackageTrackingToMQDestination queue.

Read the result from that queue, again using rfhutil. The Data tab shows the contents of the received response message (Figure 9).

Figure 9. The response message
Figure 9. The response message

Let us point out again that by doing this, you have effectively provided the synchronous Web service, which is a request-response Web service available over HTTP, over an asynchronous protocol, that is, WebSphere MQ. This brings up the question of how you correlate a response to its respective request.


Regarding message correlation

In our solution, sending the request and receiving the response are completely decoupled from each other. How, then, will a client be able to retrieve the correct response message for a request that it issued earlier from the response queue? The answer lies in the correlation ID.

The client sets a unique message identifier on the request message that travels along with the message all the way to the outbound service destination. There it will be cached and copied to the correlation ID field of the response message, then travel back to the client in this field. The client can retrieve the response message by asking for a message with the associated correlation ID that was initially set as the message ID in the original request message. No additional configuration or coding is necessary to make this work.


Conclusion

In this article, we presented the most complex scenario so far, connecting an MQ client application with a SOAP/HTTP Web service via the SIBus. Requests and responses are handled asynchronously by the client application. The required message transformation between the two involved protocols is handled by mediations that run in the bus. Additional routing and other information is configurable at run time, for example, by setting context properties on destinations.

Other solutions that use a different combination of inbound and outbound protocols can be set up in a similar fashion. Moreover, one service can be offered via multiple protocols to clients, simply by configuring the proper set of destinations and mediations for each.

In the next (and final) article, we will sum up all we have done throughout this entire series, and provide an outlook of the future of the ESB in application development.


Download

DescriptionNameSize
Code sampleesb_part7_code.zip  ( HTTP | FTP )9 KB

Resources

Learn

Get products and technologies

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere, SOA and web services
ArticleID=93809
ArticleTitle=IBM WebSphere Developer Technical Journal: Building an Enterprise Service Bus with WebSphere Application Server V6 -- Part 7
publish-date=09212005