With the proliferation of mobile devices, many organizations using IBM® WebSphere® MQ (hereafter called MQ) and IBM WebSphere Message Broker are extending their messaging networks to encompass mobile devices by using WebSphere MQ Everyplace (hereafter called MQe). The number of devices in these instances can range from a few hundred to several tens of thousands. Real world examples include a logistics organization that needs to send messages to a workforce of 16,000 employees located across 12 different countries, and a major retailer with 1500 stores. Both enterprises use WebSphere MQ at the central site and WebSphere MQ Everyplace at the end points. The normal approach requires MQ to be configured with a remote queue definition for each target device (MQe Queue Manager and queues). But this approach quickly leads to an unacceptably high number of MQ objects that need to be defined and maintained, and an administrative and management nightmare for the operations personnel. This article describes a solution that avoids the need to create MQ resources to address each device.
WebSphere MQ / MQ Everyplace Architecture
In order to send messages from a device running MQe to MQ, or from MQ to a device, the system architecture consists of three components.
- Device queue manager
- Gateway queue manager
- MQ server
Figure 1. Typical MQ/MQe architecture
Each device runs an MQe queue manager (hereafter referred to as a device queue manager). Applications on the device work with the device queue manager to send and receive messages to and from queues that are either local queues on the device, or queues that are remote on other queue managers such as the MQ Server. Applications using the MQ server can send messages that are destined for device queue managers. A gateway queue manager facilitates the exchange of messages between device queue managers and MQ servers.
The gateway queue manager is an MQe queue manager configured with an additional feature called the MQ bridge. For messages going from device queue managers to MQ servers, the gateway provides a passthrough facility, transforming messages from the compact MQe format to the MQ format. For messages going from MQ servers to device queue managers, the gateway provides a store-and-forward facility holding messages for devices that are not currently online. When a device comes online, the device queue manager pulls messages from the store-and-forward facility. As with messages going from device queue managers to MQ, the gateway does the reverse transformation from MQ to MQe message format.
Addressing Remote MQ Everyplace Devices
In order for messages to move from MQe to MQ or vice versa, resource definitions such as queues and connections must be configured on the device queue manager, the gateway queue manager and the MQ server. This article concentrates on the resource definitions required on the MQ server.
There are a number of ways to configure MQ in order to send messages to a device queue manager. Normal practice requires one remote queue definition for every device queue manager.
The remote queue definition is defined with the target device queue manager name and queue name. To complete the MQ to MQe path, the remote queue is associated with a transmission queue, which provides a store-and-forward facility, holding messages for the gateway queue manager until it is ready to receive them. Unlike communications from one MQ server to another, MQ channels are not defined for transmission queues from an MQ server to a gateway queue manager. Instead, the gateway queue manager is configured with a listener, which pulls messages from the transmission queue and maintains the assured delivery process to the MQe target destination.
Using this approach, a remote queue definition is required for every queue on every device, and therefore the number of remote queue definitions grows rapidly. But MQ has a feature you can use to reduce the number of definitions required. When trying to locate a remote destination, MQ has an built-in search order:
- A remote queue definition with the same name as the target queue
- A remote queue definition with the same name as the target queue manager
- A transmission queue with the same name as the target queue manager
- A default transmission queue
You can take advantage of the MQ search order and define just the device queue manager name as the remote queue definition. This procedure allows any queue on the device to be addressed through the one definition, reducing the number of remote queue definitions to the number of device queue managers.
A similar approach is to use one transmission queue per device queue manager. But this approach has major drawbacks. First, the number of resource definitions on the gateway queue manager increases dramatically, such that for every transmission queue, a listener must be defined on the gateway. Second, the performance of the gateway declines because of the processing involved in handling the large number of listeners.
Using a default transmission queue at first glance appears to improve matters, because it removes the need for remote queue definitions. But there are some drawbacks -- any message that does not have a corresponding remote queue or transmission queue ends up on the default transmission queue. If a message not destined for a device queue manager ends up on the default transmission queue, the gateway queue manager will attempt to deliver the message and fail. This failure results in extra error processing, non-delivery of the message, and a potential halt in the movement of messages from MQ to MQe until the problem is resolved (for more details, see Unhandled Messages below). Only one gateway queue manager can be attached to a transmission queue. As there is only one default transmission queue per MQ server, only one gateway queue manager can work with it. This restriction reduces the scalability of the system when multiple gateways are required.
An example will show the effects of the different approaches described so far. Take a scenario where each device queue manager has two queues and there are 10,000 devices. The following list shows the number of definitions required for each approach (for illustration purposes one gateway is used in the calculation):
- Approach 1
- One queue definition for each target queue on a device queue manager
Number of MQ Definitions = ( 2 q def ) * 10000 + 1 xmitq def = 20001 - Approach 2
- One queue manager definition for each target device queue manager
Number of MQ Definitions = ( 1 qmgr def ) * 10000 + 1 xmitq def = 10001 - Approach 3
- One transmission queue for each target device queue manager
Number of MQ Definitions = ( 1 xmitq def ) * 10000 = 10000 - Approach 4
- Use the default transmission queue
Number of MQ Definitions = ( 1 xmitq def ) 1
Figure 2. MQ configuration approaches
All of the options discussed so far have limitations, such as the large number of definitions required, limited scalability, and management complexity. The remainder of the article describes a solution that addresses all of these problems. The solution allows more than one gateway to be attached, removing the drawback of the single transmission queue. At the same time, it avoids the need to define remote queues for each device queue manager. This solution involves:
- Sending the message to the gateway queue manager as opposed to the target queue manager
- Encoding the target queue manager and queue in the message
- Intercepting the message between the MQ transmission queue and the gateway queue manager
- Substituting the target queue manager and queue in the MQ transmission header so that the gateway is not aware that substitution has taken place. The substitution is done using a channel receive exit, which is a feature of MQ clients. For more information, see WebSphere MQ Using Java, go to Chapter 7 "Writing WebSphere MQ base Java applications," and see the section "Writing User Exits."
Figure 3. Solution architecture
The gateway queue manager listens for messages on a transmission queue and receives them on the gateway ready for onward delivery to the target device queue manager. While the message is in transit from MQ to the MQe gateway, the actual destination details (queue and queue manager name) must be coded somewhere within the message.
One approach is to put it in the user data area (message body). Another technique, which we have used in our sample code, is to specify the destination in the queue name itself. We have used the queue name to temporarily hold the device queue manager and queue name as
"QUEUE NAME % QUEUE MANAGER NAME" (without the intervening blanks). The queue name is not interrogated
by the receiving gateway queue manager until after receive exit processing is completed, giving you ample time to remove your encoded destination.
When the MQ application or the message broker sends the message to the device queue manager, the queue manager name should be filled in with the gateway queue manager name, and the queue name should be filled in with the target MQe queue name concatenated with the separator character
%, followed by the device queue manager name. In other words, the sending application targets the gateway queue manager as opposed to the device queue manager. In so doing, only the remote queue definition for the gateway queue manager is required, reducing the number of definitions to one.
The receive exit processing simply replaces the gateway queue manager name with the device queue manager name and tidies up the queue name,
deleting the characters following the first occurrence of %. The separator can be any valid character listed in
WebSphere MQ Application Programming Guide, Chapter 4, "WebSphere MQ Objects," under the heading "Rules for naming WebSphere MQ objects."
Figure 4. MQ receive exit processing
The space allocated in the MQ header to hold the queue name (Figure 5 below) is 48 characters. The best practice for naming MQe objects is to use the shortest possible name to save on transmission costs, and 48 characters is adequate to encode both the queue and queue manager names. In a typical device queue manager naming convention, the queue managers would have a prefix followed by some numeric characters. If space for the names is at a premium, you can add the prefix as part of the receive exit processing and let MQ identify the destination with just the numeric value.
The Java code to implement the solution described in the article is shown below. The logic manipulates the MQ transmission header, the structure of which is shown in Figure 5:
Figure 5. MQ transmit queue header structure
Listing 1. Sample Receive Exit Code
package examples.mqbridge.userexit;
/* ************************************************************** */
/* Description: Extend MQ Client channel receive exit to replace */
/* the target remote queue and remote queue manager in the */
/* transmit queue header (MQXQH)with new routing information */
/* ************************************************************** */
import com.ibm.mq.*;
import com.ibm.mqe.*;
import java.io.*;
/**
*
* <p>Extends MQ Client channel receive exit to replace target remote queue and
* remote queue manager in the MQXQH header with a user specified destination.
* <p>New MQe Destination is specified in the remote queue field in the format
* <pre>
* . QueueName%QueueManagerName
* </pre>
* <p>Exceptions and error conditions returns the buffer unchanged
* <p>Trace information is logged to MQeTrace
*
**/
public class MQeMQReceiveExit implements MQReceiveExit
{
private static final int MQXQH_START = 492;
private static final int MQXQH_ID = 0x58514820;
private static final char SEPARATOR_CHAR = '/';
private static final int QUEUE_LOCN = 500;
private static final int QMGR_LOCN = QUEUE_LOCN + MQC.MQ_Q_NAME_LENGTH;
private static final int END_DEST_INFO = QUEUE_LOCN + MQC.MQ_Q_NAME_LENGTH
+ MQC.MQ_Q_MGR_NAME_LENGTH;
private static final String MODULE_NAME
= "examples.mqbridge.userexit.MQeMQReceiveExit";
//* *************************************************************************
//* Method : receiveExit
//* Parms : MQ Message buffer and channel parameters
//* Returns: MQ message buffer with new routing information
//* *************************************************************************
public byte[] receiveExit(
MQChannelExit channelExitParms,
MQChannelDefinition channelDefinition,
byte[] agentBuffer)
{
MQeTrace.trace(this, (short) 310, MQeTrace.GROUP_ENTRY,
"--> "+MODULE_NAME+", receiveExit ");
if (agentBuffer.length > MQXQH_START) {
try {
// Create an MQ message object from the received message
MQMessage mqMsg = new MQMessage();
mqMsg.write(agentBuffer,0,agentBuffer.length);
// Check that the message header has an MQXQH header
if (isHeaderMQXQH(mqMsg)) {
// The received MQXQH Header information is:
// MQXQH.remoteQMgrName = MQe Gateway Queue Manager Name
// MQXQH.remoteQName = MQe Device Queue Manager Name '/' Queue Name
//
// The code below will switch the destination as follows:
// MQXQH.remoteQMgrName = MQe Device Queue Manager Name
// MQXQH.remoteQName = MQe Device Queue Name
{
// locate the separator char
int i=QUEUE_LOCN;
while (i<END_DEST_INFO &agentBuffer[i] != SEPARATOR_CHAR ) {
i++;
}
// If a separator is found replace the destination
if ( agentBuffer[i] == SEPARATOR_CHAR ) {
agentBuffer[i++]=0x20;// clear the separator char
int j=QMGR_LOCN;// index for queue manager field
// move the queue manager to the remoteQMgrName field
while (i<END_DEST_INFO &agentBuffer[i] != 0x20 ) {
agentBuffer[j++]=agentBuffer[i];
agentBuffer[i]=0x20;
i++;
}
// clear the rest of the queue manager field
for(i=j;i<END_DEST_INFO;i++) {
agentBuffer[i]=0x20;
}
}
}
}
} catch(Exception e) {
System.out.println("Exception"+e);
MQeTrace.trace(this, (short) 315,
MQeTrace.GROUP_EXCEPTION | MQeTrace.GROUP_USER_DEFINED_1,
"Exception caught, destination left unchanged:",e);
}
}
channelExitParms.exitResponse = MQChannelExit.MQXCC_OK;
MQeTrace.trace(this, (short) 320,
MQeTrace.GROUP_EXIT | MQeTrace.GROUP_USER_DEFINED_1,
"receiveExit()");
return (agentBuffer);
}
//* *************************************************************************
//* Method : isHeaderMQXQH
//* Parms : MQ Message Buffer
//* Returns: true if a MQ Transmit header is detected
//* *************************************************************************
private boolean isHeaderMQXQH(MQMessage mqMsg)
throws IOException, MQException
{
mqMsg.seek(0);
mqMsg.skipBytes(MQXQH_START);
int hdr = mqMsg.readInt4();
return(hdr == MQXQH_ID);
}
}
|
You can download the code below.
Updating the gateway configuration
Once the receive exit is compiled and built, you need to deploy and configure it on the gateway queue manager, which involves setting the receive exit attribute of the MQ client connection object on the gateway queue manager. The attribute is given the name of the Java class that implements the receive exit. You can specify the receive exit using the MQe Explorer administration tool or the MQe Scripting Tool, both of which are available in WebSphere MQe Server SupportPac ES06. You can also specify the receive exit programmatically using MQe admin commands.
Figure 5. Updating the MQ channel exit at the gateway
If a message cannot be handled by a gateway queue manager, the movement of messages from MQ to the gateway can come to a standstill. When a gateway
queue manager receives a message from MQ, it interrogates its connection and queue definitions in order to decide how to route the message to its final destination. Typically, the message ends up on a store-and-forward queue awaiting collection by the client devices. If the gateway has no knowledge of the target queue manager,
an undelivered rule comes into force. This rule is set as a property of the gateway listener.
The default behavior of this rule is to keep retrying at longer and longer intervals until a threshold is reached, and then finally stop the listener. The undelivered message will stay in the MQ transmit queue awaiting operator intervention. You can modify this rule to take a different action, such as removing the message and writing it to a different queue. For an example, see examples.mqbridge.rules.UndeliveredMQMessageToDLQRule.
The article walked through the various options for configuring MQ to use a large number of MQe queue managers. It described the pros and cons of each option, noting that all of the standard configuration options have drawbacks. The article then described a technique that dramatically reduces the number of definitions from the number of device queue managers to the number of gateway queue managers. This technique lets you scale up to large numbers of MQe queue managers without the overhead of having to manage large numbers of resource definitions.
| Description | Name | Size | Download method |
|---|---|---|---|
| Sample MQ Receive Exit | MQeMQReceiveExit.zip | 2 KB | FTP |HTTP |
Information about download methods
-
WebSphere MQ V6 information center
A single Eclipse-based Web portal to all WebSphere MQ V6 documentation, with conceptual, task, and reference information on installing, configuring, and using your WebSphere MQ environment. -
WebSphere MQ product page
Product descriptions, product news, training information, support information, and more. -
WebSphere MQ documentation library
WebSphere MQ product manuals. -
WebSphere MQ Application Programming Guide
This book introduces the concepts of messages and queues, and shows you in detail how to design and write applications that use the services that WebSphere MQ provides. -
WebSphere MQ Using Java
This book shows you how to verify the installation of, configure, and administer WebSphere MQ JMS, and also how to write WebSphere MQ base Java and WebSphere MQ JMS applications. -
Configuring WebSphere MQ Everyplace Gateway V2 for optimal throughput
This article shows you how to set up the gateway between WebSphere MQ and WebSphere MQ Everyplace. Little programming knowledge is required because you can do the setup using the SupportPac scripting tool. -
Download a free trial version of WebSphere MQ V6
-
Download a free trial version of WebSphere MQ Everyplace V2
-
Download the MQe Explorer and MQe Scripting Tool from WebSphere MQ Everyplace Server SupportPac ES06
-
WebSphere Message Broker information center
A single Eclipse-based Web portal to all WebSphere Message Broker V6 documentation, with conceptual, task, and reference information on installing, configuring, and using your WebSphere Message Broker environment. -
WebSphere Message Broker product page
Product descriptions, product news, training information, support information, and more. -
WebSphere Message Broker documentation library
WebSphere Message Broker specifications and manuals. -
WebSphere Business Integration products page
For both business and technical users, a handy overview of all WebSphere Business Integration products. -
developerWorks WebSphere Business Integration zone
For developers, access to WebSphere Business Integration how-to articles, downloads, tutorials, education, product info, and more. -
Most popular WebSphere trial downloads
No-charge trial downloads for key WebSphere products. -
Trial downloads for IBM software products
No-charge trial downloads for selected IBM® DB2®, Lotus®, Rational®, Tivoli®, and WebSphere® products. -
Safari Bookshelf: e-library designed for developers
Complete search and download access to thousands of technical books for a one-time subscription fee. Free trial for new subscribers. -
WebSphere forums
Product-specific forums where you can ask questions and share your opinions with other WebSphere users. -
developerWorks blogs
Ongoing, free-form columns by software experts, to which you can add your comments. Check out Grady Booch's blog on software architecture.
Comments (Undergoing maintenance)
Table of contents
Next steps from IBM
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
