Many customers using IBM® WebSphere® Message Broker solutions -- from single message-flow implementations to migration projects and beyond -- have the requirement to retry messages a defined number of times after a specified time interval. The delay may be needed because of a business process or the availability of a back-end system or database. This article shows you how to implement this requirement, including a configurable bar file that can be deployed without further development to provide the message retry and re-queuing functionality. This implementation uses new function in WebSphere Message Broker V6, including timeout processing and user-defined properties. The bar file and a sample message flow implementation are available for download below.
- WebSphere Message Broker V6 or later
- WebSphere MQ V5.3 or later
This article shows you how to use WebSphere Message Broker to set up a simple message flow that provides delayed message re-queuing without requiring changes to other flows in a solution. A standard, mandatory MQ message header is used so that any message input to the re-queue flow should be identical on output, which allows the standard processing of main flows to process re-queued messages without any change in function. Here is a description of the process used by the flow -- for each step, the relevant node in the flow is indicated in Figure 1:
- Read a message from an input queue (INPUT)
- Check to see how many times the message has been retried (SET PARAMETERS)
- If the retry count exceeds the maximum number of retries, then output to a maximum retry queue (MAXIMUMRETRY)
- If the retry count is less than the maximum number of retries, then:
- Delay the message for the delay period defined (TimeoutControl).
- Remove any changes made on entry to this flow to ensure the output message is identical to input (Restructure Message)
- Output the message to a queue to allow retry processing
replyToQordestinationQueuedepending on the settings defined) (OUTPUT)
Figure 1. Flow implementation
The flow uses the MQ Message Descriptor (MQMD) feedback field to record the number of retries used. The pre-defined application values of the feedback field are used to avoid conflict with system values. Other than the feedback field, the input message remains identical on output.
Important: If your application uses the feedback field of the MQMD, then this flow should not be used.
An assumption is made in the logic of the flow that the field OutputRoot.MQRFH2.usr.TimeoutRequest
is not used in the incoming message. This field stores data needed for routing internally, and is deleted before outputting a processed message.
If this is unacceptable, then the code can be changed to use a different field in the header. Although the MQ Rules And Format Header (MQRFH2)
may be added during processing of the flow, any additions will be removed before output.
For this flow to operate correctly, any applications feeding this flow must ensure that they copy the MQMD feedback information from input to output, to ensure that the retry count can be maintained. If this does not happen, then retry state cannot be maintained and poisoned messages may result if successful processing is never achieved in the main flow.
To use the implemented flow, several properties must be customised on the bar file prior to deployment. Once these properties have been set, no further changes are necessary for the flow to function:
- Import the bar file contained with this package into WebSphere Message Broker Toolkit.
- Change the configurable properties on the bar file to define settings specific to your installation.
- Deploy the bar file.
A message flow exists that requires retry processing. For the input queues to this flow, you can define a backout threshold (BOTHRESH)
and a backout queue (BOQNAME) in WebSphere MQ.
This main flow will then attempt to process any incoming message the number of times defined by the backout threshold.
If processing fails after reaching this threshold, the failure path of the input node will be executed.
You can output messages that act as input to the re-queue flow either during processing of the failure path or
by not ending processing during execution of the failure path (for example, by throwing an exception at the end of the failure processing).
Therefore, you can set configuration options accordingly:
- Input of the re-queue flow is set to:
- The backout queue of main flow, or
- A queue output from the failure terminal
- Output of the re-queue flow is set to the input of the main flow.
- Any messages arriving on the maximum message retry output queue will require manual intervention.
Below is a list of the configurable properties that are relevant for the bar file prior to deployment. After setting these values and deploying the resultant bar file, WebSphere Message Broker will process any messages arriving at the entry to the re-queue flow:
delay*- Time in seconds to delay each message
destinationQueue- Destination queue for successfully delayed messages
destinationQueueManager- Destination Queue Manager for successfully delayed messages
failureQueue*- Destination queue for messages that could not be successfully delivered
failureQueueManager- Destination Queue Manager for messages that could not be successfully delivered
inputQueue*- Input queue to the retry flow
maxRetriesQueue*- Destination queue for messages that have exceeded the maximum retry count
maxRetriesQueueManager- Destination Queue Manager for messages that have exceeded the maximum retry count
retryCount*- The number of times to re-queue a message
Unique Identifier*- A unique identifier defined to the retry flow
useReplyToQ- Use values defined in the
replyToQandreplyToQMgrfields of the MQMD as an alternative to thedestinationQueueanddestinationQueueManager
* These fields are required for the flow to function correctly.
Figure 2. Demonstration of configurable properties
- To use more than one copy (not instance) of this flow in a broker, you must set a different flow name and
Unique Identifierfor each deployment of the bar file. If you deploy the same bar file to different execution groups, then only theUnique Identifiermust be set. - The maximum
retryCountis 999934463. (MQ defines the propertiesMQFB_APPL_LASTandMQFB_APPL_FIRST. The difference between these numbers is the flow's maximum retry count). The feedback field can be defined to any message containing an MQMD and does not affect the structure of messages output. - To retry indefinitely, use the value -1. Use with care, because it can cause an infinite loop resulting in a poison message.
- If message contents are defined in the MQRFH2 header of an incoming message, then the corresponding message sets must be deployed in the same execution group as this flow. If this is not the case, then messages are delivered to the backout queue/dead letter queue defined for the input queue of the retry flow.
- Transaction mode in the implemented flows is set to
Yes. Therefore, when a broker starts, if there are any outstanding messages, they will all be output at the same time on restart. This ensures that no messages are lost. - The WebSphere MQ system queue
SYSTEM.BROKER.TIMEOUT.QUEUE(defined when a message broker is created) is used to store the messages currently delayed. Make sure that the maximum defined queue depth for this queue is sufficient to store messages, and monitor it to ensure that the maximum depth is not exceeded.
This article has shown you how to implement re-queuing using using new function in WebSphere Message Broker V6, including timeout processing and user-defined properties. By modifying the code provided, you can extend the current implementation to apply to additional business logic.
| Description | Name | Size | Download method |
|---|---|---|---|
| Code samples in zip format | MessageRequeue.zip | 95 KB | FTP |HTTP |
Information about download methods
- WebSphere Message Broker product page
Product descriptions, product news, training information, support information, and more. - WebSphere Message Broker documentation library
WebSphere Message Broker specifications and manuals. - WebSphere Message Broker information center
A single Eclipse-based Web portal to all WebSphere Message Broker V6 documentation, with conceptual, task, and reference information on installing, configuring, and using your WebSphere Message Broker environment. - WebSphere MQ product page
Product descriptions, product news, training information, support information, and more. - WebSphere MQ documentation library
WebSphere MQ manuals in PDF format. - WebSphere MQ V6 Information Center
A single Eclipse-based Web portal to all WebSphere MQ V6 documentation, with conceptual, task, and reference information on installing, configuring, and using your WebSphere MQ environment. - WebSphere Business Integration products
page
For both business and technical users, a handy overview of all WebSphere Business Integration products - developerWorks WebSphere Business Integration zone
For developers, access to WebSphere Business Integration how-to articles, downloads, tutorials, education, product info, and more. - Most popular WebSphere trial downloads
No-charge trial downloads for key WebSphere products. - Trial downloads for IBM software products
No-charge trial downloads for selected IBM® DB2®, Lotus®, Rational®, Tivoli®, and WebSphere® products. - Safari Bookshelf: e-library designed
for developers
Complete search and download access to thousands of technical books for a one-time subscription fee. Free trial for new subscribers. - WebSphere forums
Product-specific forums where you can ask questions and share your opinions with other WebSphere users. - developerWorks blogs
Ongoing, free-form columns by software experts, to which you can add your comments. Check out Grady Booch's blog on software architecture.
Stephen Cox is a Software Engineer on the Software Lab Services team at the IBM Hursley Software Lab in the UK. His expertise includes WebSphere MQ and WebSphere Message Broker. He is an IBM Advisory Accredited Professional, certified in Systems Administration of both WebSphere MQ and WebSphere Message Broker, and he received a Bachelors degree with Honours in Computer Systems Engineering from the University of Kent in the UK. You can contact Stephen at stephencox@uk.ibm.com.
Comments (Undergoing maintenance)
Table of contents
Next steps from IBM
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
