WebSphere Application Server V5.x provides support for asynchronous messaging based on the Java™ Messaging Service (JMS) specification. Using the message listener service and either the Embedded JMS Server or an external message provider, such as WebSphere MQ, application developers can write message-driven beans (MDBs) that listen on a JMS destination (either a message queue or a topic) and that then get invoked when a message arrives at that particular destination. If the message listener service delivers a "poison message" to an MDB application, the application can choose to reject it. When this happens, what does the application server do with the message?
This article assumes a basic knowledge of JMS.
What is a poison message?
A poison message is simply a message that the receiving MDB application is unable to process. The message could be corrupt or just in an unexpected format. For example, suppose you have an MDB that handles JMS messages of type TextMessage. If the message listener service delivers a message of a different message type, then the message is considered a poison message.
Encountering a poison message
If an MDB discovers a poison message, it can do one of three things:
- Roll back the message to the queue it came from. This can be done if the MDB is running within a transaction, and ensures that the message is not lost. To do this, the MDB should call the setRollbackOnly() method on the message-driven context associated with it.
- Move the message onto a different queue. This is particularly useful when the MDB is not running within a transaction, as it prevents the poison message from being lost.
- Discard the message, by doing nothing. This means that the message is gone forever.
It is important to remember that it is the responsibility of the MDB application to determine if it has received a poison message. There is no way for the JMS provider or message listener service to detect if a message is corrupt or is of the JMS message type expected by the application.
Rolling back a poison message
MDBs can roll back poison messages if they are running within a transaction. This causes the message to be returned to the queue from which it originated. The default behaviour of the JMS Server in this situation is to shut down the listener port associated with the MDB, primarily to prevent a possible runaway condition from occurring.
If the listener port was not shut down, the message listener service would detect that a message has arrived at the destination being monitored, and would deliver the message to the MDB again. However, the message is still a poison message, so the MDB would roll it back for a second time.
Shutting down the listener port the first time a poison message is rolled back forces an application developer to carefully consider how the application and the JMS Server should handle them.
Changing the default behaviour
The exact behaviour of the JMS Server depends on three properties:
- The listener port property Maximum retries.
This property defines the number of times the message listener service will attempt to deliver a message to an MDB before the listener port is stopped. The default value for this property is 0, which means that the listener port will shut itself down the first time a message cannot be delivered or an application rejects a message as being poisoned. The Maximum retries property can be changed from the Listener Port Settings panel in the WebSphere administrative console.
Figure 1 shows the Listener Port Settings panel for the SamplePtoPListenerPort, used by a sample MDB application. This port has the Maximum retries property set to 10.
Figure 1. Listener port settings
- The JMS message property Redelivery count.
The Redelivery count property indicates the number of times a message has tried to be delivered to an application. This property is incremented if a message could not be delivered, or if an application rejects the message after delivery (for example, by rolling back the transaction).
- The JMS destination property Backout threshold.
The JMS Server that is installed as part of WebSphere Application Server allows users to define their own JMS destinations. Each destination defined in this way has the Backout threshold property fixed to the value 5. When a message has failed to be delivered a number of times equal to the Backout threshold property, the JMS Server will move the message to the default queue SYSTEM.DEAD.LETTER.QUEUE (this queue is a repository for messages which cannot be delivered, regardless of the reason).
The first thing the Embedded JMS Server does is return the message to the destination from where it came. At this point, the server will compare the Redelivery count of the message with the value of the Backout threshold property defined for the destination.
If the Redelivery count is less than the Backout threshold, the message is left on the queue. However, if the Redelivery count equals the Backout threshold, the message is moved off the queue and placed on the queue SYSTEM.DEAD.LETTER.QUEUE.
Where does the listener port's Maximum retries property fit in? As mentioned, this property specifies the maximum number of times that the listener tries to deliver a message to an MDB before the listener is stopped. The default value for this property is 0, which means that the first time a message cannot be delivered, the listener port is shut down, and the message is returned to the queue. At this point, the value of the Redelivery count property for the message is incremented.
When the listener port is restarted, it will attempt to deliver the message to the MDB again. If delivery fails or the message is rolled back again, the Redelivery count for the message will be incremented (giving it the value 2), and the listener port will be shut down.
This behaviour will continue until a message either fails to be delivered or rolls back five times, at which point the message will be placed on the SYSTEM.DEAD.LETTER.QUEUE. The reason for this is that the message's Redelivery count equals the Backout threshold for the queue.
If the value of the Maximum retries property is set to 6 or more, the listener port will never shut itself down if it is unable to deliver a message to an MDB. When the listener port fails to deliver the same message for the fifth time, the Embedded JMS Server will place the message on the SYSTEM.DEAD.LETTER.QUEUE rather than the original queue, as the Redelivery count equals the value of the Backout threshold for the queue.
When WebSphere MQ is the JMS provider
By default, queues created with WebSphere MQ have the Backout threshold property (known in WebSphere MQ terms as BOTHRESH) set to 0. Therefore, the default behaviour of WebSphere MQ is never to back out poison messages. What does this mean?
When a poison message is rolled back by an MDB, it is returned to the queue from which it originated. As the queue has no Backout threshold defined, the poison message will be left there. However, as it is now on the queue, it will be detected by the message listener service and redelivered to the MDB. But this is a poison message, so the MDB will reject the message, causing it to be rolled back and returned to the queue!
This sequence of events will continue until the number of times the poison message has been rolled back equals the value of the listener port's Maximum retries property. At this time, the listener port will shut itself down. The way to prevent this is from happening is to set the Backout threshold property on the queue to a value greater than 0 and less than the value of the listener port's Maximum retries property. This will ensure that any poison message is backed out and removed from the queue before the listener port shuts down.
Where do backed out messages go?
When using WebSphere MQ, you can specify the destination of backed out messages by modifying the Backout Requeue Name property (or BOQNAME, in WebSphere MQ terminology) on the queue. By default, this property is not set, which means that any messages that have been backed out a number of times equal to the Backout threshold for the queue will be lost. A good habit to get into is to set this property to the same value as the system's dead letter queue (the default dead letter queue is SYSTEM.DEAD.LETTER.QUEUE), as poison messages are effectively "dead letters".
The Backout threshold and Backout Requeue Name properties can be set using the command utilities provided by WebSphere MQ.
Figure 2 is taken from the WebSphere MQ Explorer utility, running on the Windows® platform. Here, a queue called CSINPUT_QUEUE has been defined with a Backout threshold of 1, and the Backout Requeue Name property set to SYSTEM.DEAD.LETTER.QUEUE. The first time a poison message is detected by an MDB and rolled back, that message will be moved from CSINPUT_QUEUE to the queue SYSTEM.DEAD.LETTER.QUEUE.
Figure 2. WebSphere MQ queue properties
Of course, you should periodically monitor the dead letter queue to check how many poison messages have been received by the application. If the dead letter queue contains a large number of poison messages, you should investigate the reasons why these messages are occurring.
This article described the default behaviour of the Embedded JMS Server provider within WebSphere Application Server V5 when encountering a poison message, how this behaviour can be changed by modifying properties of the Listener port, and what happens in this situation when WebSphere MQ is serves as the JMS provider.
- Simplify applications by using WebSphere Extended Messaging
- WebSphere V5 Extended Messaging support
- Creating extended messaging applications for WebSphere Application Server V5
- Interfacing WebSphere Extended Messaging with existing message applications