Technical Blog Post
Continuous Outbound Queue
Maximo Integration framework supports asynchronous processing using JMS queues. Queues are tied to External systems configuration such that each external system can potentially have their own set of outbound and inbound queues to communicate with Maximo.
Outbound queues are meant to facilitate asynchronous messaging from Maximo to External system. For example when a Mbo gets created/updated/deleted, the MIF Publish Channel traps the event and sends an XML message [based on the Object Structure defined] to the Queue. All outbound queues in Maximo are by default sequential. Messages in sequential queues get processed by only one consumer [jms crontask] and it has a hard stop for an error message. Crontasks will route the message to the External system based on the endpoint configuration for the system and or channel.
Continuous queue processing is not meant to stop on error. Though messages come out of the queue in order [FIFO], they are processed by multiple consumers [Message Driven Beans] simultaneously and hence will most certainly be processed out of order. This lends to the “self-correcting” nature of this queue where even if the messages were queued out of order – the messages may be processed in the right order [as needed by Maximo business rules – like GL message needs to go before the Purchase Order message which refers to that GL] as they error out and get retried by the consumers repeatedly.
A sequential inbound queue is very similar to the sequential
outbound queue – the only difference being the direction which the message is
moving. In case of sequential outbound the messages are moving from queue to
external system and for the inbound sequential it is moving from the queue to
Probably by this time you have noticed that the continuous
outbound queue is conspicuously missing. The initial design reason for this
being – when messages get stuck for outbound sequential queue, it is mostly
because of some bad [endpoint] configuration or network connectivity issue as
opposed to a business validation error. In that case it is not very useful to
try all messages as they are all bound to fail.
There is however valid use cases where the error maybe due to a business validation which can be resolved by a subsequent message [out-of-order issue] or maybe the error is localized to one particular module of the External system and other modules are functional and can consume messages. In either case it seems like a waste of resource and time to shut down the outbound queue just for one message failure. So the challenge is how to get a continuous out queue. Fortunately it is a fairly simple customization effort.
A queue by itself is neither continuous not sequential – it is just a queue which will give out messages in FIFO order. It is how you consume messages from the queue that makes it continuous or sequential. When you put multiple consumers [Message Driven Beans] for a queue – those consumers are going to process messages in parallel and are not going to wait when one of the consumers get a bad message. So the trick is to put an MDB for the out queue.
MIF already ships with 2 JMS consumers –
- A Message Driven Bean [MDB]
- A JMS crontask.
Both these components are written generically to consume messages continuously [MDB] and sequentially [crontask] agnostic of the direction [outbound or inbound]. Each of them refer to a direction specific component called the MessageProcessor
- QueueToMaximoProcessor [Inbound]
- QueueToDestinationProcessor [Outbound]
They handle processing the messages inbound to Maximo or outbound to the External System. Maximo allows customizing these processors as they are registered with crontask parameters [for JMS crontask] and EJB deployment descriptor [for MDB - ejb-jar.xml for mboejb module – as shown below].
As evident from the above ejb-jar.xml snippet – the MDB JMSContQueueProcessor-1 is used for inbound processing as the message processor registered with it is psdi.iface.jms.QueueToMaximoProcessor which handles inbound messages. There is nothing inside the MDB code to say it is inbound. So we can use this concept to create an MDB which processes messages from the outbound queue thus making it continuous instead of sequential. Listed below are the steps for that:
- First off – we need to disable the crontask that is running for the sequential outbound queue that we intend to make continuous.
- Next we can modify ejb-jar.xml to add a new MDB as shown below:
Note that the message processor has been changed to refer to the QueueToDestManagerProcessor that handles outbound messages. Also note that the ejb-name needs to be unique for that module. A corresponding container-transaction entry needs to be made in the same file for that newly added MDB as shown below.
we modify ibm-ejb-jar-bnd.xmi to add a new MDB as shown below:
<enterpriseBean xmi:type="ejb:MessageDriven" href="META-INF/ejb-jar.xml#MessageDriven_JMSContQueueProcessor_3"/>
Make sure that the id entered in step 2 [MessageDriven_JMSContQueueProcessor_3] matches up with the href attribute value in this file. Also note that the activationspec needs to get created in the Websphere and needs to have a unique JNDI name [like intjmsact3] which should match up with the activationSpecJndiName attribute value in this file. Note while creating the actiovationspec – we can reuse the same connection factory as long as they are under the same scope [cluster/server/node etc].
- Next up we need to create and error queue for this queue using the WAS admin console. This error queue will help move the error messages from this queue to the error queue thus preventing clogging this queue. In absence of that that you will run into a clog after N errors [N= Maximum concurrent MDB invocations per endpoint*Maximum batch size]. In case of sequential queue we never use error queues as it can only have one error at a given time and moving that error message away would imply we are breaking the “hard stop” contract of sequential queues.
- Rebuild and redeploy the ear file.
That should be good enough to get it working. You can also mark the queue as sequential – false in the maxqueue dialog. This is just for a clean implementation and not really used by the MIF outbound framework.
Another variation of this would have been to not tamper with the existing outbound queue and to create an entirely new queue and register it in maxqueue table and then change the External System configuration to point to that as the outbound sequential queue. External system configuration unfortunately does not let you define both sequential and continuous out queues.For more details on continuous queue configuration please refer to this blog.