Use a Message Splitter pattern to split a large XML message into smaller elements for processing by one or more targets.
Use a Message Splitter pattern to split a large XML message into smaller elements for processing by one or more targets.
You want to share information which has accumulated within an application over a period of time, with other applications. Typical examples might be sales data that has accumulated over a day, or information updates that are received from partners. The data is accumulated over a given time period and transmitted as a single large message. The end of the day processing at the target application, or target applications, requires that each element is processed individually.
Alternatively, an application might generate a message containing a number of records that must be distributed; for example, where one application maintains and distributes reference data.
Large messages which contain repetitive and processing structures can cause excessive memory use, which has a potential impact on other message processes and can result in broker failure. This pattern addresses the problem of routing the elements of a large message to different targets according to defined routing rules.
Use this pattern for applications that store information about a number of business transactions and transmit this information to one or more target applications in batches. It can be used to handle large messages, without excessive memory use.
This pattern can be used when either the source application or target applications cannot make or process near real-time service calls, or when batching of information is required for business reasons.
The solution is to implement a message flow that uses a mutable message tree, so that repeatable elements can be deleted as they are processed. Partial parsing enables single elements to be parsed as required and, because these elements are deleted on completion, the in-memory tree is prevented from growing linearly with the number of elements. As each element is extracted it is routed to the appropriate target application.
The input message and all output messages are processed in a single transaction; all output messages are persistent. A failure to read the XML input message causes the entire transaction to fail, but errors processing a single element can either add the failed element to a queue for later processing, or cause the entire transaction to fail.