IBM Integration Bus, Version 9.0.0.8 Operating Systems: AIX, HP-Itanium, Linux, Solaris, Windows, z/OS

See information about the latest product version

Working with large input messages to propagate multiple output messages

The tree representation of a message is typically bigger than the input bit stream. Manipulating a large message tree can require much storage but you can code Java™ methods that help to reduce the storage load on the broker.

When an input bit stream is parsed and a logical tree is created, the tree representation of a message is typically bigger, and in some cases much bigger, than the corresponding bit stream.

The reasons for this expansion include the following factors:

The addition of the pointers that link the objects together
Translation of character data into Unicode, which can double the size
The inclusion of field names that might have been implicit in the bit stream
The presence of control data that is associated with operation of the broker

Manipulating a large message tree can require much storage. If you design a message flow that handles large messages that are made up of repeating structures, you can code Java methods that help to reduce the storage load on the broker. These methods support both random and sequential access to the message, but assume that you do not need access to the whole message at one time.

These Java methods cause the broker to complete limited parsing of the message, and to keep in storage at one time, only that part of the message tree that reflects a single record. If your processing requires you to retain information from record to record (for example, to calculate a total price from a repeating structure of items in an order), you can either declare, initialize, and maintain Java variables, or you can save values in another part of the message tree; for example, in the local environment.

This technique reduces the memory that is used by the broker to the memory that is needed to hold the full input and output bit streams, plus the memory that is needed for the message trees of just one record. This technique also provides memory savings when even a few repeats are encountered in the message. The broker uses partial parsing and the ability to parse specified parts of the message tree, to and from the corresponding part of the bit stream.

To use these techniques in your JavaCompute node (or Java user-defined node), take any of the following steps.

Copy the input MbMessage by using the copy constructor. This action creates a modifiable copy of the input message that is not parsed and therefore uses a minimum amount of memory.
Avoid any inspection of the input message, which avoids the need to parse the message.
Use a loop and an MbElement variable to step through the message one record at a time. For each record, use the following processes:
- Use normal transforms to build a corresponding output subtree in an output MbMessage.
- Use the MbElement.delete() method to delete the current input record message tree when you have completed the record manipulation.
- Use the following code to propagate the output message for the current record.
```
MbOutputTerminal.propagate(MbMessageAssembly, true)
```
  This process allows message tree resources and parsers to be recovered and reused for the next output message iteration.

You can vary these techniques to suit the processing that is required for your messages.

The following example Java code demonstrates how to parse a large input message with many repeating records, where each record is propagated as an individual output message.

The input XMLNSC messages are in the following form, where there are many repeating Record elements.

<TestCase>
  <Record><Field1>A</Field1><Field2>B</Field2><Field3>C</Field3><Field4>D</Field4><Field5>EA</Field5></Record>
  <Record><Field1>A</Field1><Field2>B</Field2><Field3>C</Field3><Field4>D</Field4><Field5>EA</Field5></Record>
  <Record><Field1>A</Field1><Field2>B</Field2><Field3>C</Field3><Field4>D</Field4><Field5>EA</Field5></Record>
  <Record><Field1>A</Field1><Field2>B</Field2><Field3>C</Field3><Field4>D</Field4><Field5>EA</Field5></Record>
  ....
</TestCase>

The following example shows the Java code that can be used to parse this large message, one record at a time, and propagate each record.

//Make a modifiable MbMessage based on the input message passed in on the inAssembly.
MbMessage clonedInMessage = new MbMessage(inAssembly.getMessage());
//Now partially parse the cloned input message one record at a time.
MbElement inputRootElement = clonedInMessage.getRootElement();
MbElement inputPropertiesElement = inputRootElement.getFirstElementByPath("Properties");
MbElement inputMQMDElement = inputRootElement.getFirstElementByPath("MQMD");
MbElement inputXMLNSCElement = inputRootElement.getFirstElementByPath("XMLNSC");

MbElement inputXMLNSCRootTagElement = inputXMLNSCElement.getFirstChild(); //Move to the TestCase tag
MbElement currentInputRecord = inputXMLNSCRootTagElement.getFirstChild(); //Move to the Record tag

while(currentInputRecord != null)
{
  // Create a new output message for the record that we are going to be propagate.
  MbMessage outputMessage = new MbMessage();
  MbMessageAssembly outAssembly = new MbMessageAssembly(inAssembly, outputMessage);

  //Create new parsers folders in the output message.
  MbElement outputRootElement = outputMessage.getRootElement();
  MbElement outputPropertiesElement = outputRootElement.createElementAsLastChild(inputPropertiesElement.getParserClassName());
  MbElement outputMQMDElement = outputRootElement.createElementAsLastChild(inputMQMDElement.getParserClassName());
  MbElement outputXMLNSCElement = outputRootElement.createElementAsLastChild(inputXMLNSCElement.getParserClassName());       
  //Create the root tag in the output XMLNSC folder that will be used for this output record.
  MbElement outputXMLNSCRootTag = outputXMLNSCElement.createElementAsLastChild(MbElement.TYPE_NAME, "TestCase", null);
  //Create the record tag for this output message instance.
  MbElement currentOutputRecord = outputXMLNSCRootTag.createElementAsLastChild(MbElement.TYPE_NAME, "Record", null);

  //Copy the Properties Folder, MQMD header, and the current record.
  outputPropertiesElement.copyElementTree(inputPropertiesElement);
  outputMQMDElement.copyElementTree(inputMQMDElement);
  currentOutputRecord.copyElementTree(currentInputRecord);   

  //Propagate this message, requesting that the output message assembly be cleared after propagation.
  out.propagate(outAssembly, true);
  //Note: You do not need to call clearMessage on outputMessage because it was cleared after the propagation.

  //Take a reference to the current record so that it can be deleted.
  MbElement previousInputRecord = currentInputRecord;
  //Now move to the next input record ready for processing.
  currentInputRecord = currentInputRecord.getNextSibling();
  //Now that we have moved to the next sibling, delete the input record that has already been processed.
  previousInputRecord.delete();

}

This Java code produces a message for each repeating Record subtree in the input XMLNSC message:

<Record>
  <Field1>A</Field1>
  <Field2>B</Field2>
  <Field3>C</Field3>
  <Field4>D</Field4>
  <Field5>E</Field5>
</Record>

bc23802_.htm |

Last updated Friday, 21 July 2017