How the message tree is populated

The message tree is initially populated by the input node of the message flow.

When the input node receives the input message, it creates and completes the Properties tree (the first subtree of the message tree).

The node then examines the contents of the input message bit stream and creates the remainder of the message tree to reflect those contents. This process depends to some extent on the input node itself, which is governed by the transport across which the message is received:

WebSphere® MQ Enterprise Transport protocol
If your application communicates with the integration node across these protocols, and your message flow includes the corresponding MQInput node, all messages that are received must start with a Message Queue Message Descriptor (MQMD) header. If a valid MQMD is not present at the start of the message, the message is rejected, and no further processing takes place.

The input node first invokes the MQMD parser and creates the subtree for that header.

A message can have zero or more additional headers following the MQMD. These headers are chained together, with the Format field of one header defining the format of the following header, up to and including the last header, which defines the format of the message body. If an MQRFH and an MQRFH2 header exist in the chain, the name and value data in either of these two headers can also contain information about the format of the following data. If the value that is specified in Format is a recognized parser, this value always takes precedence over the name and value data.

The integration node invokes the appropriate parser to interpret each header, following the chain in the message. Each header is parsed independently. The fields in a single header are parsed in an order that is governed by the parser. You cannot predict the order that is chosen, but the order in which fields are parsed does not affect the order in which the fields are displayed in the header.

The integration node ensures that the integrity of the headers that precede a message body is maintained. The format of each part of the message is defined, either by the Format field in the immediately preceding header (if the following part is a recognized WebSphere MQ format), or by the values that are set in the MQRFH or MQRFH2 header:

  • The format of the first header is known because it must be MQMD.
  • The format of any subsequent header in the message is set in the Format field in the preceding header.
  • The format of the body corresponds to the message domain and the parser that must be invoked for the message body (for example, XMLNSC). This information is set either in the MQRFH or MQRFH2 header, or in the Message Domain property of the input node that receives the message.

This process is repeated as many times as required by the number of headers that precede the message body. You do not need to populate these fields yourself; the integration node handles this sequence for you.

The integration node completes this process to ensure that Format fields in headers correctly identify each part of the message. If the integration node does not complete this process, WebSphere MQ might be unable to deliver the message. The message body parser is not a recognized WebSphere MQ header format, therefore the integration node replaces this value in the last headers Format field with the value MQFMT_NONE. The original value in that field is stored in the Domain field in the MQRFH or MQRFH2 header to retain the information about the contents of the message body.

For example, if the MQRFH2 header immediately precedes the message body, and its Format field is set to XMLNSC, which indicates that the message body must be parsed by the XMLNSC parser, the MQRFH2 Domain field is set to XMLNSC, and its Format field is reset to MQFMT_NONE.

These actions might result in information that is stored explicitly by an ESQL or Java™ expression being replaced by the integration node.

The CodedCharSetId and Encoding fields are not populated in the same way as the Format field. In particular, the message body is not used to determine the CodedCharSetId and Encoding values. Rather, these values affect the way in which the message body is written. This can cause unexpected results if an intermediate header (for example MQRFH2) is removed without updating the preceding header in the chain with the removed header values.

When all the headers have been parsed, and the corresponding sub-trees have been created in the message tree, the input node associates the specified parser with the message body. Specify the parser that is to be associated with the message body content, either in a header in the message (for example, the <mcd> folder in the MQRFH2 header), or in the input node properties (if the message does not include headers). The input node makes the association as described in the following list:

  • If the message has an MQRFH or MQRFH2 header, the domain that is identified in the header (either in Format or the name and value data) determines the parser that is associated with this message.
  • If the message does not have an MQRFH or MQRFH2 header, or if the header does not identify the domain, the Message Domain property of the input node indicates the domain of the message, and the parser that is to be used. You can specify a user-defined domain and parser.
  • If the message domain cannot be identified by header values or by the Message Domain property of the input node, the message is handled as a binary object (BLOB). The BLOB parser is associated with the message. A BLOB can be interpreted as a string of hexadecimal characters, and can be modified or examined in the message flow by specifying the location of the subset of the string.

By default, the message body is not parsed straight away, for performance reasons. The message body might not need to be parsed during the message flow. It is parsed only when a reference is made to its contents.

For example, the message body is parsed when you refer to a field in the message body, for example: Root.XMLNSC.MyDoc.MyField. Depending on the paths that are taken in the message flow, this can take place at different points. This parsing when first needed approach is also referred to as "partial parsing" or "on-demand parsing", and in typical processing does not affect the logic of a message flow. However, there are some implications for error handling scenarios; see Handling errors in message flows.

If you want a message flow to accept messages from more than one message domain, include an MQRFH2 header in your message from which the input nodes extract the message domain and related message definition information (message set, message type, and message format).

If you set up the message headers or the input node properties to identify a user-defined domain and parser, the way in which it interprets the message and constructs the logical tree might differ from that described here.

WebSphere Broker File Transport, WebSphere Broker Adapters Transport, WebSphere MQ Web Services Transport, and WebSphere Broker JMS Transport protocols
If your application communicates with the integration node across these supported protocols, and your message flow includes the corresponding input nodes, messages that are received do not have to include a particular header. If recognized headers are included, the input node invokes the appropriate parsers to interpret the headers and to build the relevant parts of the message tree, as described for the other supported protocols.

If there are no headers, or these headers do not specify the parser for the message body, set the input node properties to define the message body parser. If you do not set the node properties in this way, the message is treated as a BLOB. You can specify a user-defined parser.

The specified parser is associated with the message body by the input node (in the same way as it is for the WebSphere MQ Enterprise Transport protocol), and by default the message body is not parsed immediately.

If you set up the message headers or the input node properties to identify a user-defined domain and parser, the way in which it interprets the message and constructs the logical tree might differ from that described here.

All other protocols
If you want your message flow to accept messages from a transport protocol for which IBM® Integration Bus does not provide built-in support, or you want it to provide some specific processing on receipt of a message, use either the Java or the C language programming interface to create a new user-defined input node.

This interface does not automatically generate a Properties subtree for a message. A message does not need to have a Properties subtree, but you might find it useful to create one to provide a consistent message tree structure, regardless of input node. If you are using a user-defined input node, you must create a Properties subtree in the message tree yourself.

To process messages that do not conform to any of the defined message domains, use the C language programming interface to create a new user-defined parser.

Refer to the node interface to understand how it uses parsers, and whether you can configure it to modify its behavior. If the node uses a user-defined parser, the tree structure that is created for the message might differ slightly from that created for built-in parsers. A user-defined input node can parse an input message completely, or it can participate in partial parsing in which the message body is parsed only when it is required.

You can also create your own output and message processing nodes in C or Java.

Properties versus MQMD folder behavior for various transports

Differences exist in the way the Properties folder and the MQMD folder are treated with respect to which folder takes precedence for the same fields. This treatment is characterized by the transport type (for example, HTTP or WebSphere MQ) that you use.

When the message flow is sourced by an MQInput node, you have an MQMD to parse. In this case, the Properties folder is sourced by the MQMD values and so the MQMD folder takes precedence over the Properties folder in terms of value propagation between the folders. This scenario means that you can perform ESQL, for example, SET OutputRoot.MQMD.CorrelId and this command updates the Properties folder value.

When the message flow is sourced from an input node that is not the MQInput node (such as the HTTPInput node or a user-defined input node), no MQMD is parsed. In this scenario, the Properties folder is not sourced from an input MQMD; it is created and populated with transport values that come from other transport specific headers. When you create an MQMD folder in a message flow that was not sourced from the WebSphere MQ transport, the MQMD header does not take precedence over the Properties folder because the WebSphere MQ transport did not populate the Properties folder. Therefore, in this case, the Properties folder overwrites any values in MQMD.

The Properties folder is constructed and represents a message received on the transport. In this scenario two entirely different transports are being used which have different meanings and, therefore, different requirements of the Properties folder. When sourced from an HTTPInput node, the HTTP headers have control over the Properties folder for like fields. When sourced from an MQInput node the MQMD has control over the Properties folder for like fields.

Therefore, when you add an MQMD folder to a tree that was created by the HTTP Transport, this MQMD folder does not have control over the Properties folder, and the value propagation direction is not MQMD to Properties, it is Properties to MQMD. The correct approach is to set the replyIdentifier field of the Properties folder and to use it to populate the MQMD: 
 SET OutputRoot.Properties.ReplyIdentifier = X' ..... ';
The behavior is not unique to just the CorrelId to ReplyIdentifier fields, but applies to the following fields between the MQMD and Properties folder:
  • CorrelId
  • Encoding
  • CodedCharSetId The Properties.CodedCharSetId represents the CodedCharSetId for the message payload, which matches the CodedCharSetId in any message header that immediately precedes the message payload.
  • Persistence
  • Expiry
  • Priority
In summary:
  1. When your message flow is sourced by an MQInput node, the MQMD takes precedence over the Properties folder in terms of value propagation between the folders.
  2. When your message flow is sourced from an input node that is not the MQInput node (such as the HTTPInput node or a user-defined input node), the MQMD header does not take precedence over Properties folder .
  3. When an MQMD folder is added in a tree that was created by the HTTP Transport, this MQMD does not have control over the Properties folder and the value propagation direction is not MQMD to Properties; it is Properties to MQMD.

Example

SET OutputRoot.Properties = InputRoot.Properties;
SET OutputRoot.MQMD.Version = 2;
SET OutputRoot.MQMD.CorrelId = X'4d454e53414a453320202020202020202020202020202020';
SET OutputRoot.MQMD.Encoding = 785;
SET OutputRoot.MQMD.CodedCharSetId = 500;
SET OutputRoot.MQMD.Persistence = 1;
SET OutputRoot.MQMD.Expiry = 10000;
SET OutputRoot.MQMD.Priority = 9;
SET OutputRoot.BLOB.BLOB = X'01';
When sourced from an HTTPInput node none of these changes take effect and the MQMD output from the Compute node is:
(0x01000000):MQMD       = (
    (0x03000000):Version        = 2
    (0x03000000):CorrelId       = X'000000000000000000000000000000000000000000000000'
    (0x03000000):Encoding       = 546
    (0x03000000):CodedCharSetId = 1208
    (0x03000000):Persistence    = FALSE
    (0x03000000):Expiry         = -1
    (0x03000000):Priority       = 0