Message tree copying

When you create a changed output message, the message flow needs to copy the message tree. Nodes such as the Compute node, mapping nodes, and DecisionService node have input and output trees where the output trees are often different from the input trees. In other language nodes such as the JavaCompute node, Java plugin nodes, and .NET nodes, one or many output messages can be created. When you consider memory usage and performance in general, you must take care to determine when the message tree is copied.

When a tree is copied between different domains, the whole of the input tree must be parsed so that it can be copied to the new domain. This behavior is referred to as an unlike parser copy. Each individual field in the source message must be individually transferred to the target domain so that the correct structure and field types apply.

SET OutputRoot = InputRoot;
SET OutputRoot.MQMD = InputRoot.MQMD;
SET OutputRoot.XMLNSC = InputRoot.XMLNSC;

SET OutputRoot = InputRoot;
SET Output.Properties.CorrelId = X'010203040506070809101112131415161718192021222324';

However, if the codepage changes during a routing, then an MQInput node must inflate the output tree to reserialize the new bitstream.

SET OutputRoot = InputRoot;SET
Output.Properties.CodedCharSetId = 500;

The MQInput node parses the output tree that it was passed. But it is important to know that even in this scenario, the original input tree that is received on the input node is not parsed. The high-speed bitstream copy for an unchanged message tree can be useful, and in fact large message handling techniques depend on this process.

In a Compute node, the InputRoot tree is not modifiable. So this technique relies on a high-speed bitstream copy to transfer the tree to the OutputRoot, such that the InputRoot is never inflated.

Do not change a message tree, and then copy it unnecessarily in other nodes.

Compute1: 
 - SET OutputRoot = InputRoot;
 - SET OutputRoot.XMLNSC.TestCase.LastUpdated = CURRENT_TIMESTAMP;
Compute2:
 - SET OutputRoot = InputRoot;
 - INSERT INTO Database.myDB .......

The general advice that is given is for the message flow developer to ensure that all the updates are done to the message tree in the same node. However, in a flow where subflows are written by other message flow developers, then this behavior might not always be possible.

Therefore, in this type of scenario it would be better for the second Compute node Compute2 not to issue the SET OutputRoot = InputRoot command, and instead set the ComputeMode property so that it is not Message.These settings mean that the node is not going to propagate a new/changed OutputRoot, and as such propagates the one it received.

The same technique can be used if Compute2 were updating the LocalEnvironment with routing information. Here the ComputeMode property must be set to LocalEnvironment.

When a message tree is copied, the high-speed bitstream transfer takes place only at the parser folder level. So SET OutputRoot.XMLNSC = InputRoot.XMLNSC considers creating the target XMLNSC folder using a bitstream copy as shown in the following example:

SET OutputRoot.XMLNSC.TestCase = InputRoot.XMLNSC.TestCase

SET Environment.Variables.XMLNSC = InputRoot.XMLNSC;

As the target field is in the Environment tree, an XMLNSC parser is not automatically created in the target, and as such an unlike parser copy takes place. This process means that the bitstream copy cannot take place, and the whole of the source tree is parsed and then copied to the Environment tree. The consequence of this behavior is that any XML-specific field types are lost. For example, if the Environment tree were copied back to the OutputRoot tree then any attributes would now be tags.

CREATE FIELD Environment.Variables.XMLNSC DOMAIN('XMLNSC');
SET Environment.Variables.XMLNSC = InputRoot.XMLNSC;

The same is true of the other language APIs when message trees are copied. The target element must be created specifying an owning parser before the copy takes place. Although ESQL OutputRoot has the special behavior such that the SET statement creates parsers for the direct children of OutputRoot, it is not true if OutputRoot is passed into a PROCEDURE/FUNCTION as a row variable. At this point, the ROW variable representation of OutputRoot no longer has the behavior of OutputRoot. This behavior is why OutputRoot is a globally recognized correlation name throughout all ESQL procedures and functions, and as such does not need to be passed in as a ROW variable.

When a message tree is copied, any existing children of target message tree field are detached first. Detaching fields is not the same as deleting fields. Detaching fields means that they are still in scope, but are no longer attached to the tree. As such, the detached fields cannot be reused by the parser, and might cause more message tree fields to be created than intended.

DECLARE inputRef REFERENCE TO InputRoot.DFDL.Parent.RepeatingRecord[1];
WHILE LASTMOVE(inputRef) = TRUE DO
  SET OutputRoot.XMLNSC.TestCase = inputRef
  DECLARE recordBytes BLOB ASBITSTREAM(OutputRoot.XMLNSC.TestCase)
  INSERT INTO Database.myDB(Records) VALUES(recordBytes);
  MOVE inputRef NEXTSIBLING NAME 'RepeatingRecord';
END WHILE;

DECLARE inputRef REFERENCE TO InputRoot.DFDL.Parent.RepeatingRecord[1];
WHILE LASTMOVE(inputRef) = TRUE  DO
  SET OutputRoot.XMLNSC.TestCase = inputRef
  DECLARE recordBytes BLOB ASBITSTREAM(OutputRoot.XMLNSC.TestCase)
  -- Record is now serialized, delete the Output fields we no longer need;
  DELETE FIELD OutputRoot.XMLNSC.TestCase;
  INSERT INTO Database.myDB(Records) VALUES(recordBytes);
  MOVE inputRef NEXTSIBLING NAME 'RepeatingRecord';
END WHILE;

In some message flow implementations, it might be necessary to ensure that the message tree fields are copied instead of the bitstream copy taking place.

SET OutputRoot.XMLNSC.*[] = InputRoot.XMLNSC.*[]

Transformation nodes can produce an entirely different message from the original that was on an input terminal. For this reason, InputRoot and OutputRoot references are provided so that the different message trees can be accessed.

SET OutputRoot = InputRoot;
SET OutputRoot.XMLNSC.TestCase.myField = 'An updated field';

DECLARE inputRef REFERENCE TO InputRoot;
SET inputRef.XMLNSC.TestCase.myField = 'An updated field';

For this ESQL, the ComputeMode property must be changed so that the original input tree is propagated. Although updating the Input trees directly is efficient, care must be taken that this behavior does not affect upstream node processing in the message flow. If a constant input tree is propagating multiple times to a subflow, then the message flow ensures that the input tree is unchanged. This ESQL example means the input tree that was propagated would get modified, and so all other propagations see the updated value.

Although XML, XMLNS, and XMLNSC are all XML domains, they are not the same domain when you copy a message tree. Therefore, the efficient bitstream transfer does not take place when the tree is copied between these domains with SET OutputRoot.XMLNSC = InputRoot.XML; for example.

Both XML and XMLNS are deprecated and should be used only for legacy message flows.

Sometimes you might need to use a Compute node, but do not need to change the tree. In these situations the Compute node should be configured to use only LocalEnvironment, which avoids a tree copy for the payload.

In the following example, a Compute node is used to inspect a version field in the header of a payload, and then propagates to a label with the content of the version:

Figure 1.

To further improve performance, the Input node parses only the header portion of the payload, the body is configured for opaque parsing. A full parse of the body is then performed in the subsequent nodes as needed.

DECLARE rVersion REFERENCE TO InputRoot.XMLNSC.ns:Inventory.ns:header.ns:version;
PROPAGATE TO LABEL(rVersion);
RETURN FALSE;