Creating the aggregation fan-out flow
The aggregation fan-out flow receives the initial input message and restructures it to present a number of requests to a number of target applications.
Before you begin
- For background information, see Message flow aggregation.
About this task
You can include the fan-out and fan-in flow in the same message flow. However, you might prefer to create two separate flows. For more information about the benefits of configuring separate message flows, see Associating fan-out and fan-in aggregation flows.
To create the fan-out flow:
- Create a message flow to provide the fan-out processing.
For more information, see Creating a message flow.
- Add the following nodes to the message flow, then configure
and connect them as described.
- Input node
- The input node receives an input message from which multiple request
messages are generated. This node can be any one of the built-in nodes,
or a user-defined input node.
- Select the input node to open the Properties view. The node properties are displayed.
- Specify the source of input messages for this node. For example, specify the name of a IBM® MQ queue in the Queue name property. The MQInput node retrieves messages from this queue.
- Optional: set values for any other properties that you want to configure for this node. For example, to ensure that aggregate request messages are put under sync point, set the Transaction mode property to Yes. This option avoids the situation where the AggregateReply node receives response messages before it has received the control message that informs it of the aggregation instance. Putting the fan-out flow under transactional control ensures that the fan-out flow completes before any response messages get to the AggregateReply.
- Connect the input node's Out terminal to the In terminal of an AggregateControl node. This option represents the simplest configuration; if appropriate, you can include other nodes between the input node and the AggregateControl node. For example, store the request for audit purposes (in a Database node), or add a unique identifier to the message (in a Compute node).
- Optional: if your fan-out and fan-in flows are combined in one message flow, modify the Order mode property on the Advanced tab. Select the By Queue Order option and ensure that the Logical Order property is also selected. These options force the input node to be single threaded to maintain the logical order of the messages that arrive on the queue. Any additional instance threads that you make available are then shared among only the fan-in input nodes to improve the performance of aggregation. If your fan-in and fan-out flows are in separate message flows this step is not required because you can make additional threads available specifically to the fan-in flow.
- AggregateControl node
- The AggregateControl node
updates the local environment that is associated with the input message
with information required by the AggregateRequest node. The AggregateControl node
creates the LocalEnvironment.ComIbmAggregateControlNode folder.
This folder and the fields in it are for internal use by IBM Integration Bus and you should not rely on
their existence or values when developing your message flows.
- Select the AggregateControl node to open the Properties view. The node properties are displayed.
- Set the Aggregate name property of the AggregateControl node to identify this particular aggregation. It is used later to associate this AggregateControl node with a specific AggregateReply node. The Aggregate name that you specify must be contextually unique in an integration node.
- Optional: set the Timeout property to specify how long the integration node waits for replies to arrive before taking action (described in Setting timeout values for aggregation). If a timeout is not set on the AggregateControl node then aggregate requests stored internally will not be removed unless all aggregate reply messages return. This situation might lead to a gradual build-up of messages on the internal queues. To avoid this situation, set the timeout to a value other than zero (zero means that a timeout never occurs) so that when the timeout is reached the requests are removed and the queues do not fill up with redundant requests. Even if timeouts are not required or expected, it is good practice to set the timeout value to a large value; for example, 86400 seconds (24 hours) so that the queues occasionally get cleared of old aggregations.
- Connect the Out terminal of the AggregateControl node to the In terminal of one or more Compute nodes that provide the analysis and breakdown of the request in the input message that is propagated on this terminal.
- Compute node
- The Compute node extracts
information from the input message and constructs a new output message.
If the target applications that handle the subtask requests can extract the information that they require from the single input message, you do not need to include a Compute node to split the message. You can pass the whole input message to all target applications.
If your target applications expect to receive an individual request, not the whole input message, you must include a Compute node to generate each individual subtask output message from the input message. Configure each Compute node in the following way, copying the appropriate subset of the input message to each output message:
- Select the Compute node to open the Properties view. The node properties are displayed.
- Select a value for the Basic property Compute mode. This property specifies
which sections of the message tree are modified by the node. The AggregateControl node inserts
elements into the local environment tree in the input message that
the AggregateRequest node
reads when the message reaches it. Ensure that the local environment
is copied from the input message to the output message in the Compute node. This configuration
happens automatically unless you specify a value that includes local
environment (one of All, LocalEnvironment, LocalEnvironment and Message,
or Exception and LocalEnvironment).
If you specify one of these values, the integration node assumes that you are customizing the Compute node with ESQL that writes to local environment, and that you are copying any elements in that tree that are required in the output message.
To modify the local environment, add the following statement to copy the required aggregate information from input message to output message:
SET OutputLocalEnvironment.ComIbmAggregateControlNode = InputLocalEnvironment.ComIbmAggregateControlNode;
- Optional: set values for any other properties that you want to configure for this node.
- Connect the Out terminal of each Compute node to the In terminal of the output node that represents the destination of the output request message that you have created from the input message in this node.
- Output node
- Include an output node for each output message that you generate
in your fan-out flow. Configure each node, as described later in this
section, with the appropriate modifications for each destination.
The output node must be an output node that supports the request/reply model, such as an MQOutput node, or a mixture of these nodes (depending on the requirements of the target applications).
- Select the output node to open the Properties view. The node properties are displayed.
- Specify the destination for the output messages for this node. For example, specify the name of a IBM MQ queue in the Queue name property to which the MQOutput node sends messages. The target application must process its request, and send the response to the reply destination indicated in its input message (for example the IBM MQ ReplyToQueue).
- Click Request in the left view and set values for these properties to specify that replies are sent to the input queue of the fan-in flow.
- Optional: set values for any other properties that you want to configure for this node.
- Connect the Out terminal of the output node to the In terminal
of an AggregateRequest node.
When the message is propagated through the output node's Out terminal,
the built-in output node updates the WrittenDestination folder in
the associated local environment with additional information required
by the AggregateRequest node.
The information written by the built-in nodes is queue name, queue manager name, message ID, and correlation ID (from the MQMD), and message reply identifier (set to the same value as message ID).
- AggregateRequest node
- Include an AggregateRequest node
for each output message that you generate in your fan-out flow.
- Select the AggregateRequest node to open the Properties view. The node properties are displayed.
- Set the Basic property Folder name to a value that identifies the type of request that has been sent out. This value is used by the AggregateReply node to match up with the reply message when it is received in the fan-in flow. The folder name that you specify for each request that the fan-out flow generates must be unique.
The AggregateRequest node writes a record in IBM MQ for each message that it processes. This record enables the AggregateReply node to identify which request each response is associated with. If your output nodes are non-transactional, the response message might arrive at the fan-in flow before this IBM MQ update is committed. For details on how you can use timeouts to avoid this situation, see Setting timeout values for aggregation.CAUTION:Although the use of timeouts can help to avoid this situation described previously, configure your fan-out flow to be transactional. Therefore, response messages cannot get to the fan-in flow before the corresponding AggregateRequest nodes have committed their IBM MQ updates.
- To save the message flow and validate its configuration, press Ctrl-S or click .