Aggregation is a powerful and important enterprise service bus (ESB) pattern. An inbound
request might map into several individual outbound requests. The responses from these requests can
be used to enrich a final request to a service or can be aggregated into a single response for the
original request.
Common scenarios are:
A car comparison insurance quote system where a single request is submitted to an ESB that
causes multiple insurance service providers to be queried before the best quote is returned.
A holiday booking request arrives at the ESB and needs to be forwarded to separate hotel, flight
and car hire booking systems, before a combined response can be returned.
A store ordering system, where a salesperson can submit a request for a number of orders and
have them dispatched. The original message coming into the ESB identifies the customer for each
order via a customer ID. So, the order request message needs to be enriched with each customer's
shipping details so that the dispatch step has all the information it needs to succeed.
When creating aggregation solutions, there are several core parts of the programming model that
you must understand before going into the details of the patterns:
The Fan Out mediation primitive
The Fan Out context
The Fan In mediation primitive
The Service Invoke mediation primitive
Aggregation block
Shared context
Special considerations
Fan Out mediation primitive
The Fan Out mediation primitive is normally used
at the start of an aggregation. It provides the ability to either branch (Once mode) the mediation
flow, or fire its output terminals multiple times (Iterate mode), based on the size of a repeating
structure within the Service Message Object (SMO). For example, if the inbound message contains a
structure that needs to be forwarded onto two services, you can branch the flow at the Fan Out
mediation primitive. To call a service multiple times, each with individual requests from a
repeating structure, you can use the Iterate mode of the Fan Out mediation primitive. You can
configure the two modes from the Fan Out mediation primitive properties view, as shown in Figure 1. Figure 1. The Fan Out mediation primitive properties view
The mediation primitive is configured for Once mode by default, but this can be changed by
selecting the for each element in XPath expression button and specifying the
location of the repeating structure within the SMO.
Fan Out context
The Fan Out context is used to store the current item of a repeating element when Fan Out is used
in Iterate mode. The Fan Out context contains an integer field containing the index of the
occurrence of the repeating element along with an element containing the occurrence itself. For
example, if there was a repeating element of an element called order, the Fan Out
context would look like this:
The Fan In mediation primitive is a partner to
the Fan Out mediation primitive. The Fan Out mediation primitive is used to decide how many messages
to generate, whereas the Fan In mediation primitive is used to decide when the aggregation can be
considered complete. The Fan In mediation primitive is always paired with a Fan Out mediation
primitive in the same flow (for example, a Fan Out/Fan In mediation primitive pairing cannot expand
across a request/response flow boundary), and the Fan In mediation primitive acts as a decision
point for when to leave the aggregation block. The Fan In mediation primitive receives a number of
messages until a decision point is reached, at which point the last message that was received is
propagated to the output terminal. The Fan In mediation primitive can only be used in combination
with the Fan Out mediation primitive.
Three types of decision points for the Fan In mediation
primitive are supported:
Simple count: the output terminal is fired when a set number of messages is received at
the input terminal. However, more messages might be sent out by the Fan Out mediation primitive,
because the Fan In mediation primitive count decision point can be reached more than once, resulting
in multiple firings of the output terminal.
XPath decision: the output terminal is fired if an XPath expression evaluation of the
incoming message evaluates to true. It is possible for the output terminal to
fire multiple times.
Iterate: the Fan In mediation primitive waits to receive all messages produced by the
corresponding Fan Out mediation primitive. All of the occurrences in the repeating element have been
processed.
Figure 2 shows these three options, which are
available in the Fan In mediation primitive properties view.
Figure 2. The Fan In mediation primitive properties view
The Fan In mediation primitive has two input terminals (in and
stop), two output terminals (out and
incomplete) and a fail terminal (fail). The input
terminals are wired to accept a message and the other terminals are wired to propagate a message.
The in terminal accepts input messages until a decision point is
reached. When a decision point is reached the out terminal is fired. Although
the out terminal is fired when a decision point is reached, the
in terminal can still accept input messages, under some circumstances. For
example, if the count value is reached, the out terminal is fired. However,
if the Fan Out mediation primitive is still sending messages, the Fan In mediation primitive can
still receive messages.
You can stop a Fan In operation by wiring the preceding mediation
primitive to the Fan In stop terminal. The stop
terminal causes the incomplete output terminal to be fired, and this stops
the associated Fan Out mediation primitive from sending any more messages. The
incomplete output terminal is also fired if a timeout occurs. Table 1 describes terminal behavior
further.
Service Invoke mediation primitive
The Service Invoke mediation primitive is
used between the Fan Out and Fan In mediation primitives to call the services required. The Service
Invoke mediation primitive is discussed in the Invocation of Services
chapter.
Aggregation block
When you create an aggregation solution, the Fan Out mediation primitive declares the start of
the aggregation and a Fan In mediation primitive symbolizes the end of the aggregation. The logic
between these two mediation primitives can be considered a repeating block if the Fan Out is
configured in Iterate mode, or branching if configured in Once mode, as shown in Figure 3
.Figure 3. Example of an aggregation block
Shared context
There are three user-defined data store contexts within the
SMO:
Data stored in the Transient context is available within either a request or response flow.
Data stored in the Correlation context is available in both the request and response flows.
Data stored in the Shared context is available in all branches and executions of an aggregation
block.
To highlight the difference Figure 4 shows an
example.Figure 4. Shared context example
In the case of this example, the SMOs passing through branch 1 and 2, and therefore the
Transient and Correlation contexts, are different copies. This means that any changes made to the
SMO in one branch are not made to the other branch. The Shared context area of the SMO has been
designed for sharing aggregation data between the different branches of the flow inside an
aggregation block. Access to the Shared context is only available inside the flow which contains the
aggregation block that uses it.
Aggregation mediation programming model special considerations
When using
aggregation capabilities consider these points:
The Shared context is part of the SMO, and is designed for use inside aggregation blocks because
the data put into it is available in all the branches of a flow. The design of the Shared context
structure is essential to a successful aggregation design. The goal is to store the response from
each service invocation into a structure that can then be converted into a single response message
after the aggregation block.
The Shared context is shared across SMOs and should be considered read-only outside the
aggregation block.
At development time it is difficult to determine which SMO body type will be emitted from the
Fan In mediation primitive output terminal. Mediation primitives after the Fan In mediation
primitive should be used to generate a new message body using the information in the Shared context.
If a message arrives at a Fan Out mediation primitive, one or more output terminals of the Fan
In mediation primitive will fire (assuming no ServiceRuntimeExceptions occur within the aggregation
block, for example, as a result of running a Fail mediation primitive), even if a message is not
received by the Fan In mediation primitive. Table 1 highlights this behavior.
Note: In some circumstances it is possible that none of the messages
sent by the Fan Out mediation primitive arrive at the corresponding Fan In mediation primitive.
Using Figure 4 to illustrate, consider this example:
if the second output terminal on primitives branch 1 and branch 2 are fired, this does not cause the
messages to arrive at the Fan In mediation primitive, because there is no wired path. This is
represented in Table 1 by the message result
No.
Table 1. Which output terminal is run from the Fan In mediation primitive
Fan In mediation primitive mode
Message arrives at Fan In mediation primitive
Output terminal fired
SMO of Fan In mediation primitive
Iterate
No
Complete
Message body empty; SMO headers and context are the same as the message
received at the Fan Out
Iterate
Yes (in terminal)
Complete
Last message received at the Fan In mediation primitive input terminal
Iterate
Yes (stop terminal)
Incomplete
Last message received at the Fan In mediation primitive input terminal
Simple count/XPath
No
Incomplete
Message body empty; SMO headers and context are the same as the message
received at the Fan Out
Simple count/XPath
Yes (in terminal)
Incomplete/Complete *
Last message received at the Fan In mediation primitive input terminal
Simple count/XPath
Yes (stop terminal)
Incomplete
Last message received at the Fan In mediation primitive input terminal
Iterate/Simple count/XPath
Yes, timeout value expired and decision type not complete
Incomplete
Last message received at the Fan In mediation primitive input terminal
Iterate/Simple count/XPath
Yes, timeout value expired and decision type complete
Complete
Last message received at the Fan In mediation primitive input terminal
* Depends on if the condition (Simple count or XPath) evaluated to
true for the last message that was received
