Aggregation design patterns and performance considerations in WebSphere Enterprise Service Bus V7.5

This article describes the aggregation design patterns available for WebSphere ESB mediation flows, and the performance considerations you need to take into account when developing and deploying aggregation solutions.

Rachel Norris (rachel_norris@uk.ibm.com), Performance Analyst, IBM

Photo of Rachel NorrisRachel Norris is a Performance Analyst on the WebSphere ESB team at the IBM Hursley Lab in the United Kingdom. She has worked in software development at Hursley for 25 years, specializing in performance analysis for the last five years. She holds a degree in Engineering from Cambridge University. You can contact Rachel at rachel_norris@uk.ibm.com.



Martin Ross (martin.ross@uk.ibm.com), Performance Analyst, IBM China

Photo of Martin RossMartin Ross is a Performance Analyst on the WebSphere ESB team at the IBM Software Lab in Hursley Park, United Kingdom. He has worked on WebSphere ESB since 2006 and has a degree in Software Engineering from the University of Southampton. You can contact Martin at <a href="mailto:martin.ross@uk.ibm.com">martin.ross@uk.ibm.com</a>.



16 November 2011

Also available in Chinese

Introduction

Aggregation is an important enterprise service bus (ESB) pattern. It enables a single inbound request to map into multiple outbound service invocations, the responses from which can be aggregated into a single response to the original request. But before you develop mediations using aggregation design patterns with IBM® WebSphere® ESB and IBM Integration Developer, you need to take several performance considerations into account. This article describes those performance considerations so that you can create an effective and efficient aggregation solution.

Mediation primitives

This section describes the core mediation primitive components that are used in the development of aggregation design patterns.

Fan Out mediation primitives

The Fan Out mediation primitive is used at the start of an aggregation block. Two modes are currently supported:

  • Once mode -- Provides the ability to branch the mediation flow
  • Iterate mode -- Provides the ability to iterate over a repeating structure within the service message object (SMO)

You should use Once mode to branch the mediation flow for aggregation design patterns where, for example, and inbound message contains a structure that needs to be forwarded on to two services. Use the Iterate mode for aggregation design patterns where a service needs to be invoked multiple times based upon a repeating structure within the SMO.

Figure 1. Fan Out mediation primitive
Fan Out mediation primitive

The primitive is configured for Once mode by default. When developing an aggregation pattern using the Iterate mode, select For each element in XPath expression and specify the location of the repeating structure within the SMO. The following XPath defines the location of the Fan Out context within the SMO:

/context/primitiveContext/FanOutContext

The Fan Out context is used in Iterate mode to store the current occurrence of the repeating structure and the index of the occurrence within the SMO.

Fan In mediation primitives

The Fan In mediation primitive is used at the end of an aggregation block, and partners with a corresponding Fan Out primitive. Fan In supports three types of decision points that control when the output terminal of the Fan In primitive is fired. When the specified decision point has been met, the last message that was received is propagated to the output terminal. Here are the three types of decision points:

  • Simple count -- Defines the number of messages to be received at the input terminal before firing the output terminal.
  • XPath decision -- Output terminal is fired when the XPath expression evaluates to true.
  • Iterate -- Output terminal is not fired until the input terminal has received all messages produced by the corresponding Fan Out mediation primitive.

The XPath decision and Simple count decision points may be met more than once in an aggregation solution, resulting in the output terminal being fired multiple times. The Iterate decision point causes the output terminal to be fired only once.

Figure 2. Fan In mediation primitive
Fan In mediation primitive

Service Invoke mediation primitives

The Service Invoke mediation primitive is used between the Fan Out and Fan In mediation primitives to invoke required services required. Additional functionality was introduced in WebSphere ESB V7.5 and IBM Integration Developer V7.5. You can select the new Message Enrichment Mode when you add the Service Invoke mediation primitive to the canvas in IBM Integration Developer, as shown below:

Figure 3. Service Invoke mediation primitive -- selecting Message Enrichment Mode
Service Invoke mediation primitive -- selecting Message Enrichment Mode

When Message Enrichment Mode is selected, the additional properties shown below are made available to the Service Invoke mediation primitive within IBM Integration Developer:

Figure 4. Service Invoke mediation primitive -- Message Enrichment Mode properties
Service Invoke mediation primitive -- Message Enrichment Mode properties

An XPath statement is defined for the Input parameter to identify the location of the structure within the input SMO to be used to invoke the defined service. A second XPath statement is defined for the Output parameter to identify the location of the structure within the output SMO to be used to store the response. This functionality is applicable only if the structures required for the request and response for the given service are available within the inbound and outbound messages and are addressable by single XPath statements.

Design patterns

This section describes two design patterns that correlate to the two modes of operation supported by the Fan Out mediation primitive:

  • Branching design pattern
  • Iterate design pattern

Branching design pattern

This section describes the differences and advantages of the new Message Enrichment Mode of the Service Invoke mediation primitive.

The following example of a Branching aggregation design pattern describes a banking scenario where two services are invoked for a single inbound request. The inbound request contains information pertaining to a given account, and a Message Element Setter mediation primitive stores a RequestID field in the transient context of the SMO prior to the request entering the aggregation block. The Fan Out mediation primitive is configured in Once mode and the mediation flow is split into two branches: the first branch invokes a service to retrieve a summary of the account information, and the second branch invokes a service to retrieve the transaction history of the account.

Prior to the invocation of each service, a BOMapper mediation primitive converts the message to the correct format, and a BOMapper mediation primitive immediately following each Service Invoke mediation primitive then stores the given responses in the shared context of the SMO. The output terminal of the Fan In mediation primitive is fired when the two responses are received on the input terminals, and a final BOMapper mediation primitive aggregates the information stored within the shared context and transient context to create a response to the initial request:

Figure 5. Branching aggregation design pattern
Branching aggergation design pattern

Service Invoke Message Enrichment Mode

New functionality in WebSphere ESB V7.5 and IBM Integration Developer V7.5 enables you to simplify the Branching design pattern. Using the new Service Invoke functionality in the Branching aggregation design pattern removes the need for the BOMapper mediation primitives on either side of the Service Invoke mediation primitives, as shown below:

Figure 6. Branching aggregation design pattern -- Service Invoke Message Enrichment Mode
Branching aggregation design pattern -- Service Invoke Message Enrichment Mode

Integrating this functionality within the Service Invoke mediation primitive has the added benefit of a slight improvement in runtime performance by removing the overhead from the additional BOMapper mediation primitives otherwise required.

Iterate design pattern

The following example of an Iterate aggregation design pattern describes a banking scenario where a service is invoked multiple times for a single inbound request. The inbound request contains a repeating structure, each occurrence of which contains information pertaining to a particular account for a customer, and a Message Element Setter mediation primitive stores a RequestID field in the transient context of the SMO prior to the request message entering the aggregation block. The Fan Out mediation primitive is configured in Iterate mode and therefore a message is produced for each occurrence of the AccountInfo element within the inbound request message. A service is invoked for each occurrence to retrieve a summary of the account information.

Prior to the invocation of the service, a BOMapper mediation primitive replaces the multiple occurrences of the element within the message body with the single occurrence for the current iteration stored within the FanOutContext of the SMO. A Message Element Setter mediation primitive immediately following the Service Invoke mediation primitive then appends information from each response to an array of structures defined within the shared context of the SMO. The output terminal of the Fan In mediation primitive is fired after the Fan Out mediation primitive has iterated over all occurrences of the repeating structure and the Fan In mediation primitive has received all corresponding messages on the input terminal. A final BOMapper mediation primitive aggregates the information stored within the shared context and transient context to create a response to the initial request:

Figure 7. Iterate aggregation design pattern
Iterate aggregation design pattern

Performance considerations

This section describes performance considerations that you need to understand when creating aggregation solutions.

XSL transformation vs. BOMapper mediation primitives

Aggregation patterns often require multiple requests to be built from a single input request and multiple responses to be aggregated into a single response. In the examples above, the BOMapper mediation primitive is used to perform the transformations in the mediation flow. If the functionality supported by the BOMapper mediation primitive is sufficient for the data transformations, then runtime performance will be improved by using the BOMapper mediation primitive in place of the XSL Transformation (XSLT) mediation primitive, with the performance improvement proportional to the size of the message being processed. The XSLT mediation primitive provides additional functionality not supported by the BOMapper mediation primitive, so there may be circumstances when using BOMapper is not feasible.

The following chart illustrates the runtime performance for three variations of the Branching aggregation design pattern scenario illustrated in Figure 5:

  • Branch-XSLT -- XSLT mediation primitive used for all transformations
  • Branch-BOMapper -- BOMapper mediation primitive used for all transformations
  • Branch-Enrichment -- Using the new Message Enrichment Mode of the Service Invoke mediation primitive and BOMapper mediation primitive for final aggregation (shown above in Figure 6)
Figure 8. Branching aggregation design pattern -- runtime performance
Branching aggregation design pattern -- runtime performance

The results show that significant runtime performance improvements can be achieved by using the BOMapper mediation primitive instead of the XSLT mediation primitive in aggregation solutions, when feasible. The results also show the additional improvements in runtime performance that you can achieve with the new Message Enrichment Mode functionality in WebSphere ESB V7.5 and IBM Integration Developer V7.5 for the Service Invoke mediation primitive.

Iterate design pattern -- Batch processing considerations

When using the iterate design pattern, consider the impact that processing a large batched request may have on other applications hosted on the same server. Processing requests with a large number of repeating elements, such as sales orders, can use lots of system memory, which will remain in use until all processing is complete. Depending on the response times of the services invoked within the aggregation, block processing may tie up system resources for long periods. If the same server used for processing batch requests is also being used to host applications that need to respond quickly to requests, there may be service degradation while the batched request is being processed. To avoid this problem, process large batched requests on a separate server or during periods of low system usage.

For the reasons explained above, it is important to minimize the amount of memory used when processing large batched request messages. When using the batched processing pattern, the body of the original request message is rarely required inside the aggregation block, because the Fan Out mediation primitive stores the current occurrence of a repeating element in the Fan Out context of the SMO. At the end of the aggregation block, the response data is held in the shared context and is used to build a new response message body. If the request is large, then you can significantly reduce memory usage and processing time by deleting the message body immediately after the Fan Out mediation primitive by using the Message Element Setter mediation primitive, or by using a BOMapper mediation primitive and not propagating the body of the SMO.

Figure 9. BOMapper mediation primitive -- not propagating /body
BOMapper mediation primitive -- not propagating /body

Additional memory considerations

After an aggregation block is completed, the responses that have been stored in the shared context are combined into a new response message body, which is performed immediately after the Fan In mediation primitive. At this point, the contents of the shared context are not typically required, and performance is improved if the shared context is not propagated through the remainder of the flow. If you do not require any of the fields in the context, then leave the context unmapped. Otherwise, expand the context element and map only those parts that are required in the rest of the mediation flow.

Figure 10. BOMapper mediation primitive -- not propagating /context
BOMapper mediation primitive -- not propagating /context

Conclusions

Aggregation is an important and powerful ESB pattern, but you need to understand and evaluate several performance considerations before developing and deploying WebSphere ESB mediation flows that use it, including:

  • Runtime performance of the BOMapper and XSLT mediation primitives
  • The effect of large batch processing scenarios on requests to co-located services
  • Message data propagation through the aggregation block and mediation flow

Resources

  • WebSphere ESB resources
  • WebSphere resources
  • developerWorks resources
    • Trial downloads for IBM software products
      No-charge trial downloads for selected IBM® DB2®, Lotus®, Rational®, Tivoli®, and WebSphere® products.
    • developerWorks blogs
      Join a conversation with developerWorks users and authors, and IBM editors and developers.
    • developerWorks cloud computing resources
      Access the IBM or Amazon EC2 cloud, test an IBM cloud computing product in a sandbox, see demos of cloud computing products and services, read cloud articles, and access other cloud resources.
    • developerWorks tech briefings
      Free technical sessions by IBM experts to accelerate your learning curve and help you succeed in your most challenging software projects. Sessions range from one-hour virtual briefings to half-day and full-day live sessions in cities worldwide.
    • developerWorks podcasts
      Listen to interesting and offbeat interviews and discussions with software innovators.
    • developerWorks on Twitter
      Check out recent Twitter messages and URLs.
    • IBM Education Assistant
      A collection of multimedia educational modules that will help you better understand IBM software products and use them more effectively to meet your business requirements.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=775280
ArticleTitle=Aggregation design patterns and performance considerations in WebSphere Enterprise Service Bus V7.5
publish-date=11162011