Aggregation functionality in IBM WebSphere Enterprise Service Bus V6.1, Part 1: Introduction to aggregation

Document options PDF - Fits A4 and Letter 442KB Get Adobe® Reader® Document options requiring JavaScript are not displayed Discuss Rate this page Help us improve this content

Level: Intermediate

Callum Jackson (callumj@uk.ibm.com), Software Engineer, IBM
Brian Hulse (brian_hulse@uk.ibm.com), Software Engineer, IBM

13 Mar 2008

Get up to speed on the newly added IBM® WebSphere® Enterprise Service Bus V6.1 functionality, namely aggregation. This three-part article series takes you from an introduction to the basic mediation primitives—which you can use to build realistic scenarios—to a description of useful patterns of aggregation.

Introduction

WebSphere Enterprise Service Bus V6.1 mediation primitives are reusable building blocks available to application developers to build mediation flows. The final article in this series explains how these useful aggregation patterns are built from the new mediation primitives using real business situations. This highlights the differences between the patterns while taking you through the important steps needed to achieve a successful mediation flow using aggregation.

Don't consider aggregation as simply the splitting and joining of messages, but more as the complete business flow, including any service invocations that form part of that flow. In this respect, Part 2 of this series examines in more detail the usage of one of the new primitives, the service invoke primitive. This includes the different types of invocation available and how you can use them within the standard aggregation patterns. You should have a basic knowledge of mediation module development for WebSphere Enterprise Service Bus to follow along with this series.

This article discusses the new aggregation capabilities within WebSphere Enterprise Service Bus V6.1 and briefly introduces the service invoke capabilities. You also get details about the process of creating a mediation flow, including aggregation logic.

Back to top

The power of aggregation

WebSphere Enterprise Service Bus V6.1 introduces aggregation into mediation flow logic that previously required extensive custom code. Before diving into the technical detail, let's look at the power of aggregation and why it's an important enterprise service bus (ESB) pattern. In simple terms, an inbound request may map into several individual outbound requests. The responses from these requests may be aggregated into a single response for the original request.

Potential usages of aggregation are wide and varied; a common example is the car insurance quote system. The end user uses a Web interface to submit a request for pricing a car insurance policy, and a business process (such as BPEL) executes the business logic and then submits a single request for quotes. In this way, you can use aggregation logic to query a set of car insurance companies and aggregate the result into a single response, passed to the business process to continue processing. A mocked-up example of this scenario is detailed later in this article.

Enhancements to the mediation component that provide aggregation support

Integrating first class support for aggregation within WebSphere Enterprise Service Bus V6.1 has involved several extensions to the mediation component programming model. This section covers these enhancements.

New mediation primitives

As part of the aggregation support introduced in WebSphere Enterprise Service Bus V6.1, three new primitives and an enhancement to an existing primitive have been implemented:

• Fan Out
• Fan In
• Service Invoke
• Message Element Setter (enhanced)

New Fan Out primitive

You can use the Fan Out primitive to fire the output terminal once (with the input message) or to fire the output terminal n times, where n is the number of items within a defined repeating element. You can use Fan Out in isolation or as part of a Fan Out and Fan In combination.

As with other mediation primitives, IBM WebSphere Integration Developer provides the tooling capabilities to associate properties and terminals with the primitive, providing customization of the aggregation behaviour. The FanOut properties view allows for the configuration of the two firing modes, as shown in Figure 1.

If the repeating element option is selected, then an XPath expression must be provided. This XPath expression must point to an array of elements that can be iterated over during the processing of the Fan Out primitive. For each element within the array, a message is fired with the selected item of the array stored within the Fan Out Context, which is covered in more detail later in this article.

When you choose the once option, the aggregation is accomplished by wiring multiple flows from the output terminal where each of these flows is initiated by the one firing of the message. More information about how this works is provided later in the article.

The Fan Out primitive has one input terminal and three output terminals, as shown in Figure 2.

New Fan In primitive

Fan In is always partnered with a Fan Out in the same flow and acts as a decision point for when to continue flow execution. It receives a number of messages until a decision point is reached, at which point the last message to be received is propagated to the output terminal. The Fan In primitive may only be used in combination with Fan Out, and this rule is enforced by validation within WebSphere Integration Developer.

Three types of decision points for Fan In are supported:

• Simple count: The output terminal is fired when a set number of messages are received at the input terminal. This doesn't stop the Fan Out from sending any more messages, because it's possible that the Fan In count decision point will be reached more than once, resulting in multiple firings of the output terminal.
• XPath decision: The output terminal is fired if an XPath expression evaluation of the incoming message evaluates to true.
• Iterate: The Fan In waits to receive all messages produced by the corresponding Fan Out primitive when in iterate mode; all of the occurrences in the repeating element have been traversed.

Within WebSphere Integration Developer, a Fan In primitive is matched to a single Fan Out primitive. The Details section of the Properties tab shows the configuration of the associated Fan Out (in a noneditable format), along with the configuration options available to set up the decision point to be used. This is useful in ensuring that the configuration of the Fan In and corresponding Fan Out complement each other.

In addition to the decision point configuration, it's also possible to set a timeout value for Fan In. The timeout period starts when the associated Fan Out fires an output terminal for the first time. If a message arrives at the Fan In in terminal after this timeout period, it's considered late, and the incomplete terminal is fired. The default value is -1, which means there's no timeout, and no messages are considered late.

The Fan In has two input and three output terminals, as shown in Table 2.

New Service Invoke primitive

The Service Invoke primitive is used to make a service request in either a request or response mediation flow. The service may be Request/Response or One-Way. Multiple instances of the Service Invoke primitive are permitted in a flow, allowing a series of service invocations to be performed. This primitive is similar in concept to the invoke capability of the Custom mediation in V6.0.2, but extends the capabilities in three ways:

• The service invocation made by the new primitive may be synchronous or asynchronous with respect to the execution of the mediation flow.
• Retry support has been added, allowing a retry count to be specified on the primitive. The runtime support uses this retry to automatically resend the request to a service that responds with a failure. The retry settings can be configured within the Retry section of the Properties tab, which provides the following settings:
  • Retry on: Determines whether, and how, fault responses cause a retry. The following values are valid:
    • Never
    • Any fault
    • Unmodeled fault
    • Modeled fault
  • Retry count: Dictates how many times a service call should be retried before either the failure or timeout terminal is fired.
  • Retry delay: Sets the delay (in seconds) between retry attempts.
  • Try alternate endpoints: It's possible to store alternative endpoints within the Service Message Object (SMO) header, allowing the retry logic to cycle through these alternative locations. This function is activated by selecting the appropriate check box, as shown in Figure 5.
    
    Figure 5. Service Invoke retry panel
    
    
    

The Custom mediation invoke fixed the method signature to include a single DataObject argument (representing either the SMO body or the entire SMO). With the Service Invoke, the signature can be anything. This enables users to call any existing service, not just a service that was designed specifically to be called from a Custom mediation.

The Service Invoke primitive has a single input terminal and a variable number of output terminals. If the operation being invoked is one way, there's no fault or out terminals because these can only exist for two-way operations (see Figure 6).

Message Element Setter primitive enhancement

The Message Element Setter primitive has been enhanced to provide support to append an item to an array with the SMO. This feature is critical to providing aggregation support where a dynamic number of multiple requests can be submitted and the responses to these multiple requests need to be stored. Usage patterns of this enhanced primitive are discussed in the third article in this series.

The append option is similar to the copy in usage. The target array is selected using an XPath expression (note that the [ ] brackets aren't included on the XPath expression). The type is set to append, and the value is what will be appended to the array. The element type of the array and the type of the value must be of the same type. It's important to realize that the parent object of the array (in the above case, currentOutlet) needs to exist before a value can be appended.

Back to top

New contexts add to the Service Message Object

Typical aggregation scenarios need to make use of specific message context areas. WebSphere Enterprise Service Bus V6.1 has introduced two new ones to aid in construction of these scenarios.

Shared Context

WebSphere enterprise Service Bus V6.0.2 had two user-defined data store contexts within the SMO:

• Transient context, which is used within either a request or response flow
• Correlation context, which is used in both the request and response flows.

In complex branching logic, it's possible to have two separate copies of these contexts; for instance, consider the flow shown in Figure 8.

If MessageFilter1 fires both output terminals, then a clone of the output message occurs, and the message sent to CustomMediation1 is a copy of the one sent to CustomMediation2. If these two mediations append to the correlation or transient contexts their primitive name, each of the Message Logger primitives only see the name relevant to their branch of the flow. In a typical aggregation scenario, there's a requirement to have a data store that can exist across aggregation branching to allow the multiple responses to be aggregated into a single response. This data store is called the Shared Context in WebSphere Enterprise Service Bus V6.1. The existing contexts, transient and correlation, are still available and work in exactly the same way as they did in V6.0.2.

Like the transient and correlation contexts, the Shared Context is defined as a user-provided business object. Once defined, the Shared Context can be used to store data during aggregation operations. For this reason, you need to design the Shared Context business object carefully to ensure that it's suitable for all aggregation scenarios in a specific flow.

The Shared Context area of the SMO has been designed for storing aggregation data between a Fan Out primitive and a Fan In primitive. More specifically, the Shared Context is a thread-based storage area that's shared in the same thread. The content of the Shared Context business object doesn't persist across a request flow and a response flow, through callout invocation; whatever data is in the Shared Context of the request flow can't be reused during the response flow. Therefore, you can only aggregate in a particular flow: a Fan In mediation primitive in a response flow can't be used to aggregate messages from a Fan Out mediation primitive in a request flow.

After the Fan In primitive, it's expected that in most cases the flow performs a map/transform against the information in Shared Context to produce a new SMO body message type, or to augment the existing type with aggregated information. You can see an example of this type of manipulation later in this article.

Fan Out Context

The Fan Out Context is used to store the current item of a repeating element when Fan Out is used in iterate mode. The Fan Out Context contains an integer field containing the index of the occurrence of the repeating element along with an element containing the occurrence itself. For instance, if there was a repeating element of an element called order, the Fan Out Context would look like Listing 1.

                
<FanOutContext>
<iteration>0</iteration >
<occurrence>
<order>
…….
</order>
</occurrence>
</FanOutContext>

Back to top

How to create your first aggregation

This section describes the process of creating a mediation component that includes aggregation logic. It's based on a car insurance quote comparison Web site use case that was mentioned earlier, and assumes that you're familiar with the basic concepts of application construction within WebSphere Integration Developer and more specifically, mediation development.

Let's go through the steps involved to create the aggregation:

1. Start WebSphere Integration Developer with a new workspace, and create a new mediation module called InsuranceQuote.
2. Create the following four business objects, which are shown in Figure 9:
   • Quote: Represents the data in the request message for the mediation flow and also the insurance systems
   • QuoteResult: The representation of a response from a single insurance system that's grouped together into an overall QuoteResponse
   • SharedContent: Designed to contain the responses from the two invocations
   • QuoteResponse: Returned by the mediation flow and shown here as the final object
   
   
   Figure 9. Business objects
   
   
   
3. The interfaces into the mediation and the insurance quote services need to be defined. Therefore, next you want to create the two interfaces shown in Figure 10.
   
   Figure 10. Insurance interfaces
   
   
   
4. Open the mediation module's Assembly Diagram, and drag the Quotation interface onto the canvas. When the Component Creation window appears, select Component with no Implementation Type, and click OK.
5. Select the newly created component, right-click it, and select Rename. Rename the component to InsuranceCompanyA.
6. Select the InsuranceCompanyA component, right-click it, and from the menu, select Generate Implementation - Java.
7. When the Generate Implementation window appears, click OK.
8. A Java™ editor appears with the stub implementation automatically generated. The only modification required is the quote method: Change the value return null; to return 250.50;. The method should now read as shown in Listing 2.
   
   Listing 2. Stub implementation
   

   public Double quote(DataObject quoteDetails) { return 250.50; }

   
   
9. Repeat steps 4 through 8 for the InsuranceCompanyB service, but rename the component to InsuranceCompanyB instead of InsuranceCompanyA.
10. A Java editor appears with the stub implementation automatically generated. The only modification required is the quote method: Change the value return null; to return 271.60;. The method should now read:
    
    Listing 3. Stub implementation
    

    public Double quote(DataObject quoteDetails) { return 271.60; }

    
    
11. Drag the InsuranceQuote interface onto the canvas. In the Component Creation window, select Export with Web Service Binding. Then select OK, and select OK again (in the Transport Selection window).
12. Complete the following wiring:
    • InsuranceQuoteExport1 --> InsuranceQuote
    • InsuranceQuote --> InsuranceCompanyA
    • InsuranceQuote --> InsuranceCompanyB

    The assembly diagram should look like Figure 11.

    
    
    Figure 11. Assembly diagram
    
    
    
13. Right-click the mediation component (InsuranceQuote), and select Regenerate Implementation in the window that appears (if a window appears warning that an existing implementation exists, click OK).
14. On the far right of the mediation flow editor, the context types for the flow are displayed. Click the plus sign associated with the Shared Context.
15. The Data Type Selection window appears. Select SharedContext from the list, and click OK.
16. The Palette is located left of the mediation flow. Expand the Routing section, and select FanOut. Move the mouse to the canvas and click. This adds the Fan In primitive to the canvas.
17. Repeat the previous step for the following primitives:
    1. Fan In primitive: When adding the primitive, the Fan Out Primitive Selection window appears. Select FanOut1, and click OK.
    2. Service Invoke primitive: When adding the primitive, the Select Reference Operation window appears. Select QuotationPartner from the Reference section, and click OK.
    3. Service Invoke primitive: When adding the primitive, the Select Reference Operation window appears. Select QuotationPartner1 from the Reference section, and click OK.
18. On the Palette, expand the Transformation section, and create three XSL Transformation and two Message Element Setter primitives on the canvas.
19. Position the primitives into the following structure:
    
    Figure 12. Mediation primitives
    
    
    
20. Complete the following wiring:
    • Input Node out terminal -> FanOut1 in terminal
    • FanOut1 out terminal -> XSLTransformation1 in terminal
    • FanOut1 out terminal -> XSLTransformation2 in terminal
    • XSLTransformation1 out terminal -> ServiceInvoke1 in terminal
    • XSLTransformation2 out terminal -> ServiceInvoke2 in terminal
    • ServiceInvoke1 out terminal -> MessageElementSetter1 in terminal
    • ServiceInvoke2 out terminal -> MessageElementSetter2 in terminal
    • MessageElementSetter1 out terminal -> FanIn1 in terminal
    • MessageElementSetter2 out terminal -> FanIn1 in terminal
    • FanIn1 out terminal -> XSLTransformation3 in terminal
    • XSLTransformation3 out terminal -> InputResponse in terminal

    The mediation flow should be similar to the diagram in Figure 13.

    
    
    Figure 13. Mediation flow
    
    
    
21. All the required primitives have been added to the canvas. Some of the default configuration needs to be modified. Start by selecting the XSLTransformation1 primitive and in the Properties tab, select the Details section.
22. An error message displays regarding a missing mapping file. Select the New button, and in the New XML Mapping window, click Finish.
23. The new XSLT editor displays. Create the required mapping automatically using the match mapping icon. An inline map is created. Save the mapping file.
24. Repeat the previous steps for the XSLTranformation2 primitive.
25. Select the MessageElementSetter1 primitive. Within the Properties pane, select Details. Click Add, and in the new window fill in the values as shown in Figure 14.
    
    Figure 14. Store company name
    
    
    
26. Click Finish. Similarly, create a second property with the values shown in Figure 15.
    
    Figure 15. Store insurance quote
    
    
    

    The Details section should contain two entries.

27. Repeat this process for MessageElementSetter2, using CompanyB instead of CompanyA (see Figure 16).
    
    Figure 16. Store CompanyB quote data
    
    
    
28. For XSLTransformation3, click New within the Details sections for the Properties tab. Then select Next in the New XML Mapping window.
29. Change the Message Root to / instead of the default /body, then click Finish.
30. Within the XSLT editor, click the Map source and target based on name and types button. This automatically generates default wiring for the transformation.
31. Navigate back up to the top level, and create the following connections:
    • /smo/context/shared/companyA --> /body/quoteResponse/quoteResults/quoteResponses
    • /smo/context/shared/companyB --> /body/quoteResponse/quoteResults/quoteResponses

    Each of the Inline map operators will have a small explanation mark (!), because the target is an array and requires an index. Therefore, select each Inline operator (one at a time, companyA then companyB). In the Properties pane, select the Cardinality section, and fill in 1 and 2, respectively, for the index (see Figure 17).

    
    
    Figure 17. Transformation to response message
    
    
    
32. Select the yellow down arrow associated with the companyA Inline map, and within the map create the following mappings (see Figure 18):
    • companyA/Company --> quoteResponses/Company
    • companyA/Cost --> quoteResponses/Cost
    
    
    Figure 18. Quote mapping inside response transformation
    
    
    
33. Repeat the same process for the companyB Inline map, then save the mapping file.
34. Return to the mediation flow editor, and select the FanIn1 primitive in the Properties tab. Change the input messages have been received field from 1 to 2, as shown in Figure 19.
    
    Figure 19. Fan In property panel
    
    
    
35. Save the mediation flow (Control + S).
36. Start the built-in WebSphere Enterprise Service Bus or IBM WebSphere Process Server test environment, and deploy the InsuranceQuoteApp application.
37. Within the assembly diagram, select the InsuranceQuote component and right-click it. From the menu, select Test Component.
38. Fill in some sensible values for the parameters that don't include default values, as shown in Table 4.

39. Click Continue.
40. A response is received with the two insurance quotes included.

Conclusion

This article covered mediation primitives, introduced as part of WebSphere Enterprise Service Bus V6.1, that you can use to create aggregation scenarios in mediation modules. You explored the input, output, and fail terminals of each of these primitives, along with the primitives' modes of operation. You should now have an understanding of how these primitives behave and how you can use them in a simple aggregation scenario.

Resources

• Learn more about IBM WebSphere Enterprise Service Bus.
  
  
• Find out more about IBM WebSphere Integration Developer.
  
  
• The SOA and Web services zone on IBM developerWorks hosts hundreds of informative articles and introductory, intermediate, and advanced tutorials on how to develop Web services applications.
  
  
• Play in the IBM SOA Sandbox! Increase your SOA skills through practical, hands-on experience with the IBM SOA entry points.
  
  
• The IBM SOA Web site offers an overview of SOA and how IBM can help you get there.
  
  
• Stay current with developerWorks technical events and webcasts.
  
  
• Browse for books on these and other technical topics at the Safari bookstore.
  
  
• Check out a quick Web services on demand demo.
  
  
• Get an RSS feed for this series. (Find out more about RSS.)

• Innovate your next development project with IBM trial software, available for download or on DVD.
  
  

• Participate in the discussion forum.
  
  
• Get involved in the developerWorks community by participating in developerWorks blogs.
  
  

About the authors

		Callum Jackson is a software engineer for WebSphere Enterprise Service Bus based in Hursley, United Kingdom, and has worked in the product area since 2005. Prior to this he worked in ISSW on SOA applications for the telecommunication industry.

		Brian Hulse is a senior software engineer for WebSphere Enterprise Service Bus based in Hursley, United Kingdom. He has worked in software development at Hursley for 20 years, the last five of which have been in the SOA arena.

Rate this page

Please take a moment to complete this form to help us better serve you. Did the information help you to achieve your goal? Yes No Don't know   Please provide us with comments to help improve this page:   How useful is the information? 1 2 3 4 5 Not useful Extremely useful

Share this.... Digg this story del.icio.us Slashdot it!

Back to top