Designing ESB mediations for deployment to WebSphere Enterprise Service Bus

This article provides a "holistic" view of concepts and best practices for designing mediations for deployment to WebSphere ESB. The article describes pre-requisite artifacts (system context, requirements, and use cases) that form the basis for design. You can also use this approach for designing ESB solutions for other IBM ESB products. The article is for SOA solution architects and design teams tasked with designing solutions that incorporate core ESB patterns to address complex enterprise system interactions.

Share:

Tendai Chinoda (tendai@us.ibm.com), Advisory Software Engineer, IBM

PhotoTendai Chinoda is an IBM Certified IT Specialist and Advisory Software Engineer in IBM Software Services for WebSphere (ISSW). He provides mentoring, consulting, and enablement services to IBM customers for the WebSphere family of products. His expertise includes SOA and enterprise application architecture and solution design along with Java EE and Web services. You can contact Tendai at tendai@us.ibm.com.



30 December 2008

Also available in Chinese

Introduction

This article shows you how to design mediations for deployment to IBM® WebSphere® Enterprise Service Bus. The article outlines the design effort in the context of a business scenario and the generation of prerequisite design artifacts -- system context, requirements and use case documents.

Solution architecture and design methodologies are well known to the IT architect and specialist communities. When architecting and designing SOA-based solutions, we expand on known solution architecture principals and design solutions that extend across layers of the SOA solution stack:

Figure 1. SOA Reference Architecture -- Solution Stack View
SOA Reference Architecture -- Solution Stack View

Within an SOA the ESB architectural pattern is applied to facilitate communication between service providers and requestors. See the article Design an SOA solution using a Reference Architecture and Figure 2 below:

Responsibilities of the ESB include:

  • Interface Mapping -- Mapping of interfaces between source and target systems
  • Routing -- Routing of requests to one or more endpoints (provider needs not be aware of routing semantics to consumer endpoints)
  • Data Transformation -- Transformation of data to/from application specific formats
  • Protocol Transformation -- Transformation of communication channel protocols between source/target systems.
  • Quality of Service -- Enable Quality of Service, including assured delivery, logging, security, error detection, and auditing.
Figure 2. SOA Reference Architecture -- Middleware Infrastructure View
SOA Reference Architecture -- Middleware Infrastructure View

Design an SOA solution using a Reference Architecture addresses design concepts for ESB solutions targeted for deployment to the WebSphere ESB but which may also be applied when designing ESB solutions targeted for any of IBM's ESB product offerings, as shown below:

Figure 3. WebSphere ESB products
WebSphere ESB products

Business scenario

The business scenario referred to in this article uses a fabricated corporation and problem space to portray real world system integration challenges and how they can be solved via SOA solutions.

Background

TPCBanking Corporation's Strategic Business and IT teams have decided to expand their existing Customer Support business portfolio. The expansion will extend the services provided by Customer Support Representatives to include Loan Processing and initiation of Automatic Bill Payments for customers. Such an expansion would require the integration of three independent TPCBanking IT systems;

  • Branch System
  • Loan Processing System
  • Automatic Bill Payment System

A key decision made by the Strategic Business and IT team was to participate in a joint SOA initiative with IBM. In accordance with this initiative and by adopting the IBM Smart SOA Approach, the team defined high level requirements that map to the IBM SOA Reuse and IBM SOA Connectivity entry points.

SOA reuse entry point requirements

TPCBanking will service enable existing IT Assets to support expansion of their Customer Support business portfolio.

  • The Loan Processing and Auto Bill Pay systems will each have services exposed to allow the synchronization of Customer Account information from the Branch System
  • The Branch System will service enable existing application code to allow the Auto Bill Pay system to update account information when bill payments occur.

SOA connectivity entry point requirements

TPCBanking will integrate systems via an ESB to facilitate seamless flow of messages and information.

  • The Branch System will submit account updates to an ESB which will perform the necessary mediations and route the account updates (for synchronization) to the Loan Processing and Auto Bill Pay Systems.
  • The Auto Bill Pay System will submit Account balance updates to an ESB which will perform the necessary mediations and forward the account balance updates to the Branch System.

The Strategic Business and IT teams have requested that the SOA Solution Architecture and Design teams design a solution that leverages capabilities of WebSphere ESB for system integration. Along with the high-level requirements outlined above, the Strategic Business and IT teams also provided the SOA Solution Architecture team with their proposed view of the flow of information between systems:

Figure 4. TPCBanking Proposed Solution: System Context Diagram
TPCBanking Proposed Solution: System Context Diagram

Where to start?

As a best practice, prior to designing ESB solutions, it is important that the requirements and functional behavior of the solution are well understood, identified and outlined. Thus, common architectural and design artifacts such as System Context, Requirements and Use Case documents should be considered design pre-requisites even when the solution is ESB specific. Ad hoc approaches to SOA-Based Solution Design which exclude tasks that outline the scope, system context, requirements and functional behavior (specific to the ESB implementation) increase the risk of designing a solution with inadequacies and missed requirements.

Business scenario: Work breakdown structure

The work breakdown structure (WBS) compiled by the Solution Architecture and Design team is shown below and includes tasks to generate the prerequisite design artifacts followed by design of the mediations.

TPCBanking Project Management Team roles and abbreviations

  • Auto Bill Pay System Technical Owner -- ABPSTO
  • Branch System Technical Owner -- BSTO
  • Business Analysts -- BA
  • Customer Support Manager -- CSM
  • Design Team -- DT
  • IT Manager -- ITM
  • Loan Processing System Technical Owner -- LPSTO
  • Project Manager -- PM
  • Solution Architect -- SA
  • Strategic Business Manager -- SBM
TPCBanking ESB solution design WBS
Task Sub-Task Owner Participants
1: Conduct Kick-off Discussions and Interviews 1.1: Overview of Business Scenario and Challenges
SBM, ITM, CSM SBM, ITM, CSM, BA, PM, SA
1.2: Review High Level Requirements and Proposed View of SystemSBM and ITMSBM, ITM, CSM, BA, PM, SA
1.3: Define and Document Goals and Scope of Design effort PM SBM, ITM, CSM, BA, PM, SA
2: Conduct System Level Definition Exercise (per System) 2.1: System Level Technical Deep Dive System Technical Owners BSTO, LPSTO, ABPSTO, SA, PM
2.2: Document System Descriptions, Exposed Interfaces, Standards, Protocols, Security, Specifications SA BSTO, LPSTO, ABPSTO, SA, PM
3: Conduct Information Flow Analysis 3.1: Technical Deep dive on payload for each information flow between systems BA SA, BA, BSTO, LPSTO, ABPSTO
3.2: Document source, target, interaction pattern, frequency, volume of data etc. SA SA, BA, BSTO, LPSTO, ABPSTO
4: Conduct End-to-End (E2E) Scenario Analysis 4.1: Outline E2E scenarios that trigger data flows between systems. BA SA, BA
4.2: Identify mediation flow logic that would benefit from ESB services (i.e. protocol transformation, interface mapping, data aggregation etc.) SA SA, BA
4.3: Document E2E Information flow descriptions SA SA,BA
4.4: Generate and Document revised System Context Diagram SA SA, BA
Requirements and Use Case definition per E2E Scenario
5: Define/Refine Requirements 5.1: Identify/Review ESB Functional requirements to support flow of information from source to target systems. SA, BA SA, BA
5.2: Identify/Review ESB Non-Functional requirements to support flow of information from source to target systems (protocols, standards, security, logging, auditing etc.) SA, BA SA, BA
5.3: Document ESB Solution Requirements BA SA, BA
6: Define Use Cases (Functional Behavior) for Information Flows 6.1: Define Use Cases within the context of the E2E scenarios SA, BA SA, BA
6.2: Capture additional requirements identified during Use Case definition SA, BA SA, BA
6.3: Validate traceability of all Requirements to Use cases BA SA, BA
6.4: Document Use Cases BA SA, BA
7: Approve System Context, Requirements and Use Cases 7.1 Team Review/Sign-off of System Context, Requirements and Use Cases BA SBM, ITM, CSM, BA, PM, SA
7.2: Update documentation with feedback SA, BA SBM, ITM, CSM, BA, PM, SA
Mediation Design Activities -- Iterative
8: Define Inbound and Outbound Service Contracts 8.1: Identify Inbound and Outbound Service Contracts required to support each information flow. SA SA, DT, BA
8.2: Per Outbound Interface, gather existing Web Service Definition Language (WSDL) files for services exposed by the Branch, Auto Bill Pay and Loan Processing Systems. SA BSTO, LPSTO, ABPSTO, SA, DT
8.3: Per Inbound Interface, define WSDL for services exposed by the ESB. SA BSTO, LPSTO, ABPSTO, SA, DT
9: Design Mediations 9.1 Use Case Elaboration to Identify Mediations SA SA, DT
9.2: Per Inbound Service Contract (ESB Export) design:
Mediation Module
Mediation SCA Assembly Diagram
Mediation Flow Component
Request Flow
Response Flow
SA SA, DT
9.2: Design Review SA SA, DT
9.3: Document Mediation Design SA SA, DT

Kickoff discussions and interviews

The purpose of the kickoff discussions is for the Solution Architect to obtain a comprehensive view and understanding of the business scenario and current challenges requiring a solution. The discussions should conclude with a clear definition and agreement on the goals and scope of the design effort.

Business scenario: Design goals and scope

The goals and scope documented by the Solution Architecture and Design team at the conclusion of the kick-off discussions are shown below:

TPCBanking ESB: Solution architecture and design goals

  • Define and document the physical and logical ESB Architecture to support connectivity and information flow between TPCBanking Branch, Loan Processing and Auto Bill Pay Systems.
  • Design and document the WebSphere ESB Mediations to support implementation of End-to-End (E2E) information flows between TPCBanking Branch, Loan Processing and Auto Bill Pay Systems.

TPCBanking ESB: Solution architecture and design scope

The scope for the ESB solution architecture and design effort is to design the WebSphere ESB Mediations and Supporting Services that facilitate:

  • Information flows for synchronization of Customer Account Updates from the Branch System to the Loan Processing and Auto Bill Pay Systems.
  • Information flows for adjustment of Customer Account Balances from Auto Bill Pay System to the Branch System.

System context

Once the goals and scope of the design effort are defined Tasks 2 thru 4 enable the Solution Architect to define the characteristics of the external systems and their interactions with the ESB. These characteristics should include:

  • Descriptions of the external systems that interact with the ESB
  • Technical aspects of the external systems such as security and communication protocols for integration with the system
  • Identification of source and/or target systems along with the information flow (payload) and interaction patterns.
  • Origin of the data being transferred as well as the triggers for each event that results in information flow between systems

A System Context Diagram provides a graphical view of the system under design as a black-box and shows the information/data flows through the system from external source to target systems. Each of these flows will require mediation logic within the ESB. The interfaces exposed to source systems, the implementation of the mediation logic and the invocation of target systems by the ESB represent the system boundaries of the design effort.

Business scenario: System context

Upon conclusion of tasks 2 through 4, the Solution Architecture and Design team documented the characteristics of each system and generated a revised view (see Figure 5) of the proposed solution (from Figure 4). The revised view shows the ESB as a black box facilitating connectivity and interactions between systems.

TPCBanking ESB: External system characteristics and revised proposed solution

Branch System:

The Branch System hosts the Branch Management Application (BMA)

  • BMA is a Java EE application that supports SOAP 1.1 Web Service client and server invocations.
  • Web Services hosted by BMA are exposed and accessible via SOAP/HTTP.
  • BMA is WS-I Basic Profile 1.1 compliant.

Loan Processing System:

The Loan Processing System hosts the Loan Processing Application (LPA):

  • LPA is a Portal application that supports SOAP 1.1 Web Service client invocations
  • Web Services hosted by LPA are exposed and accessible via SOAP/HTTP.
  • LPA is WS-I Basic Profile 1.1 compliant
  • Authentication of incoming Web Service requests to LPA requires client support of WS-Security 1.0.

Auto Bill Pay System:

The Auto Bill Pay System hosts the Auto Bill Pay Application (ABPA):

  • ABPA is a CICS application with extensions that support SOAP 1.1 Web Service client invocations
  • Web Services hosted by ABPA are exposed and accessible via SOAP/JMS.
  • ABPA is WS-I Basic Profile 1.1 compliant.
Figure 5. TPCBanking Revised ESB Solution System Context Diagram
TPCBanking Revised ESB Solution System Context Diagram

Requirements definition

The information flows between systems outlined in the System Context can be further analyzed to identify the ESB specific requirements to support the flows. The complete list of requirements for an ESB solution should include both generic and scenario specific functional/non-functional requirements. Generic requirements are those that could be common across different ESB implementations (i.e. non-functional) whereby scenario specific requirements are unique to the business scenario.

Strategy for requirements identification

Generic requirements (standards, specifications, interaction patterns etc.) can be identified using information obtained and documented in the System Context; in particular, technical information documented in Tasks 2 and 3. An analysis, by external system, of their supported standards, specifications, security and interaction patterns, facilitates identification of the requirements that the ESB must support in order to interact with each external system.

Scenario specific requirements (mediation flow logic to/from source/target systems) can also be identified from information documented in the System Context; in particular, mediation flow logic documented in Task 4. For each flow documented in the System Context, we would identify the functional and non-functional requirements that the ESB solution must support in order to mediate the flow from source to target system.

Business scenario: Requirements

A subset of the generic and scenario specific requirements identified during Task 5 by the Solution Architecture and Design team are listed below:

TPCBanking ESB: Non-functional requirements

ESB client support:

  • The ESB should support service endpoints accessible from SOAP Web Service Clients (from BMA, ABPA)

ESB Protocol Support:

  • The ESB should support service endpoints accessible via SOAP/HTTP (from BMA)
  • The ESB should support service endpoints accessible via SOAP/JMS (from ABPA)

ESB Standards and Specifications Support:

  • The ESB should support the following Web Service Standards and Specifications
    • SOAP 1.1
    • WS-Security1.0
    • WS-I BasicProfile 1.1

ESB Interaction Pattern Support:

  • The ESB should support outbound asynchronous invocation of Web Services hosted by external systems.
  • The ESB should support outbound synchronous invocation of Web Services hosted by external systems.
  • The ESB should support inbound asynchronous invocation of Web Services hosted by the ESB from external systems
  • The ESB should support inbound synchronous invocation of Web Services hosted by the ESB from external systems

TPCBanking ESB: Functional Requirements -- Synchronize Account Info

Branch System -- Synchronize Account Info (Inbound):

  • The ESB should provide and expose a synchronous Synchronize Account Information Service for invocation by the Branch System
  • On Invocation of the ESB's Synchronize Account Information Service by the Branch System, the ESB should perform the necessary mediation logic and forward the request to the Loan Processing System
  • On Invocation of the ESB's Synchronize Account Information Service by the Branch System, the ESB should perform the necessary mediation logic and forward the request to the Auto Bill Pay System
  • On invocation of the ESB's Synchronize Account Information Service by the Branch System, the ESB should support transactional demarcation between service invocation and forwarding of requests to Loan Processing and Auto Bill Pay Systems.
  • On invocation of the ESB's Synchronize Account Information Service by the Branch System, the ESB should support guaranteed and ordered delivery to the Loan Processing and Auto Bill Pay Systems

Loan Processing System -- Synchronize Account Info (Outbound):

  • The ESB should support synchronous invocation of the Synchronize Account Info Service exposed by the Loan Processing System.

Auto Bill Pay System -- Synchronize Account Info (Outbound):

  • The ESB should support synchronous invocation of the Synchronize Account Info Service exposed by the Auto Bill Pay System.

TPCBanking ESB: Functional requirements -- Adjust account balance

Auto Bill Pay System -- Adjust Account Balance (Inbound):

  • The ESB should provide and expose a synchronous Adjust Account Balance Service for invocation by the Auto Bill Pay System
  • On Invocation of ESB's Adjust Account Balance Service by the Auto Bill Pay System the ESB should perform the necessary interface mapping and data transformation and forward the request to the Branch System
  • On Invocation of ESB's Adjust Account Balance Service by the Auto Bill Pay System the ESB should support transactional demarcation between service invocation and forwarding of requests to the Branch System.

Branch System -- Adjust Account Balance (Outbound):

  • The ESB should support synchronous invocation of the Adjust Account Balance Service exposed by the Branch System.

Use case definition

Use cases help to describe E2E business scenarios in a pre-defined manner that captures the functional behavior of the application or system being designed. Requirements are best understood in the context of Use Cases and when applied to ESB solutions, Use Cases provide a blueprint for mediation design.

The Use Cases defined for an ESB solution should describe the functional behavior from a system interactions view point with an emphasis on defining the scope of functional requirements implemented by the ESB. Use Case definitions can be derived from the E2E Scenarios outlined in Task 4 and once defined all requirements should be traceable to one or more Use Cases. It is also, a common occurrence that new requirements are discovered during the Use Case definition exercise.

Business scenario: Use Cases

A segment of the Use Case defined for the Synchronize Account Information flow, during Task 6, by the Solution Architecture and Design team is shown below:

TPCBanking ESB: Use Case 001 -- Synchronize Account Information

Trigger/Precondition:

  • Customer Account Update transaction occurs and commits on Branch System

Successful outcome:

  • Branch system invokes ESB and Customer Account Updates are synchronized to downstream systems

Failure outcomes:

  • Branch System is unable to invoke the ESB and Customer Account Updates are not synchronized to downstream systems
  • ESB is unable to invoke downstream systems and Customer Account Updates are not synchronized to downstream systems

Primary actors:

  • ESB

Secondary actors:

  • Branch Management Application (BMA)
  • Loan Processing Application (LPA)
  • Auto Bill Pay Application (ABPA)

Sequence of events:

  1. The BMA processes a Customer Account Update transaction.
  2. The BMA submits a Synchronize Customer Account Information (SynchCustAcctInfo) request to the ESB SynchCustAcctInfo Service and waits for response.
  3. The ESB receives the SynchCustAcctInfo request and queues the request for sequential (ordered by delivery) processing.
  4. The ESB returns a response to BMA that the request was been received and is being processed.
  5. The BMA receives and logs the response from the ESB that the request was been received and is being processed.
  6. The ESB determines the account update transaction type from the SynchCustAcctInfo request.
  7. For Customer Address/Contact and Account Balance updates, the ESB should perform the necessary interface mapping and transformations and then forward the SynchCustAcctInfo request to the LPA SynchCustAcctInfo Service and await a response.
  8. For Account Balance updates, the ESB should perform the necessary interface mapping and transformations and then forward the SynchCustAcctInfo request to the ABPA SynchCustAcctInfo Service.
  9. The ESB receives the response from the LPA that the request was been received and is being processed
  10. The ESB logs the completion of all transactions

Transactional demarcation:

  • Invocation of ESB service by BMA and forwarding to LPA and ABPA services should occur in separate transactions

Variations to sequence of events:

  • 2.a The BMA is unable to invoke (SynchCustAcctInfo) Service on the ESB and updates are not synchronized to downstream systems
    2.b The BMA processes the communication error according to error handling policies.
  • 7.a The ESB is unable to invoke the SynchCustAcctInfo Service on the LPA and updates are not synchronized on LPA.
    7.b The ESB processes the communication error according to error handling policies
  • 8.a The ESB is unable to invoke the SynchCustAcctInfo Service on the ABPA and updates are not synchronized on APBA.
    8.b The ESB processes the communication error according to error handling policies

Additional information:

  • Processing and submission of requests to downstream systems should be in parallel.

Based on event numbers 6, 7 and 8 in Use Case 001 the Solution Architecture and Design team identified the following missed requirements that needed to be included as functional requirements for the Synchronize Account Info flow:

TPCBanking ESB: Functional Requirements -- Synchronize Account Info

Branch System -- Synchronize Account Info (Inbound):

  • On invocation of the ESB Synchronize Account Information Service by the Branch System the ESB should determine the transaction type and only route Customer Address/Contact and Account Balance updates to the Loan Processing System.
  • On invocation of the ESB Synchronize Account Information Service by the Branch System the ESB should determine the transaction type and only route Account Balance updates to the Auto Bill Pay System.

Requirements verification

A logical checkpoint for verification and sign-off of the requirements for the ESB solution to be designed is upon completion of defining the Use Cases. Task 7 in the WBS provides an opportunity for the business team to review the System Context, Requirements and Use Cases and ensure compliance with the scope and goals of the business solution.

Mediation design

Mediation design can be approached in various ways and Tasks 8 and 9 in the WBS include subtasks that prescribe an optimal approach for mediation design.

  • 1. Identify and Gather service contracts and definitions
  • 2. Use Case Elaboration -- Identify mediations
  • 3. White Board Sessions -- Outline mediation logic
  • 4. Mediation Design -- Generate design artifacts for documentation

Identify and gather service contracts and definitions

For each flow, identify the inbound and outbound service contracts between the ESB and external systems. For Web Services the contracts are described as Web Services Description Language (WSDL) files.

Inbound service contracts

For services that the ESB will expose (inbound service calls) the service contract should be negotiated with the source system and provide a service endpoint for routing requests from source to target system. The detailed schema, messages, operations and other elements that constitute the WSDL are plugged in and finalized as the design develops.

Outbound service contracts

For services that the ESB will invoke (outbound service calls) the WSDL files provided by the Service Providers serve as the outbound service contracts. The ESB serves as the client and mediation point and therefore must adhere to the service contracts supplied by the Service Providers.

Business scenario: Inbound service contracts

By reviewing the System Context and holding discussions with the Branch Management Application owners the Solution and Architecture team were able to define the following preliminary inbound service contract for the Synchronize Customer Account Info flow into the ESB.

TPCBanking ESB: Inbound Service Contract -- Synchronize Account Information

Service Name:

  • CustomerAccountManagementService

Service Operations:

  • synchAccountInfo (synch)

In Parameter:

  • ESBSynchCustAccountInfoBO
Name Type Required
accountNumber INT Y
accountType STRING Y
routingNumber INT Y
accountOpenDate DATETIME Y
branchNumber INT Y
autoBillPayAuthCode INT Y
customerSSN STRING Y
customerFirstName STRING Y
customerLastName STRING Y
homeAddress ESBADDRESSBO Y
homePhone STRING Y
businessAddress ESBADDRESSBO Y
businessPhone STRING Y
otherPhone STRING N
emailAddress STRING N
txCode INT Y
txDate DATETIME Y
previousBalance INT Y
currentBalance INT Y

Embedded:

  • ESBAddressBO
Name Type Required
addressLine1 STRING Y
addressLine2 STRING Y
city STRING Y
state STRING Y
zip STRING Y

Out Parameter:

  • ESBResponseBO
Name Type Required
retCode INT Y
details STRING N

Application Exception:

  • ESBExceptionBO
Name Type Required
errorCode INT Y
details STRING N

Business Scenario: Outbound Service Contracts

The outbound service contract documented by the Solution Architecture and Design team from the WSDL file supplied by the Loan Processing Application (LPA) and Auto Bill Pay Application (ABPA) owners consists of the following Service Names, Operations and Payload.

TPCBanking ESB: Outbound Service Contracts -- Synchronize Account Information

Service Name:

  • LPACustomerAccountService

Service Operations:

  • LPACustomerAccountService.postCustAccountUpdateTx (synch)

In Parameter:

  • LPACustAccountTxBO
Name Type Required
accountNo INT Y
accountType STRING Y
accountOwnerFN STRING Y
accountOwnerLN STRING Y
accountOwnerSSNo STRING Y
openDate DATETIME Y
transactionType INT Y
transactionDate DATETIME Y
previousBalance INT Y
newBalance INT Y
description STRING N

Out Parameter:

  • LPAResponseBO
Name Type Required
retCode INT Y
details STRING N

Application Exception:

  • LPAProcessExceptionBO
Name Type Required
errorCode INT Y
details STRING N

Service Name:

  • ABPACustomerAccountService

Service Operations:

  • ABPACustomerAccountService.postCustAccountUpdateTx (asynch)

In Parameter:

  • ABPACustAccountTxBO
Name Type Required
accountNumber INT Y
routingNumber INT Y
accountType STRING Y
transactionDate DATETIME Y
previousBalance INT Y
newBalance INT Y
autoBillPayAuthCode INT Y
accountOwnerFName STRING Y
accountOwnerLName STRING Y
accountOwnerAddr1 STRING Y
accountOwnerAddr2 STRING Y
accountOwnerCity STRING Y
accountOwnerState ENUM Y
accountOwnerZip STRING Y

Use Case Elaboration -- Identify Mediations

Once the Service Contracts are well understood the next step is to identify the mediations required to facilitate source to target system data transfer. The Use Cases already outline the functional behavior of the system and at this point in the process; we can elaborate on the sequence of events in the Use Case to identify the mediations to support the functionality.

Business Scenario: Use Case Elaboration -- Identify Mediations

The Solution Architecture team's elaboration of Use Case 001 to identify the mediations to support the Synchronize Account Information flow is shown below:

TPCBanking ESB: Use Case 001 -- Synchronize Account Information (With Mediations)

Trigger/Precondition:

  • Customer Account Update transaction occurs and commits on Branch System

Successful Outcome:

  • Branch system invokes ESB and Customer Account Updates are synchronized to downstream systems

Failure Outcomes:

  • Branch System is unable to invoke the ESB and Customer Account Updates are not synchronized to downstream systems
  • ESB is unable to invoke downstream systems and Customer Account Updates are not synchronized to downstream systems

Primary Actors:

  • ESB

Secondary Actors:

  • Branch Management Application (BMA)
  • Loan Processing Application (LPA)
  • Auto Bill Pay Application (ABPA)

ESB Mediations:

  • SynchCustAcctInfoMediation
  • PostSynchCustAcctInfoMediation

Service Endpoints:

  • ESB -- CustomerAccountManagementService
  • LPA -- CustomerAccountService
  • ABPA -- CustomerAccountService

Sequence of events: (with mediation details):

  • 1. BMA processes a Customer Account Update transaction.
  • 2. BMA submits the Synchronize Customer Account Information (SynchCustAcctInfo) request to the ESB's CustomerAccountManagementService and waits for response.
  • 3. The ESB's SynchCustAcctInfoMediation receives the SynchCustAcctInfo request and forwards the SynchCustAcctInfo to the PostSynchCustAcctInfoMediation for further processing.
  • 4. The ESB's SynchCustAcctInfoMediation returns a response to the BMA that the request has been received and is being processed.
  • 5. The BMA receives the response from the ESB that the request was received and is being processed.
  • 6. The ESB PostSynchCustAcctInfoMediation determines the account update transaction type for the SynchCustAcctInfo request.
  • 7. For Customer Address/Contact and Account Balance updates; the ESB's PostSynchCustAcctInfoMediation performs the necessary interface mapping and transformations, forwards the SynchCustAcctInfo request to the LPACustomerAccountService and awaits a response.
  • 8. For Account Balance updates; the ESB's PostSynchCustAcctInfoMediation performs the necessary interface mapping and transformations and forwards the SynchCustAcctInfo request to the ABPACustomerAccountService.
  • 9. The ESB's PostSynchCustAcctInfoMediation receives the response from LPA that the request was been received and is being processed.
  • 10. The ESB logs the completion of all transactions

Transactional Demarcation: (with mediation details):

  • Invocation of the ESB's CustomerAccountManagementService, routing to the SynchCustAcctInfoMediation and delivery to the PostCustAcctInfoMediation should occur in a separate transactional context from the processing and forwarding of the request to LPA and ABPA services (see Figure 6)
Figure 6. Synchronize Customer Account Info -- Transactional Demarcation
Synchronize Customer Account Info -- Transactional Demarcation

Variations to Sequence of events: (with Error Handling Service)

  • 2.a The BMA is unable to invoke the ESB's CustomerAccountManagementService and updates are not synchronized to downstream systems
  • 2.b The BMA processes the communication error according to error handling policies.
  • 7.a The ESB's PostSynchCustAcctInfoMediation is unable to invoke LPA's CustomerAccountService and updates are not synchronized on LPA.
  • 7.b The ESB's PostSynchCustAcctInfoMediation forwards the request to the ESB Error Handling Service (see figure 7).
  • 7.c. The ESB Error Handling Service processes the communication error according to error handling policies.
  • 8.a The ESB's PostSynchCustAcctInfoMediation is unable to invoke ABPA's CustomerAccountService and updates are not synchronized on APBA.
  • 8.b The ESB's PostSynchCustAcctInfoMediation forwards the request to the ESB Error Handling Service (see figure 7).
  • 8.c. The ESB Error Handling Service processes the communication error according to error handling policies
Figure 7. Synchronize Customer Account Info -- Error Handling
Synchronize Customer Account Info -- Error Handling

White Board -- Outline Mediation Logic

With the sequence of events laid out and mediations identified the design team may now host white board design sessions to outline the internals of the mediation flow logic. The white board sessions should produce diagrams that depict the mediation flow logic.

Design Mediations in WebSphere Integration Developer

WebSphere Integration Developer V6.1 enables developers to build SOA-based integration solutions across WebSphere Process Server, WebSphere ESB, and WebSphere Adapters. Solution Architecture and Design teams can also use the tool for design of solutions targeted for deployment to these environments. WebSphere ESB V6.1.2 provides a predefined set of supported mediation primitives and using WebSphere Integration Developer V6.1 the design team can create graphical representations of the mediation flows with a common taxonomy (mediation primitives) for communication with development.

The intended use of the tool by the integration developer would be the detailed implementation of the mediations to support the design whereby that of the design team would be to generate design artifacts that communicate the mediation logic needed to support the requirements.

Business scenario: WebSphere Integration Developer design artifacts

The design effort in WebSphere Integration Developer V6.1 performed by the Solution Architecture and Design Team to generate the detailed mediation design artifacts are as follows:

1. For each Inbound and Outbound Service Contract

  • Define Shared Libraries for WSDL and Schema definitions

2. For each Inbound Service Contract (to be exposed as an SCA export)

  • Create Mediation Module
  • Generate SCA Assembly diagram
  • Create Mediation Flow Component

3. For each operation on the Inbound Service Contract

  • Construct Mediation Flow Logic -- Request and Response Flows

The design artifacts generated from these steps can then be packaged and exported as a Project Interchange file (see downloads) to be used as a reference by development. The following sections are extracts from the design documentation provided to development by the Solution Architecture and Design Team for the Synchronize Account Information flow.

TPC Banking ESB: External Artifacts Library:

Contains the WSDL and Schema definitions for services exposed to external ESB clients (i.e. CustomerAccountManagementService).

TPC Banking ESB: Internal Artifacts Library:

Contains the WSDL and Schema definitions for services exposed to internal ESB mediations (i.e. PostSynchCustAcctInfoService, ErrorHandlingService, ExceptionHandlingService).

TPC Banking ESB: LPA Artifacts Library:

Contains the WSDL and Schema definitions for the Loan Processing Application's LPACustomerAccountService.

TPC Banking ESB: ABPA Artifacts Library:

Contains the WSDL and Schema definitions for the Auto Bill Pay Application's ABPACustomerAccountService.

TPC Banking ESB: Synch Customer Account Info Mediation:

The Synch Customer Account Info Mediation is the ESB entry point for processing SynchCustAcctInfo requests from BMA. The mediation initiates the processing of the request by forwarding the request to the ESB's Post Synch Customer Account Info mediation. Once the request has been successfully forwarded for processing the mediation returns a response to BMA that the request has been received and submitted for processing.

The SCA Assembly Diagram and Mediation Flow Design are shown in figures 8 and 9:

Figure 8. Synch Customer Account Mediation -- SCA Assembly Diagram
Synch Customer Account Mediation -- SCA Assembly Diagram
Figure 9. Synch Customer Account Info Mediation -- Mediation Flow Design
Synch Customer Account Info Mediation -- Mediation Flow Design

TPC Banking ESB: Post-Synch Customer Account Info Mediation:

The Post-Synch Customer Account Info Mediation is invoked by the Synch Customer Account Info Mediation to mediate and forward the requests to the LPA and BMA Customer Account Services. The mediation receives the request and filters the requests based on the transaction type.

For Account Balance update transactions the mediation transforms and forwards the request to the ABPA Customer Service.

For Account Balance and Customer Data (contact/address) updates the mediation transforms and forwards the request to the LPA Customer Service.

All other transaction types are not forwarded to the ABPA or LPA Customer Account Services.

If a communication error occurs when invoking the ABPA and/or LPA Customer Account Services the mediation forwards the request to the Error Handling Service.

If a LPA Process Exception is returned upon invocation of the LPA Customer Account Service the mediation maps the exception to an ESB exception and forwards the exception to the Exception Handling Service. The SCA Assembly Diagram and Mediation Flow Design are shown in figures 10 and 11:

Figure 10. Post Synch Customer Account Mediation -- SCA Assembly Diagram
Post Synch Customer Account Mediation -- SCA Assembly Diagram
Figure 11. Post Synch Customer Account Mediation -- Mediation Flow Design
Post Synch Customer Account Mediation -- Mediation Flow Design

Design decision points to support requirements

As in every design it is important to document architecture and design decisions. In particular decisions on design patterns and applied best practices should be communicated and documented for reference by the development team.

Business scenario: Decision points

The decisions and design patterns applied by the Solution and Architecture team for the Synchronize Account Information Flow are listed below:

TPC Banking ESB: Solution Design Decisions -- Synch Account Information

Design decision:

  • Encapsulate implementation of Error Handling Policies for communication errors using an Error Handling Service.

Design pattern (Error and Exception Handling):

  • The ESB should invoke the Error Handling Service when an unresolvable communication or system exception error condition is detected.

Advantages:

  • Externalized and centralized error handling policy management
  • Separation of concerns; mediation logic for data transfer in ESB mediation and business logic for Error Handling Policies in Error Handling Service implementations.

Disadvantages:

  • Sequential and ordered delivery of requests is lost with hand-off to external service.
  • Event sequencing imposes data dependency between mediation and Error Handling Services to ensure incoming requests are not processed before those forwarded to the Error Handling Services.

Design decision (Transactional demarcation between receive and process requests):

  • Transaction demarcation between receiving of synchronous requests and routing of the request to target systems implemented with handoff to asynchronous endpoint.

Design pattern:

  • Source system submits a synchronous request to ESB.
  • T1: Source system sends request to ESB, ESB forwards request to a queue, ESB responds to request
  • T2: ESB retrieves message from queue, mediates and routes request to target system

Advantages:

  • Assured delivery of message for processing guaranteed.
  • Source system is not held hostage to downstream system processing of messages
  • Send from source and receive from target systems transactionally decoupled.

Disadvantages:

  • In event of communication failure to downstream systems source system is unaware of failed transaction.

Design decision (Ordered processing of requests):

  • Ordered processing of requests managed by FIFO Queuing policy.

Design pattern:

  • Source system submits a synchronous request to ESB. The receiving mediation asynchronously invokes (put on queue) a post mediation which retrieves message from queue, mediates and routes requests to downstream systems in the order they were received (Queue with FIFO). Post Mediation does not process next message until in-process message is committed as processed.

Advantages:

  • Ordered processing (FIFO) inherent on asynchronous endpoints.

Disadvantages:

  • Serial processing of requests where only one is in process at a time.

Error handling policies

Defining error handling policies should be an essential part of any Architectural Design, including that for an ESB Solution. Here are some important guidelines when defining policies for an ESB solution:

  • Policy definitions should outline the business impact of the error condition.
  • Each policy should have as its goal; reduce and eliminate the business impact in the most efficient and effective ways (not always easy). Thus as part of the policy's definition there should be a well articulated strategy for resolution of the error condition and indication on how it reduces or eliminates the business impact.
  • Each policy should be mapped to an originating triggered event upon detection of an irresolvable error condition. For instance, after the number of retries to invoke a service has exceeded the limit or when a system exception is returned on invocation of the service.
  • Policy definitions should categorize the types of irresolvable errors based on failure instances defined in the Use Cases. This ensures the policies are inline with the requirements. Note: It may be the case that new requirements are identified
  • Policy definitions should avoid explicit storing of transient payload within ESB beyond attempts to mediate and transfer the data.
  • The ESB should not be responsible for managing the failed processing of data by a downstream system. As a best practice the ESB should instead be responsible for mitigating errors in processing the transfer of data between systems.
  • Policy definitions for application errors (returned as faults or application exceptions) that occur at a target endpoint should not include resolution by the Error Handling Service. The ESB may have as a requirement the logging and/or forwarding of the fault from target to the source system. If a fault or application exception occurs it is the policy defined at the application level that applies. If a resubmit is required the source should initiate the resubmission using the ESB to facilitate data transfer.
  • Policy definitions should include implementation options and preferred choices for design consideration. It is important to have the business team aware of the effort required to implement a policy. This allows the business team the ability to weigh the business impact against the implementation effort and cost. If need be, it is not an uncommon practice to revise a policy to minimize the cost of implementation.
  • Policy definition should specify roles to associate with error handling tasks (e.g. ESB, Error Handling Service, System Administrator etc.)

Business scenario: Error handling policies

In collaboration with the Business Team and Application Owners the Solution and Architecture team were able to define error handling policies for the Synchronize Account Information and Adjust Account Balance flows. The policy definitions for when the ESB has detected that it cannot connect to one or more downstream services is summarized below:

TPC Banking ESB: Error Handling Policy -- Synch Customer Account Info (Outbound)

Failure Occurrence:

  • Down Stream System Connectivity Failure

Failure Outcome:

  • ESB Unable to invoke application services hosted on system

Error Condition Category:

  • CommSysError

Source System:

  • TPC Banking Co. -- Branch System

Mediator:

  • ESB

Target Systems:

  • Loan Processing System
  • Auto Bill Pay System

Business Relevance:

  • Loan Processing decisions are made based on account history information
  • Auto Bill Pay transactions are processed based on account balance information

Business Impact:

  • Errant Loan Decisions and Auto Bill Pay transactions

Policy Actors (roles):

  • ESB System
  • Error Handling Service
  • Loan System IT Administrator
  • AutoBillPay System IT Administrator

Policy Error Resolution Strategy:

Actor = ESB System:

1. ESB System detects irresolvable communication error condition and submits a HandleCommError request to Error Handling Service.
2. ESB System goes into a CommSysError state and forwards subsequent data requests to the Error Handling Service until CommSysError state is indicated as cleared by the Error Handling Service.
3. ESB resumes normal processing of subsequent requests once CommSysError state is indicated as cleared.
4. The successful processing of subsequent requests to downstream systems resolves the backlogged business impact.

Actor = Error Handling Service:

1. Error Handling Service receives a HandleCommError request and notifies Loan Processing and/or Auto Bill Pay System IT Administrators of Error Condition.
2. Error Handling Service queues incoming SynchCustAcct requests and logs the transaction for auditing purposes.
3. Error Handling Service exposes interaction mechanism for Loan Processing and/or Auto Bill Pay System Administrators to indicate error resolution.
4. Error Handling Service awaits indication from Loan Processing and/or Auto Bill Pay System Administrators that communication error is resolved.

If Loan System:

5. Error Handling Service processes the SynchCustAcct requests (per account) in the order they were received for queued transactions. NOTE: Account history is of the utmost importance for making loan decisions

If Auto Bill Pay System:

6. Error Handling Service processes the latest SynchCustAcct requests (per account) for queued transactions. The current balance on the account is what is required for Auto Bill Pay transactions.
7. Error Handling Service continues to accept incoming requests until queue backlog is processed at which points it indicates to the ESB that the CommSysError resolved. This should be synchronized with the processing of the queue backlog.
8. Error Handling Service notifies Loan Processing and/or Auto Bill Pay System IT Administrators of Error Backlog Processed.

Actor = Loan System Administrator:

1. Loan System Administrator receives error condition notification and works to resolve Loan System issues.
2. Loan System Administrator updates Loan Processing Application (LPA) to indicate all loan processing decisions require manual Branch confirmation of accuracy of account history until error condition is resolved.
3. Loan System Administrator interacts with Error Handling Service to indicate when problem is resolved.
4. Loan System Administrator files report of error occurrence and resolution.
5. Loan System Administrator receives notification from Error Handling Service that backlog has been processed and clears indicator on LPA to remove manual Branch confirmation requirement.

Actor = AutoBillPay System IT Administrator:

1. Auto Bill Pay System IT Administrator receives error condition notification and works to resolve Loan System issues.
2. Auto Bill Pay System Administrator sets status of Auto Bill Pay Application (LPA) to decline all incoming Auto Bill Pay requests; customers will have to send manual payments until problem is resolved so none are processed with errant account balance information.
3. Auto Bill Pay System Administrator interacts with Error Handling Service to indicate when problem is resolved.
4. Auto Bill Pay System Administrator files report of error occurrence and resolution.
5. Auto Bill Pay System Administrator receives notification from Error Handling Service that backlog has been processed and resets status to allow processing of incoming Auto Bill Pay Requests with current account balance information.

Implementation options
ActionOptions
ESB forwards request to Error Handling Service



Option 1: ESB Forward request via synchronous Service Invoke primitive
Pros:
§Simple invocation from mediation flow
Cons:
§None
Error Handling Services notifies Loan and/or Auto Bill Pay System IT Administrators



Option 1: Error Handling Service automates the generation and submission of notifications via WebSphere Email Adapter
Pros:
§Minimal coded logic
§Reuse of Adapter functionality to submit emails
§SCA binding and service call
Cons:
§Additional framework of design for submitting email via Adapters

Option 2: Error Handling Service automates the generation and submission of notifications programmatically via JavaMail APIs
Pros:
§Programmatic and controlled by application
Cons:
§Additional business logic for programmatic implementation
§Additional framework of design for submitting email via JavaMail APIs
Error Handling service provides interaction mechanism for Loan System and/or Auto Bill Pay IT Administrators Option 1: Human Interaction via WebSphere Process Server Human Tasks
§Minimal coded logic
§SCA binding and service call to Human Task Component
§Pre-existing UI provided out of the box for rapid deploy and test.
Cons:
§Additional framework of design for Human Task initiation and processing

Option 2: Interaction via Web Interface to programmatic interface Java API or Web Service Call
Pros:
§Technology agnostic
Cons:
§Additional Coding of User Interface not already in plan
§Extensive coding on the client and server side to support interactions
§User interface for initiating programmatic invocation still required
Preferred implementation options
ActionOption
ESB forwards request to Error Handling Service
ESB Forward request via synchronous Service Invoke primitive
Error Handling Services notifies Loan and/or AutoBillPay System IT Administrators
Error Handling Service automates the generation and submission of notifications via WebSphere Email Adapter
Error Handling service provides interaction mechanism for Loan System and/or AutoBillPay IT Administrators Human Interaction via WebSphere Process Server Human Tasks

New ESB requirements and change requests:

  • ESB should detect irresolvable communications errors to downstream services and should support invocation of the Error Handling Service and functional behavior as outlined in the Error Handling Policy to detect error resolution and proceed with normal processing of requests.

Requirements for error handling service -- LPA and ABPA are documented separately.

Conclusion

This article has prescribed a holistic approach to designing mediations for deployment to WebSphere ESB. The work breakdown structure used to illustrate the design effort, within the context of a business scenario, included tasks to generate the prerequisite design artifacts as well as those for the design and documentation of the mediations.


Download

DescriptionNameSize
Code sampleESBDesignArtifacts.zip89 KB

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=360782
ArticleTitle=Designing ESB mediations for deployment to WebSphere Enterprise Service Bus
publish-date=12302008