Part 1 of the article series focuses on the design of the integration scenario. After introducing the specific integration pattern used for this scenario, it takes you through the process of identifying the integration requirements, choosing the middleware, and designing the mediations and flows in the middleware. The upcoming Part 2 will focus on using WebSphere Integration Developer and WebSphere Commerce Developer to implement the scenario.
Service-oriented architecture (SOA) is an integration architecture approach that uses services as the basic building blocks for building business functionality. SOA specifies that with any integration there is a consistent, but yet loosely coupled communication mechanism between the services. By adopting an SOA-based approach, you ensure that the integration is flexible, but yet can rapidly adapt to changes in the business.
The Enterprise Service Bus (ESB) is a middleware infrastructure component that enables enterprises to implement an SOA. The ESB addresses three concepts of SOA:
- Decoupling service implementation from the consumer's view of the service
- Integrating, deploying, and managing services in the enterprise
- Decoupling the technical aspects of the interactions between the services
SOA runtime patterns are used to identify logical middleware structures that support integration of applications or of other application patterns. This article does not delve into the details of the patterns. For more information, see IBM Patterns for e-business.
The SOA ESB runtime pattern can route and transport service requests and read messages from the service consumer to the correct service provider. Figure 1 gives a high level overview of the ESB runtime pattern.
Figure 1. ESB runtime pattern
The ESB hub, in the center of Figure 1, is responsible for routing, transforming, and transporting messages among a variety of other functions. You can also augment the support for transporting and messaging protocols using specific adapters to extend the ESB to adapt to:
- Technologies like CORBA, COM JDBC, and so on
- Applications like SAP®, Siebel®, and so on
- Legacy applications like the ones in a CICS® transaction server
In conclusion, you can use the ESB to integrate applications using a variety of messaging and transport protocols even though the message structures may not be supported or understood by the various applications that are being integrated. This pattern provides a good infrastructure that you can use to connect, exchange, transform, and transport messages to and from applications. Therefore, the applications become service providers or service consumers, which can render a service to the enterprise or consume one from it.
This ability to connect applications and services also forms one of the integral components of the IBM SOA architecture as shown in Figure 2.
Figure 2. IBM SOA reference architecture
The remainder of this article uses a real life scenario that goes through the process of identifying, designing, implementing, and testing the ESB.
This section outlines a scenario of a typical e-Commerce system that captures an order from the Web front end, and then sends the order back to the order management system for fulfillment.
An order captured in the WebSphere Commerce application is processed by WebSphere Commerce and then sent to a backend system for fulfillment. WebSphere Commerce does a good job of capturing the order and persisting the order information to the database. But when it comes to fulfillment, typically done at a warehouse, WebSphere Commerce has to rely on a third party order management system.
We will extend the business functionality of WebSphere Commerce by sending the order to the order management system for fulfillment. The end-to-end process flow is depicted in Figure 3.
Figure 3. End-to-end order process flow
The scenario defined enacts the process of capturing the order from WebSphere Commerce and the order information being sent to a backend. The backend system acknowledges the order, and then proceeds to update the status of the order back into the WebSphere Commerce application at various stages of the order fulfillment.
A J2EE application is used as a sample stub for the order management system or the enterprise resource planning (ERP). This application receives and stores the orders. As part of simulating the order fulfillment steps, the J2EE application also has an interface where you can poll the order status. The updated status is then sent back to the WebSphere Commerce application through the ESB.
The detailed scenario is as follows:
- WebSphere Commerce captures the order and sends it to the order management system.
- The order management system automatically acknowledges the receipt of the order.
- After the order is shipped, the order management system is manually updated, and the status of the order now changes to "Shipped". After the update is done on the order management system, it sends out a message with the updated order status back to the WebSphere Commerce application.
- You can make a manual request to generate the invoice, which then flows into the WebSphere Commerce application.
Figure 4. Order process scenario
Application integration patterns
This section briefly studies the various application integration approaches that you can use for the scenario. There are multiple ways of integrating applications. The integration approaches are broadly classified as:
- Process integration: Functional integration where the process flow spans multiple applications.
- Data integration: Information integration where the data replicates or flows from one application (database) to another (database).
We are interested in the former, process integration, because we have defined a process flow that spans two applications, WebSphere Commerce and the legacy order management system.
In process integration, there are four variants:
- Serial processes
- Parallel processes
- Direct connection
Figure 5. Process integration application patterns
More details of these variants are found in IBM Patterns for e-business. However, for the purpose of this article, we ignore the serial and parallel processes, and choose from a direct connection versus a brokered approach.
A comparison of the relevant features of these two patterns is shown in Table 1.
Table 1. Feature comparison of direct and brokered integration patterns
|Support structured exchange within the organization||Yes||Yes|
|Support real-time one way message flows||Yes||Yes|
|Support real-time request-reply message flows||Yes||Yes|
|Support dynamic routing of messages to one or many target applications||No||Yes|
|Support dynamic distribution of messages to many target applications||No||Yes|
|Reduce enterprise complexity||No||Yes|
Choosing between the direct connection and broker patterns
Direct connection pattern
Direct integration between applications can be inflexible, in that any changes to one application may have knock-on effects on other applications. Changes to the target application may also require changes to the source application. Such changes can become both expensive and time consuming, especially when the target application is being accessed by a number of different source applications.
Different IT departments may also be responsible for developing and maintaining the source and target applications. Under such a scenario, development might be difficult to coordinate, especially if the interfaces between the applications being integrated are not properly defined and documented. Because of this, it is important to clearly define such interfaces in advance.
The broker pattern ensures a higher level of de-coupling between the source and the target applications. Changes in the individual applications can more or less be localized to them.
To increase the flexibility of the solution and responsiveness to changing business requirements, you can define reusable messages that flow through the broker.
You have seen that various methods of integration and all of them are perfect for the scenario that they address. We now need to identify the integration pattern that best suits the scenario. The first step is to identify the integration requirements from the business scenario given. There are three requirements to address: transport, transformation, and routing. The following sections describe these requirements within the context of the scenario.
WebSphere Commerce has a rich set of business services exposed through the Web services framework, which is described in subsequent sections. The order management system also exposes business services on a Web service port. The order management system exposes Order Receive as a service. WebSphere Commerce exposes Order Status Update and Invoice status update as services, which are used in this article.
The first requirement is that the applications connect to the middleware using Web services.
The applications exchange information in the form of messages that follow specific interchange standards. In an ideal world, the two applications that need to be integrated exchange messages directly without any change in the structure of the messages. However, the world we live in is far from ideal so we come to our second requirement, which is to transform the messages that are being exchanged between the applications.
In a real life scenario, the orders that originate on the e-Commerce site in WebSphere Commerce are physically fulfilled by the fulfillment centers or warehouses where the goods are stored. Most businesses have more than one warehouse and it is important that the orders are routed to specific warehouses for fulfillment. This brings us to the third integration requirement, message routing. Routing messages based on information contained in the message or any other externally need to enforce business rules.
Choosing the pattern
Given the requirements outlined above and the feature comparison of the patterns in Application integration patterns, we can safely identify the broker pattern to be the one that addresses most of the requirements. We also choose the router variation of the broker pattern because of the requirement to route messages dynamically. Now we have identified the application integration pattern to use, the broker pattern with the router variation.
Let us now focus on how an ESB will fit into the requirements that have been specified up to this point. This will also further validate the pattern that we have selected.
- Provide a connection mechanism using standard protocols: Web services provide a flexible, but yet robust way of exchanging information between WebSphere Commerce and the ERP application. An ESB provides a seamless way of integrating applications that can expose business functionality using a variety of protocols.
- Transform the messages from one format to another: An ESB also transforms messages from a wide variety of standard formats. It also has the capability to do legacy transformations.
- Route messagesi: An ESB has advanced message routing as service aggregation features.
The primary driver for an ESB is that it increases the decoupling between WebSphere Commerce and the ERP application. Therefore, an ESB is the chosen integration method for this form of integration flow.
Figure 6 shows a high level overview of the ESB.
Figure 6. The Enterprise Service Bus
Choice of the middleware platform
Given the integration requirements up to this point, there are three middleware platforms to choose from:
- IBM WebSphere Enterprise Service Bus
- IBM WebSphere Message Broker
- IBM WebSphere Datapower® Integration Appliance
The choice of the middleware platform is an involved task and there are various considerations. We will not discuss the pros and cons of the different ESBs in this article. For information about the ESBs and choices, see the WebSphere Enterprise Service Bus site.
For the purpose of this article, we decided on using WebSphere Enterprise Servcice Bus based on the assumption that the load will be minimal and the clustering and advanced security features offered by Message Broker and DataPower.
We also do not use any connectors to augment the ESB functionality to keep the scenario simple and easier to understand.
WebSphere Enterprise Service Bus
WebSphere ESB provides a seamless way of integrating applications that can expose business functionality using a variety of protocols, including any JMS provider, WebSphere MQ, TCP/IP, SSL, and HTTP. WebSphere ESB also supports a wide range of interaction models and Web services support using SOAP/HTTP, SOAP/JMS, and so on.
For this scenario, we use the transport, transform, and routing capabilities offered by WebSphere ESB. WebSphere ESB has many more features that you can use in specific scenarios that warrant them. These include, but are not limited to, the following:
- Routing: This function removes the need for applications to know anything about the bus topology or its traversal. The interaction that a requester initiates is sent to one provider or multiple providers where there is a service aggregation needed.
- Addressingi: Addressing complements routing to provide location transparency and support service substitution. Service addresses are transparent to the service consumer and can be transformed by the ESB.
- Messaging styles: The ESB supports many messaging styles, including request and response, fire and forget, events, publish and subscribe, and synchronous and asynchronous messaging.
- Transport protocols: The ESB supports a wide range of transport protocols like HTTP, HTTPS, and so on. It can also provide protocol transformation. If a protocol transformation is required that is not supported by the ESB, then you can us a specific connector to perform the transformation.
- Service messaging model: The ESB supports a range of models, including SOAP, XML, or a proprietary Enterprise Application Integration (EAI) model.
In conclusion, WebSphere Enterprise Service Bus provides the necessary features that you can use as the middleware infrastructure for integration.
WebSphere Commerce Web services framework
To integrate with external application by industry-accepted standards for service definitions, WebSphere Commerce provides its own Web service framework. WebSphere Commerce can exist as both a service provider and a service consumer.
This article uses business operations available in WebSphere Commerce that are exposed as Web services. For example, the Order Status update is a service that is invoked through a Web service to update the status of an order that has been captured. WebSphere Commerce thus acts as a service provider. The Web Services Description Languages (WSDLs) for the supported services are published, which is used by any external system to invoke the service.
When handling Web service requests, the WebSphere Application Server Web services engine delegates the request to the WebSphere Commerce Web services framework. The framework processes the request and generates the response. To generate such SOAP responses, the framework leverages JSP composition services. Figure 7 shows an overview of WebSphere Commerce's Web services framework.
Figure 7. Web services framework in WebSphere Commerce
When business operations that reside outside WebSphere Commerce are invoked, WebSphere Commerce becomes the service consumer. For example, when the order is sent out to the external order management system, WebSphere Commerce is actually invoking an operation in the order management system. Therefore, WebSphere Commerce becomes the service consumer.
To conclude, you can use Web services exposed by WebSphere Commerce to use WebSphere Commerce as a service provider. WebSphere Commerce can also invoke services available in the ERP, thus becoming the service consumer. Both these invocations are routed through the ESB for flexibility in the overall integration process. For example, if at a later date the structure of the messages returned from the ERP changes, the ESB can transform those messages into a format that is acceptable to WebSphere Commerce.
Order management system touch points
The order management system used is a simple J2EE application that has been built specifically for this article. It mocks the behavior of a real world order management system in that it receives the order, acknowledges it, and then sends out messages to update the order status.
The order management system application exposes business services as Web services and accepts WebSphere Commerce messages directly. In a real life scenario, this might not be the case. However, for this scenario, we keep the integration simple.
The order management system exposes a Web service used by WebSphere Commerce to submit the order. This is acknowledged by the order management system to which the order will pass through various stages. Status updates are sent back to the WebSphere Commerce application.
Designing the ESB
The first step in this integration scenario, as with any other scenario, is to identify a means to connect with the applications or service providers that need to be integrated. A variety of integration options exist. WebSphere Commerce exposes business operations as Web services. This provides a good method to extend the processes predefined in WebSphere Commerce. We chose Web services as the method of interaction with WebSphere Commerce.
From the ERP, the business processes are exposed as services and therefore, we use Web services as the chosen form of interaction with the application.
Mediation modules and flows
The mediation module is the unit of deployment and runs within the ESB. The mediation module processes the messages between the service consumer and the service provider, therefore providing a loosely coupled connectivity and mediation services between the components connected to the bus.
Service requestors interact with the mediation module via the module exports and the module interacts with the service providers via the module imports. The exports and imports are defined using WSDLs.
Figure 8 shows the order flow mediation.
Figure 8. Order flow mediation
Let's consider the order flow. The order is captured in WebSphere Commerce and a scheduled job in the order processing subsystem releases this order to the ERP for fulfillment. This means that the order is sent to the ESB through a Web service interface.
In the ESB, the order is received by the mediation component and for WebSphere Commerce to connect to the ESB. The mediation component has to expose a Web service interface in the form of an export. In Figure 8, the OrderMediation is the mediation mediation flow and it exports the WCSOrderServiceExport for WebSphere Commerce to invoke.
Similarly on the ERP side, the OrderMediation invokes the Web service through an import, ERPOrder ServiceImport. The order that needs to be processed by the ERP is sent out to the ERP using this import. Both the export and the import are bound to the Web services ports.
The ERPCustomMap is a Java™ component used to map the input message structure to the output message structure.
Similarly for the order status and invoice status updates, there are imports from the WebSphere Commerce side, which will be used by the ESB to send messages to WebSphere Commerce. The order management system sends the status messages to WebSphere Commerce by invoking these exports on the ESB.
Figure 9. Order status mediation
The details on how to generate the implementations for these mediation flows will be described in the upcoming Part 2 of this series.
Message structures and mapping
WebSphere Commerce uses OAGIS standards for the interchange of messages. A sample message that WebSphere Commerce sends out is listed in OrderSample.xml that you can download from the samples.zip file.
This message is received by the ESB Order Mediation Module, where it is transformed into a structure that is understood by the ERP application. The transformation is done in a custom Java component (ERPCustomMap) where the ESB invokes the Web service on the ERP to submit the message. Details of the custom Java transformation and other sample transformations will be described in Part 2 of this series.
Similar messages are used for order acknowledgement, order status update, and invoice status update. Sample messages are given in ReceiveStatus.xml, StatusSample.xml, and InvoiceSample.xml, respectively. These files are provided in the samples.zip file.
This article identified patterns available to integrate two applications, WebSphere Commerce and a custom order management system. It described how to develop the requirements and the integration patterns that you can use, which led to products that are available for the integration, such as WebSphere Enterprise Service Bus. We then validated the product fitment against the requirements and then proceeded to design the ESB after having identified the transport, transform, and routing specifications.
Part 2 of this series will cover:
- Setting up the environment for implementation
- Creating the modules and implementing them in WebSphere Integration Developer
- Setting up the order management system
- Setting up the WebSphere Commerce environment
- Deploying the developed code in the respective test environments
- Testing the end-to-end flow
- IBM Patterns for e-Business
- Patterns for e-business: Application Integration pattern
- developerWorks WebSphere Enterprise Service Bus zone
- developerWorks WebSphere SOA zone
- developerWorks WebSphere Commerce zone
- Getting started with the ESB