Ever since the IBM WebSphere Enterprise Service Bus (WebSphere ESB) product became available in December of 2005, we have been asked how this relates to the solutions we described in our earlier article series, which created an ESB based on WebSphere Application Server V6 -- or, more specifically, the Service Integration Bus (SIBus) technology in WebSphere Application Server V6. We received questions asking what is the relationship (and, respectively, what is the difference) between SIBus and WebSphere ESB, and which technology to use when. Some people simply wondered how they could use the features of WebSphere ESB to deliver the same ESB capabilities we described in our articles. And finally, there were questions about best practices for using WebSphere ESB in conjunction with another IBM product, namely WebSphere Process Server.
We are writing this new series to help accelerate your use of WebSphere ESB as well as to address these questions, which you also may have been asking. In this introduction to our series, we will explore how WebSphere ESB offers features beyond SIBus, how it in fact makes use of SIBus features, and set the stage for what you can expect in the remainder of this series. At times we will reuse context and information from the previous articles (hence, we will repeatedly refer to it), and also point you to complementary, informative articles on WebSphere ESB and ESB technology in general, rather than repeat content that is already available.
With that, let's begin.
In short, an ESB offers capabilities to connect new and existing software applications within and across enterprises, with a rich set of features enabling management and monitoring of interactions between applications. An ESB enables service virtualization, which provides many dimensions of decoupling between service requesters and service providers. A thorough discussion of the ESB as an architectural pattern and its components can be found in other articles available on developerWorks (see Resources).
What ESB features do we consider key?
First, is the ability for the ESB to interact with service requesters and service providers, in a variety of ways: by sending to and receiving messages from a persistent messaging backbone, particularly MQ, and being able to send and receive Web services request and response messages over HTTP and JMS, with with support for standards like the Web Services Interoperability (WS-I) Basic Profile 1.1. We will focus initially on standards based messages using SOAP and XML, though supporting other message formats such as text and binary are important, too.
Next, is the ability to convert between different message and transport protocols, such as SOAP over HTTP to SOAP over JMS.
Then, we want to be able to transform XML messages using the popular transformation language XSLT.
Another basic feature is to be able to apply message mediations such as logging. Moreover, we very frequently have some type of context-based dynamic routing taking place.
Advanced features that we may look at include monitoring of messages, lookup of endpoints in a services registry, and asynchronous request/response.
In our earlier articles, we explained and implemented many of the key items listed above, and chose to use SIBus in WebSphere Application Server V6 as the underlying platform. All of the examples were set in a common business context which is revisited below. For those of you who read our previous series, these new articles will seem somewhat familiar, since we will be reusing the scenarios and business context from the earlier series, and showing you how to implement these same features in WebSphere ESB.
We assume that you probably already know some basics about WebSphere ESB; the basic architecture and functionality of WebSphere ESB and its related tooling (IBM WebSphere Integration Developer) have been covered well in other articles (see Resources). Rather than cover the same ground, we will instead share with you the things we find particularly interesting in WebSphere ESB.
WebSphere ESB is very standards focused, with first class support for XML and SOAP. We like standards because they make developing service requesters and providers a lot simpler and faster, given the tooling and run time support that exist for the standards. WebSphere ESB leverages the strategic SCA/SDO programming model, where the focus is to enable the (visual) assembly of components via tooling. At the core of WebSphere ESB is the Mediation Flow Component, which is a specific type of SCA component. The SCA/SDO programming model is being standardized and is reused across many different component types. For example, WebSphere Process Server also supports the SCA programming model and introduces a large set of component types, like business processes and human tasks. Being based on the same programming model and associated tooling enables WebSphere ESB to be easily integrated with WebSphere Process Server components.
Moreover, we like the extension to SDOs -- called Service Message Objects (SMO) -- which enables us to access message context and content, which we need for routing and other QoS mediations. We like the pre-built mediation primitives -- for example, we use the XSLT mediation primitive in almost every project. And when we need to get to existing systems that do not support SOAP or XML out of the box, the WebSphere Adapters can save us a lot of development time, since they serve as just another SCA component type that can be "wired" to the WebSphere ESB mediation.
Furthermore, we are excited about the new features and enhancements coming in WebSphere ESB V6.0.2 in December (we will switch to using it for this series as soon as it is available!). WebSphere ESB 6.0.2 has added key features to enable greater dynamic behavior:
- We can administratively set properties and values on a mediation module, which can provide control over this mediation at run time.
- There is a WebSphere Service Registry and Repository mediation primitive to lookup the endpoint address of a target service provider at run time; alternatively, a custom mediation primitive can be written to do the lookup and set the endpoint. (See Resources)
- Administrators can change endpoint addresses in the administration console.
Moreover, in Version 6.0.2, there is improved support for connecting to WebSphere MQ through a WebSphere MQ SCA native binding, plus enhancements to the JMS SCA binding to make it easier to handle non-XML data. There are also more options in V6.0.2 for management with a new mediation primitive that generates events for the Common Event Infrastructure (CEI). IBM Tivoli® Composite Application Manager (ITCAM) for SOA 6.1 will be released this year with support for monitoring SCA modules, along with mediation primitives for WebSphere ESB that can be administratively enabled and disabled.
Last, but not least, there are significant performance enhancements for WebSphere ESB and usability improvements in WebSphere Integration Developer V6.0.2.
In the previous series, we described a sample business scenario and the resulting challenge for which an ESB would offer a good solution: Our ESB scenarios were based on a fictional shipping company called Posts-R-Us, and the common business challenge they faced involved many disparate systems that should be integrated with one another. Integrating these systems would enable them to adjust to new and changed business processes more quickly, without the need to manually develop new functions or continuously develop integration logic.
Now, what is important to keep in mind here is that the business level requirement did not directly ask for an Enterprise Service Bus. We assume that Posts-R-Us decided to start building a Service Oriented Architecture (SOA) so they could create a set of services to directly address business-related problems. An ESB provides a level of indirection and decoupling between service requesters and service providers that enables better separation of concerns. In other words, integration logic (such as protocol transformation, message format transformation, and so on) is separated from business logic (that is, the actual function that a service performs). An ESB can provide virtualized services that can effectively wrap existing application logic with a standards-based service interface. Hence, the ESB helps to bridge the gap between the business problem (that is, providing a flexible set of services that can implement required business processes) and the existing IT environment with all its protocols and formats and existing legacy applications.
For this article series, we will assume the exact same business scenario, and simply map it to a different product for implementation, namely WebSphere ESB.
The term "SIBus" actually includes multiple different pieces, which we need to analyze separately when comparing it to WebSphere ESB. At the core of SIBus is the messaging infrastructure and the default JMS provider in WebSphere Application Server, and hence, every WebSphere product built on this base uses this default JMS provider for messaging. Moreover, there are the so-called mediation handlers which enable in-flight access to messages. Finally, there is a feature called MQLink that enables integration between SIBus and a separate WebSphere MQ environment. With that in mind, let's take a look at the relationship between WebSphere ESB and SIBus, and specifically at these five ways in which the two are related:
- Features that WebSphere ESB and WebSphere Integration Developer provide beyond SIBus for building an ESB
- The WebSphere Application Server/SIBus default JMS provider as the JMS provider for the WebSphere ESB JMS bindings
- Differences between SIBus mediations and WebSphere ESB mediations
- Using SIBus MQLink with WebSphere ESB
- Co-existence and integration of SIBus and WebSphere ESB infrastructures
However, when using bare SIBus functionality, the configuration process can be cumbersome and requires several manual steps. Moreover, mediations (in SIBus, called mediation handlers) must be coded manually with no tool support.
Since WebSphere ESB is built with the strategic SCA/SDO programming model, it has strong tooling in WebSphere Integration Developer that simplifies the development of mediation flow components with visual construction of imports and export interfaces. Moreover, as the name indicates, a mediation flow component itself contains a flow, which describes how a message is processed as it traverses through the ESB. WebSphere Integration Developer provides another visual tool to assemble a mediation flow, letting you drag and drop pre-built mediation primitives that form the mediation flow component through which a message flows. For example, there is a mediation primitive that supports message conversion with XSLT stylesheets, and another that supports automatic logging of messages that go through the flow. A good article describing how this works is Getting started with WebSphere Enterprise Service Bus and WebSphere Integration Developer. You can also build your own custom mediation primitives containing specific logic that is otherwise not already available in the tool. Developing custom mediations for WebSphere Enterprise Service Bus is an article that describes how to build a custom mediation primitive.
The notion of visually modeling the flow is something not supported by the SIBus mediation handler framework at all. As you may recall from our earlier series, the handlers had to be created as plain Java™ classes implementing a specific interface. No visual tooling is provided, and no pre-defined handlers exist.
WebSphere ESB also has additional support for different interacation patterns with automatic correlation of request and response messages for synchronous invocations and an SCA API for invoking components asynchronously, providing both a callback mechanism and correlation between requests and responses. Using SIBus, a developer must custom code asynchronous support and correlation logic.
Therefore, the combination of WebSphere ESB and WebSphere Integration Developer creates a powerful development and run time tool set to speed development of the ESB significantly beyond what is available in SIBus.
WebSphere ESB, just like many other products in the WebSphere family, is built on top of WebSphere Application Server. It uses the J2EE™ runtime as well as other features supported by the application server; for example, its ability to support clustering and workload management. SIBus is a functional feature of WebSphere Application Server; hence, each installation of WebSphere ESB automatically has SIBus functionality available as well.
One way of communicating with an SCA component (for example, a WebSphere ESB mediation flow component) is to use JMS bindings. A client, or requester, sends messages to the ESB via JMS and -- in the case of a request/response operation -- receives the response message over a separate JMS queue. WebSphere ESB utilizes the default JMS provider to provide the JMS implementation. It is therefore fair to say that, from a JMS perspective, WebSphere ESB sits on top of and uses the SIBus JMS provider -- or, that WebSphere ESB and SIBus are complementary. Note that when creating JMS bindings for WebSphere ESB, the WebSphere Integration Developer tool will automatically generate the required SIBus artifacts (such as destinations, and so on), making the use of the JMS provider transparent to the user.
As mentioned above, both WebSphere ESB and SIBus offer mediation functionality. WebSphere ESB lets you develop your own custom mediation primitives, and SIBus utilizes a mediation handler framework. It is important to note, though, that the mediation framework implementations, APIs, packaging, and administration model for each are different.
In almost all cases, mediations require access to the message itself so that it can be manipulated, logged, transformed, or whatever the specific use case may be. Both SIBus and WebSphere ESB use SDO as a representation mechanism for messages flowing through the bus. In the case of WebSphere ESB, an extension to SDO called Service Message Objects (SMO) is used. Hence, even though the exact representation of a message is different in the two technologies, the code required to access the message content is similar.
In most cases, migrating from SIBus to WebSphere ESB (or visa versa) will need to be done manually -- the complexity of the migration will depend on what the mediation code is doing. In an upcoming article in this series, we will show how you can replace the existing mediation handler that we developed for use with SIBus with a custom mediation primitive when using WebSphere ESB, and explore considerations for migrating.
And again, SIBus offers no prebuilt mediation handlers, whereas WebSphere ESB comes with a number of mediation primitives so that you might not be required to write custom code at all.
Since SIBus acts as the JMS provider for WebSphere Application Server, and is hence reused by WebSphere ESB whenever JMS functionality is required, you can easily integrate WebSphere ESB with an existing WebSphere MQ environment by leveraging additional functionality offered by SIBus.
SIBus offers connectivity into WebSphere MQ via its MQLink feature, which enables a SIBus instance to communicate with an MQ queue manager as if it were yet another queue manager. This enables sending a message from an MQ client to an MQ queue, and then forwarding the message automatically to a destination in SIBus. From there, the message is sent into a mediation flow component through a JMS export. Similarly, a message can be sent from within WebSphere ESB/SIBus to a destination that is really an MQ queue.
However there are a couple of drawbacks to this approach of integrating WebSphere ESB with WebSphere MQ. The developer must be familiar with and use the SIBus resources for the configuration, plus there are limitations on the configurations for failover which are supported by MQLink.
In WebSphere ESB V6.0.2, a new MQ SCA native binding is added and we recommend using that over MQLink.
Finally, WebSphere ESB and SIBus can coexist and separate instances can be integrated. If you have an existing SIBus environment already, you can integrate one or more instances of WebSphere ESB to the environment through configuration of messaging engines and across cells via foreign bus links. SIBus and WebSphere ESB can exchange messages via several protocols; for example, via SOAP/HTTP or JMS. You can start running messages through mediation flow components without affecting the existing SIBus mediations at all and then, over time whenever you are ready, you can migrate the mediations from SIBus to WebSphere ESB.
For the subsequent articles in this series, we will pretty much stick with the technical scenarios we used before. In these articles, we will:
Detail how to perform some advanced Web services functions; for example, how to manipulate SOAP headers.
Go over the protocol transformation example we used before, where a JMS-based client communicates with a SOAP/HTTP Web service via the bus.
Discuss using WebSphere ESB and WebSphere Process Server together, with an example.
Discuss how to migrate an SIBus mediation handler to a WebSphere ESB mediation primitive, using one of our existing mediation handlers as an example. We will use the JMS messaging example and logging mediation from the previous series, this time implementing and deploying it on WebSphere ESB.
An introduction to the IBM Enterprise Service Bus
Building an Enterprise Service Bus with WebSphere Application Server V6
Building a powerful, reliable SOA with JMS and WebSphere ESB
WebSphere Enterprise Service Bus
WebSphere Enterprise Service Bus V6.0.2 announcement
Getting started with WebSphere Enterprise Service Bus and WebSphere Integration Developer
Developing custom mediations for WebSphere Enterprise Service Bus
Introducing IBM WebSphere Service and Repository, Part 1: A day in the life of WebSphere Service Registry and Repository in the SOA life cycle
Dynamic routing at run time in WebSphere Enterprise Service Bus
Tutorial: Invoking a Web service with a JMS client
Redbook: Enabling SOA Using WebSphere Messaging
Rachel Reinitz is an IBM Distinguished Engineer with IBM Software Services for WebSphere and a member of the IBM Academy of Technology. She has built much of IBM’s internal and external intellectual capital and education on SOA, Enterprise Service Bus, and Web services. Rachel focuses on how to get started with SOA and ESB best practices. She consults with clients and ISVs on how SOA and ESB can be used to achieve their business and technical objectives. She is a frequent conference presenter and written many developerWorks articles on ESB and Web services. Rachel lives in the Bay Area in California, and enjoys hiking, socializing, and international travel.
Andre Tost works as a Senior Technical Staff Member in the Software Group's Enterprise Integration Solutions organization, where he helps IBM's customers establishing Service-Oriented Architectures. His special focus is on Web services technology. Before his current assignment, he spent ten years in various partner enablement, development and architecture roles in IBM software development, most recently for the WebSphere Business Development group. Originally from Germany, he now lives and works in Rochester, Minnesota. In his spare time, he likes to spend time with his family and play and watch soccer whenever possible.