We covered a lot of ground throughout this series of articles on the subject of the Enterprise Service Bus (ESB). We established some of the base characteristics of an ESB, and described a sample business scenario and the inherent challenge, for which an ESB would offer a good solution. We chose to implement several key design features included in many ESB implementations, then took the messaging support in IBM WebSphere Application Server V6 (as an example) to implement some of the business functionality we needed. This included support for basic messaging over JMS, the use of Web services, message mediation, and finally a multi-protocol message exchange. In this article, we will briefly revisit all of the pieces of the story we laid out to show how all of the individual parts come together to build one ESB-based solution.
In addition, we will talk about where you can expect things to go next. There have been announcements of several new IBM products and new product releases lately that relate to the Enterprise Service Bus. We will introduce you to some of the new technology that has been delivered, and explain how it relates to and complements the SIBus that we used for this series. This includes the arrival of a new and exciting programming model specification that is backed by IBM and a number of other vendors: the Service Component Architecture (SCA).
The series in review
Part 1: Service Integration Bus revisited
In Part 1 of this series, we described the basic functionality offered by what we call the Service Integration Bus (SIBus) in WebSphere Application Server V6. The SIBus offers basic ESB capabilities by providing "destinations" that serve as the targets for messages being sent to the bus. To manipulate messages that arrive at a destination (for example, to change its format or its routing) the SIBus uses the notion of mediations, each of which consists of one or more mediation handlers that have access to the message as its traverses the bus. Messages are converted into Service Data Objects (SDO) as soon as they arrive at a destination, which enables common access to the message content. SDO is one cornerstone of the IBM SOA programming model. It enables one common form of data access across many different data sources, making it easier to develop code (and tooling!) that works with data. Of course, this is useful for an ESB implementation in that it makes it possible to develop code that accesses messages independent from the source that created the message. IBM recently published another key piece of its programming model, which addresses common access to services, namely Service Component Architecture. SCA and SDO combined form a common model for dealing with services and the data they offer. More on this later.
Part 2: Business Scenario revisited
We based our ESB scenarios on the example of a shipping company called Posts-R-Us. The challenge that this fictional company faces is typical for a number of businesses, namely that it has a large number of disparate systems that should be integrated with one another to adjust to new and changed business processes more quickly, without the need to manually develop new functions or integration logic all the time.
What is important to keep in mind here is that the business level requirement did not directly ask for an ESB. We assumed that Posts-R-Us decided to start building a service-oriented architecture to build a set of services that directly addressed the business-related problems. The ESB provides a level of indirection and decoupling between service requestors and service providers that enables better separation of concerns. In other words, integration logic (for example, protocol transformation, message format transformation) is separated from business logic (that is, the actual function that a service performs). An ESB can provide virtualized services that can effectively wrap existing application logic with a standards-based service interface. Hence, the ESB helps to bridge the gap between the business problem (providing a flexible set of services that can implement required business processes) and the existing IT environment with all its protocols, formats, and existing legacy applications.
Eventually, this results in the high level architecture that is shown in the following figure, which is taken from Part 2 of our series:
Figure 1. Required connections simplified using ESB
Part 3: JMS Messaging revisited
Most ESBs will need to support a variety of message formats and protocols, including Web services and messaging, such as IBM WebSphere MQ and JMS. We implemented examples using JMS support, and later MQ support, to demonstrate the importance of non-Web services protocols and how to use these key protocols with SIBus. We defined support for JMS messaging as one of the core requirements toward our ESB solution in Part 3. The JMS protocol provides Qualities of Service which are frequently needed for the integration of systems, and therefore is a common ESB requirement in a Java 2 Enterprise Edition (J2EE) environment. For example, JMS contains support for assured delivery of messages, supports transactions, and provides an API for the publish/subscribe message pattern, to name a few. Moreover, it is a standard API that is part of J2EE, which is supported by many application servers across many different platforms, thus it ensuring good connectivity for heterogeneous environments.
The messaging resources in WebSphere Application Server V6 are the default JMS provider for the application server. JMS applications can directly tie into SIBus destinations by sending and receiving messages to and from JMS queues, making the SIBus transparent to message consumers and producers.
Part 4: Mediations revisited
As we mentioned above, mediations are SIBus components that enable the manipulation of messages. We covered a detailed example of how to create a mediation handler and deploy it into a mediation in WebSphere Application Server V6 in Part 4. Whenever a message arrives at a destination, be it sent from a JMS client or as the result of a web service invocation, it is converted into an SDO DataGraph and forwarded to any mediation that is associated with that destination. The mediation handlers that are contained in the mediation can read and change the message, the contextual information, and even the routing (that is, where the message is sent next). This effectively lets you apply integration logic to the message, which is one of the core features of the ESB. The mediation model has flexibility to meet a large variety of requirements, including content-based routing, logging, and data transformation.
Part 5: Web services revisited
In Part 5, we discussed the Web services support in the WebSphere Application Server V6 SIBus. An important feature of an ESB is support for multiple protocols, both messaging based protocols and Web services. Web services are becoming a main steam technology for implementing services and providing interoperability between heterogeneous systems. The implementation independent, standards-based interface described by WSDL decouples the implementation technology of the provider and the consumer. Use of an ESB between the provider and consumer provides an opportunity to add additional dimensions of decoupling:
- The consumer is no longer impacted by changing the location of the provider.
- The ESB may support the consumer using a different protocol than the provider.
- The ESB may transform between the consumer's message format and the provider's format.
- The ESB may provide persistent queueing support which can be used to decouple the provider having to be available when the consumer sends the service request.
In earlier articles, we described how to build an ESB as a level of indirection for service interactions, with support for certain quality of services (QoS), plus message manipulation, and routing, just to name the key features.
WebSphere Application Server V6 and SIBus fully support Web services interactions using SOAP over HTTP and SOAP over JMS. It uses so-called outbound services to forward messages to an existing Web service, and inbound services to provide a Web services interface to a requestor. Under the covers, it uses the built-in Web services support for WebSphere Application Server V6, which is based on the relevant J2EE standards for Web services. Advanced Web services features such as JAX-RPC handlers and WS-Security can be used with inbound and outbound services in SIBus. Information on these features is available on developerWorks and the WebSphere Application Server Information Center.
Part 6: WebSphere MQ connectivity revisited
Another common requirement for an ESB is support for native WebSphere MQ connectivity, which was described in Part 6. There is a strong requirement for this in many organizations, simply because WebSphere MQ has been around for a long time and has essentially established itself as the de-facto solution for enterprise messaging. It is widely used for application integration, so that any new integration technology (like our ESB) must take support for it into consideration. In fact, many applications already provide interfaces into WebSphere MQ, making it easier for the ESB to take advantage of them.
WebSphere Application Server V6 comes with a feature called MQLink that makes SIBus appear as just another WebSphere MQ queue manager. This enables seamless integration of SIBus into any existing WebSphere MQ functionality, keeping the impact of its introduction to existing legacy applications to a minimum. To use MQLink, configuration is required in both the SIBus and the MQ queue manager, which is shown in Part 6.
Part 7: Message protocol switching revisited
One remaining key requirement toward the building of an ESB is the ability to switch messages between different protocols, which was covered in Part 7.
In our example, Posts-R-Us implements a common reusable service as a Web service and makes it available via the ESB. The ESB can also make this service available for clients that cannot use the offered SOAP/HTTP interface directly. This is where one of the advantages of the ESB becomes obvious. We demonstrated how you can use the SIBus to connect an MQ message queue that interacts with client applications with an existing Web service that offers the actual business function. This reusable service now only needs to be built once, and again, we have separation of concerns between developing the appropriate business logic and enabling this function to be accessible across the enterprise.
The protocol and message transformations, which we call integration logic, is built using mediations in the SIBus, without affecting the rest of the system at all. The mediations know about incoming and outgoing protocols and use the support of SIBus to utilize them. Potential message format changes are done using the relevant SDO interfaces.
This final scenario closed the initial set of ESB capabilities we set out to describe: support for messaging, routing, data transformation, Web services, and multi-protocol messaging. On top of that, many more issues remain: support for more advanced capabilities (like SOA management and security), tooling that offers easy mechanisms for developing integration logic, and a common service invocation programming model. The next section covers some of the support you will find in the next generation of ESB products that IBM is rolling out right now.
IBM's ESB strategy going forward
IBM recently announced the new WebSphere Enterprise Service Bus product (WebSphere ESB), WebSphere Message Broker V6, and the new SOA programming model composed of Service Component Architecture and Service Data Objects.
IBM continues to believe that an enterprise service bus is an architectural pattern. Customers implement their own unique ESBs based upon their requirements. As every business is different, so every business's need for connecting processes, applications, and data together is different. In fact, it is possible to construct service buses from a variety of different ESB products, including WebSphere Application Server and its SIBus functionality, WebSphere Message Broker, and WebSphere ESB. WebSphere ESB is in fact an ESB-centric product that facilitates the creation of specific instantiations of that pattern.
WebSphere ESB and WebSphere Message Broker offer a variety of pre-built mediation functions, visual construction tooling, and other features far beyond the basic messaging support in SIBus. WebSphere ESB is tightly integrated with WebSphere Process Server, and its mediation model uses SCA and Service Message Objects (an extension of SDOs to include message headers and context to the data graph).
IBM, in collaboration with industry partners, has introduced a high-level abstraction for constructing SOA applications that enables you to assemble components without direct knowledge of the underlying technologies being used by those components, and then customize the implementation of components in your system through metadata properties. SCA is a language- and technology-neutral architecture to enable service composition without regard for what language and technology is used to implement components. The service component should capture the logic that is intended for that component. Wires can be defined between components with support for late binding to implementations of other components; integration developers focus on wiring together service components. SDO complements SCA. They provide a common way to help users tap into data residing in multiple locations and formats, making it easier for developers to use application programming interfaces without having to code to them.
So what does WebSphere ESB, WebSphere Message Broker V6, and SCA mean to the future of SIBus? SIBus continues to be the default JMS provider and core messaging platform for WebSphere Application Server. WebSphere ESB and WebSphere Process Server add functionality on top of WebSphere Application Server and use SIBus for core message processing. You can think of SIBus as providing the basic plumbing for WebSphere ESB and WebSphere Process Server. IBM reinforces its commitment to customers who currently use SIBus as a core piece of their ESB implementation. Over time, SIBus will evolve so that its programming model will converge with that of WebSphere ESB.
When designing an ESB the requirements should drive the ESB product selection. WebSphere Application Server, with its SIBus function, provides a foundation for building an Enterprise Service Bus, as described in our series of articles. You need to evaluate SIBus and other more advanced ESB products to determine which is best for building your ESB.
In this article series, we covered how to build the core functionality of an Enterprise Service Bus using the WebSphere Message Resources introduced in WebSphere Application Server V6, aka SIBus. In this final installment, we have summarized the scenario, requirements, and features covered in the series and provided a glimpse into the future of IBM's ESB technologies.
The articles in this series:
- Part 1: Introduction to WebSphere V6 messaging resources
- Part 2: Business requirements and the bus
- Part 3: A simple JMS messaging example
- Part 4: Building a better bus with mediations
- Part 5: A simple example of SIBus Web service support
- Part 6: A simple message exchange between SIBus and WebSphere MQ
- Part 7: Switching between message protocols
- Benefits of ESB
- WebSphere Enterprise Service Bus
- WebSphere Message Broker
- SOA programming model for implementing Web services, Part 4: An introduction to the IBM Enterprise Service Bus
- WebSphere Application Server Information Center
- Good introduction to SDO
- Service Component Architecture specification
- WebSphere Application Server V6 Information Center
- Apache Xerces