Building an Enterprise Service Bus using WebSphere ESB: Part 6: Exploring advanced WebSphere ESB features

The conclusion to this series about building an Enterprise Service Bus with IBM® WebSphere® ESB recaps all the articles in the series, and provides an overview of more advanced scenarios that you can investigate next. This content is part of the IBM WebSphere Developer Technical Journal.

Share:

Rachel Reinitz (rreinitz@us.ibm.com), Distinguished Engineer, IBM

Rachel ReinitzRachel Reinitz is an IBM Distinguished Engineer with IBM Software Services for WebSphere and a member of the IBM Academy of Technology. She has built much of IBM’s internal and external intellectual capital and education on SOA, Enterprise Service Bus, and Web services. Rachel focuses on how to get started with SOA and ESB best practices. She consults with clients and ISVs on how SOA and ESB can be used to achieve their business and technical objectives. She is a frequent conference presenter and written many developerWorks articles on ESB and Web services. Rachel lives in the Bay Area in California, and enjoys hiking, socializing, and international travel.


developerWorks Professional author
        level

Andre Tost, Senior Technical Staff Member, IBM

Andre TostAndre Tost works as a Senior Technical Staff Member in the Software Group's Enterprise Integration Solutions organization, where he helps IBM's customers establishing Service-Oriented Architectures. His special focus is on Web services technology. Before his current assignment, he spent ten years in various partner enablement, development and architecture roles in IBM software development, most recently for the WebSphere Business Development group. Originally from Germany, he now lives and works in Rochester, Minnesota. In his spare time, he likes to spend time with his family and play and watch soccer whenever possible.



18 July 2007

Also available in Chinese

Introduction

The first five installments of this series covered several basic scenarios that showed how to take advantage of an Enterprise Service Bus (ESB) in a real life business environment, with a set of examples illustrating how those scenarios could be implemented using the IBM WebSphere ESB product. In this final article, we will summarize what we have accomplished with our fictitious Posts-R-Us company, and then point out areas in the ESB space that you might want to focus on next. Some of these areas are already covered in other articles, others will be addressed by articles yet to come, and all of them might influence where the product will go in the future.


Let's recap

The examples and scenarios we described in these articles were all based on the fictional Posts-R-Us company, which has a heterogeneous set of existing systems and solutions for the various parts of its business, and was looking to improve the accessibility, connectivity, and integration of these systems. The company decided to use the architectural pattern of an Enterprise Service Bus to achieve these goals, and thus take the first steps toward establishing a Service Oriented Architecture (SOA).

While Posts-R-Us is not a real company, we do see many cases in our real life projects that are quite similar. Existing business functions and business processes are increasingly represented by a service layer that is realized through an ESB so that the investment in these solutions can be preserved. At the very minimum, the ESB must offer support for a set of network protocols and be able to convert messages as they flow through the system. These are exactly the types of scenarios that we have included in our articles, and we have shown how they can easily be implemented by using WebSphere ESB.

Part 1 described the basic functionality of the ESB and positioned the functionality of the new WebSphere ESB product against its predecessor, the SIBus feature in IBM WebSphere Application Server. WebSphere ESB's additional features include better tooling (WebSphere Integration Developer) and a new service-oriented programming model (Service Component Architecture, or SCA).

Part 2 presented a basic, but powerful, scenario: inserting the ESB between a JMS-based service requester and a JMS-based service provider, represented by a message-driven bean (MDB). This decoupling of the requester and the provider creates a degree of independence between the two, from both a message format perspective and from a network protocol perspective. In other words, the message can be changed as it flows through the ESB, transparent to both requester and provider. Moreover, even though both requester and provider use JMS as the transport protocol in this case, one of them could switch to a different protocol without affecting the other. The interaction we built in this example was "one-way," meaning the request message was sent asynchronously without any response.

Part 3 introduced WebSphere ESB's support for Web services. Mediations in WebSphere ESB are service components following the SCA standard, and thus can be accessed as Web services. In turn, a mediation component can invoke external service providers through a Web services interface. As in Part 2, the service requester and the service provider used the same protocol, namely SOAP/HTTP. The ESB merely served as a level of indirection between the two to decouple them. Part 3 also showed how the endpoint address of the service provider could be changed at run time using the WebSphere ESB administrative console. However, with the advent of the IBM WebSphere Services Registry and Repository product, much more sophisticated and robust ways of performing dynamic routing became available. (We'll come back to this subject later.)

Finally, Part 3 showed a very important feature of WebSphere ESB that also enables making changes to how a mediation behaves at run time: promoted properties. Each mediation primitive within a mediation flow component has a set of properties that can be "promoted" so that they can be changed at run time, via the admin console, and thus add dynamic behavior to the mediation. The interaction in this case was synchronous; the service requester sends a request message and blocks all messages until the appropriate response message is received.

After using JMS and SOAP/HTTP as the underlying protocols in the first couple of sample scenarios, Part 4 took advantage of the native WebSphere ESB support for WebSphere MQ, which was introduced with Version 6.0.2. The example in the article highlighted a really powerful feature of SCA: the ability to add additional protocol support to an existing mediation component without affecting the component itself, and without affecting any already existing requesters or providers. The JMS example from Part 2 was reused in Part 4 and simply added a new service provider. The request message was now not only sent via JMS to the JMS-based service provider, it was also forwarded via WebSphere MQ to a queue controlled by a remote MQ queue manager. The service requester and the mediation flow component itself did not change at all, hence no programming was involved.

Part 5 investigated a scenario in which we switched between protocols supported by the service requester and the service provider. Instead of adding a new service provider to the scenario, which we had done in Part 4, we added an additional service requester. In fact, we did it twice: we reused both the JMS scenario and the SOAP/HTTP scenario from previous installments, thus creating a complete mix of protocols and interaction styles for requesters and providers. For the asynchronous JMS example, we added a (synchronous) SOAP/HTTP service requester. For the (synchronous) SOAP/HTTP example, we added an (asynchronous) MQ-based service requester. And again, all of this was done without ever touching the mediation flow component itself, and without changing any of the existing code. Instead, we used the import and export support offered by the SCA, which can be developed through the assembly editor in WebSphere Integration Developer.


Where to go next

The scenarios covered in these articles primarily showed WebSphere ESB support for different protocols and exchange patterns. They covered how to set up a fairly basic mediation flow component that would process messages, and then focused on how to connect various types of service requesters and providers to it. Again, this is what we see many of our customers start with: connecting applications that use different protocols, and replacing existing point-to-point connections with an ESB that decouples them.

Let's now take a look at how you might extend your ESB using additional WebSphere ESB features. One obvious area you might explore further is the processing capabilities within the mediation flow component itself. While we simply logged messages that came through the ESB in our examples, there is much more that can be done.

Just a sampling
The examples in this section barely touch the surface of what additional requirements might be satisfied with an ESB. We leave it up to your creativity to come up with much more complex problems to solve!

Dynamic lookup and routing

One of the core functions of an ESB is to support the concept of "virtual services;" that is, the ESB appears to be the real service provider to the requester, but in reality it only supports a virtual interface and routes messages to the actual service provider. In our example scenarios, the location of this real provider was basically hardcoded into the mediation by associating the SCA import with a WSDL definition containing a specific endpoint URL.

A much more elegant and robust solution is to store the location of a service provider in an external registry, where the ESB can look it up at run time. This way, endpoint information (among other things, of course) can be managed centrally, as part of an overall SOA governance model. The WebSphere Services Registry and Repository product offers support for this, and, in fact, there is a developerWorks article that describes in detail how to integrate WebSphere ESB and WebSphere Services Registry and Repository.

Content-based routing

A special case of dynamic routing is content-based routing. The difference is that the lookup happens based on criteria that can be found inside the message itself. For example, assume that a mediation in WebSphere ESB supports multiple different service providers. Moreover, its interface is so generic that it can be used by multiple different service requesters. In this case, the mediation would parse the incoming message and perform a lookup of the right service provider based on differentiating content of the message. In WebSphere ESB, this would most likely be accomplished with a custom mediation primitive that contains the logic to extract the message differentiator and feed it into the lookup primitive that determines the right provider. A developerWorks article about content-based routing shows a complete example. (By the way, this article is the first of a series of articles about advanced WebSphere ESB topics, so watch for future articles in this series.)

A further, even more elegant extension to this scenario is offered by the dynamic assembler component that is included in the IBM WebSphere Business Services Fabric product. The dynamic assembler, which is implemented as an independent SCA component, can select service endpoints based on the content and the context of the message. The context information is in the message header and can include things like the channel that a request was sent over, or the identity of the requester, and so on. See Resources for an article that describes a complete sample scenario.

Message aggregation and distribution

In all the scenarios we introduced, one service requester was invoking one service provider. There are cases, though, where there is a one-to-many relationship between the two. For example, a requester might send a request message, resulting in multiple request messages being sent to multiple service providers. The response messages coming back from these providers then have to be aggregated back into one consolidated response message that goes back to the requester. WebSphere ESB currently does not offer out-of-the-box support for those types of mediations, so you would need to leverage custom mediation primitives to do this. An approach that requires no programming uses JMS as the protocol for multiple service providers, and one-way interaction. In this case, the message can be sent to a JMS topic by the mediation, from which it can be picked up by multiple subscribers. By the way, utilizing publish/subscribe capabilities in the context of an ESB probably deserves an article all by itself.

Message transformation

The messages in our examples were all XML-based. Not much (if any) transformation was required to adapt a specific requester message format to a specific provider message format. In real life, however, this will be common requirement. The simplest case of transformation is when one XML format must be transformed into another XML format. WebSphere ESB offers the XSLT mediation primitive that simply executes an XSLT transformation on a message flowing through the mediation. The XSLT transformation can be built using tooling available in WebSphere Integration Developer, or any other XSLT authoring environment.

Some complex transformations cannot easily be done in XSLT. As soon as you have things like extremely large messages, recursive and repeating structures, the need to callout into Java™ (or other logic) during a transformation, and so on, a more sophisticated transformation engine is required. WebSphere Transformation Extender provides support for very complex transformations and offers an API that can be invoked from within a WebSphere ESB mediation flow component. Look for improved support for this type of integration in future product releases.

Data binding

When talking about message transformation, you also need to look at how data is sent into a mediation flow component to begin with. As you probably know by now, each message is represented as a Service Message Object (SMO) within the mediation flow component. This enables primitives (custom or pre-built) to access the message in a uniform way, regardless of the protocol that was used to send it. For messages that come into the mediation via SOAP/HTTP, this is done automatically, since the runtime knows the structure of the message, and can thus map it into an SMO. For JMS or WebSphere MQ, this is not necessarily the case. The message can contain an XML document that maps to a BO, it can contain a serialized Java object, or it can contain just plain text, to name a few examples. WebSphere ESB has built-in support for the most common data formats, and support is available to write your own custom data binding. You can see an example for this in an earlier developerWorks article series. (The example shown in the referenced article was built using WebSphere ESB Version 6.0.1. Version 6.0.2 supports the shown message formats out-of-the-box.)

Other protocols and formats are usually supported by utilizing adapters, preferably one of the WebSphere Adapter products. For example, reading data from a flat file into a mediation module is made possible via the IBM WebSphere Adapter for Flat Files. The adapter is imported and connected to the mediation module using the Enterprise Discovery Service tooling in WebSphere Integration Developer. A developerWorks tutorial takes you through a complete and detailed example.

When to use a mediation

The situations described above all implicitly assume that building a WebSphere ESB mediation is the right way to solve the problem. However, that is not always the case. One example for when it is better to step outside of the ESB altogether is: business logic. The ESB focuses on connectivity and integration issues that are usually independent of the business problem that the involved services try to solve. Hence, if certain logic touches on the business specific aspects of a solution, it should typically be located elsewhere. In cases where the implementation of a service interface consists of an orchestration of other services, this service should be implemented as a business process using the BPEL language. A BPEL-based business process can be executed in WebSphere Process Server, which includes WebSphere ESB, and since any service orchestration also has connectivity aspects, both products will often run together. Service interactions that are modeled in BPEL are often redirected via the ESB to support decoupling and virtualization of those services. For a thorough analysis of what runs in the ESB -- and what doesn't! -- see Greg Flurry's great article on developerWorks.


Conclusion

In this final article, we tried to show that this series of articles about building an Enterprise Service Bus is not the entire answer; it merely provides a basic introduction to a more complex and interesting technology area. In this series, we covered basic connectivity between service requesters and service providers across a selection of commonly used protocols and message formats. The examples shown are all meant to be easily reproducible and should serve as a good way of familiarizing yourself with the WebSphere ESB product and its tooling.

Resources

Learn

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Business process management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Business process management, WebSphere, SOA and web services
ArticleID=240360
ArticleTitle=Building an Enterprise Service Bus using WebSphere ESB: Part 6: Exploring advanced WebSphere ESB features
publish-date=07182007