Planning for the future
Many companies, over time, have integrated multiple systems in a point to point manner. This might be because the business requirements evolved in an ad hoc way over time, or because they acquired packaged applications that needed to communicate with existing homegrown ones, or because there was a corporate merger and things had to be made to work together quickly.
For whatever reason, this results in a situation that can be very time consuming and expensive to maintain. One common approach to resolve this issue is to introduce an Enterprise Service Bus (ESB), which replaces the point to point approach with a single, centralized place to integrate systems, and does so in a service-oriented manner. However, if done incorrectly, this can still create maintenance headaches.
The purpose of this article is to look at a pattern that can “future proof” your integration work, allowing for the upgrading or replacement of legacy systems without an excessive overhead in additional integration work.
Ideally, when an ESB is introduced into an architecture, all of the applications that provide capabilities will provide them as web services. These web services will provide WSDL and XSD artifacts that confirm to corporate standards, and can be deployed as proxy services to the ESB. Clients of the services will use a service registry to find the appropriate service endpoint in the bus, and will call it directly. The bus implementation will use appropriate routing rules to find a version of the service that supports the client. However, in many cases this nice, clean approach to integration is not possible. Some issues that will crop up might include:
- The application only supports some other protocol, such as message queueing, and there is no time or budget to upgrade it to support web services.
- The application is a batch system that can only exchange files, and has no message or transaction based interoperability.
- The application is a package, and has fixed data formats that cannot be modified to match those available to the applications that need to communicate with it. A typical example of this might be an eCommerce front end hosted on IBM® WebSphere® Commerce that needs to integrate with an SAP system that only communicates using iDoc or BAPI.
In order to solve these types of issues, the ESB needs to perform mediations between the service consumer and provider. The types of mediations include protocol conversion, data format conversion, security token conversion, and others. Here, we’ll focus primarily on the issues of protocol and data format, as these are the most common issues that need to be addressed.
There are certain problems that need to be avoided when building the necessary mediations to support this type of integration environment. One obvious mistake is to simulate the point to point environment by implementing a unique service in the bus for every combination of service consumer and provider. While this might seem the least complicated option, it creates future maintenance issues. If the backend service provider is modified, every one of the service endpoints that calls it will need to be modified to support the change.
Another common mistake is to try to do everything in a single mediation flow, which accepts data in all possible incoming formats and protocols, and transforms them to all possible outgoing formats. While this minimizes the number of “moving parts,” it means that a much more complex piece of code needs to be modified and tested for any change in any one of the touch points to the external systems. What might be as simple as adding a new field to an XSD for one endpoint could lead to re-testing a set of complex scenarios that include many applications, along with all the inherent difficulty of setting up such a test.
A better solution to this problem is to define a “pure” canonical service that is an idealized version of how the service would look if all the external applications could be modified to support web services. This is shown in Figure 1.
Figure 1. Description of Figure 1
The central box inside the ESB, labeled “ESB Service,” is intended to represent a standard, reusable SOA service that is hosted in the ESB. This is the canonical service mentioned above. By “hosted,” it should be clear that what is meant is that the ESB provides an endpoint that can be looked up and called by service requestors. It does not mean that all the implementation functionality resides within the ESB. In fact, the service interface provided by the ESB is typically a proxy for some external business logic hosted within an application (custom or packaged). The SOA service is there to standardize access to that business capability, and the purpose of the ESB is to provide a managed environment that enables non-conformant applications to be adapted in such a way that they can interact with the service without needing to be modified, as well as to provide any and all required Quality of Service (QoS) capabilities needed for effective management of the service.
ESB services should always natively communicate using the most standard protocols and syntaxes available. What this means in practice is that they should be defined as web services using WSDL, and that the XSDs that make up their message interfaces should be based on a published internal canonical message format. As already mentioned, in the ideal scenario all clients of the bus would also communicate using the formal published interface. However, this will not be possible in many cases for reasons already discussed. In these situations, adapter services will be used to integrate the requestor or provider with the ESB service. The adapter service will only deal with protocol switches and data transformations; it will not perform any business logic. Any given use case may, therefore, traverse an optional inbound service adapter, an ESB service, and one or more optional outbound service adapters.
In the diagram, three consumers of a given service are shown. One of them is able to use the native web service interface, whereas the other two require some level of mediation. Similarly, three other applications are potential providers of the service (or the service is aggregating from multiple backend systems). Again, two of the three require mediations in data format or protocol..
In order to provide a more flexible run time environment, a service registry such as IBM WebSphere Registry and Repository can be used to support dynamic endpoint lookups. This can help to manage quality of service and SLA requirements. WebSphere Registry and Repository can also be used as a design time tool to support service governance and lifecycle management.
The benefits of this approach are that different functional capabilities are isolated, and therefore the scope of maintenance is limited. If, for example, “Application 5” is upgraded and now directly exposes the canonical web service interface that has already been defined for the ESB service, the outbound adapter service that currently interfaces to it can be retired, and nothing else in the architecture is affected. Eventually, the hope is that all adapters will be removed as more and more service endpoints become conformant to the standard.
- The Enterprise Service Bus, re-examined
- Redbook: Patterns: Implementing an SOA using an Enterprise Service Bus
- Redbook: Patterns: SOA Design Using WebSphere Message Broker and WebSphere ESB
- Understand Enterprise Service Bus scenarios and solutions in Service-Oriented Architecture, Part 1
- Getting started with WebSphere Service Registry and Repository
- IBM developerWorks SOA and web services zone
- IBM developerWorks WebSphere