 | Level: Introductory Rick Robinson (rick_robinson@uk.ibm.com), IT Architect, IBM
23 Jun 2004 In Part 2 of this series on the Enterprise Service Bus (ESB), the author describes and analyzes some commonly observed scenarios in which ESBs and other Service-Oriented Architecture (SOA) solutions are implemented.
The first article in this series described the basic concepts and role of the Enterprise Service Bus (ESB). This article focuses on describing scenarios and issues for ESB deployment to support a Service-Oriented Architecture (SOA). One or more of these scenarios might apply to the SOA and ESB needs of your organization.
ESB scenarios and analysis
The ESB scenarios in SOA section describes the starting points for many SOA and ESB implementations. Each scenario indicates specific issues driving the architecture and design decisions that affect the selection of solution patterns (covered in Part 3 of this series). You can read about these issues in more detail in the Issues driving ESB architecture and design decisions section. These solution patterns represent an evolution from a basic use of service technology, through simple ESB implementations, to a sophisticated SOA infrastructure.
It is not the intention of the ESB scenarios to represent the entirety of an organization?s requirements for SOA or ESB. For example, whereas one of the scenarios, such as Basic integration of two systems, might seem a good match to a particular current requirement, that requirement might evolve over time into something more sophisticated, such as the Enable wider connectivity to one or more applications scenario. Alternatively, there might be many separate requirements for SOA or ESB infrastructure which individually match simple scenarios, but overall represent something more complex.
A balance needs to be struck between implementing a solution that meets the immediately apparent needs, attempting to anticipate future needs, and defining a consistent solution across an Enterprise. It might be appropriate to recognize the organization?s needs as a whole as a relatively sophisticated scenario such as Implement an SOA infrastructure with high quality of service and Web services standards support. An alternative would be to treat individual situations separately as simple scenarios, but define a roadmap for evolving the resulting solutions over time into a common infrastructure.
ESB scenarios in SOA
The scenarios characterized in the sections below are:
Basic integration of two systems
|
Scenario
| |
A business need exists to provide basic integration between two systems implemented in different technologies, such as J2EE, .NET, CICS®, and so forth. The Web services SOAP standard or messaging middleware might be candidate integration technologies. The environments of both applications need to support, to some extent, the integration technology. An important question for this scenario is whether the need to integrate additional systems will likely arise in the future. The use of an extensible solution in the first place might provide support for future requirements; however, the additional work of building such a solution needs to be balanced against the initial need to solve a simple problem.
| | Most relevant issues | Relevant solution patterns (see next article) | | 1,3,4,6,10,13 |
- Implement basic integration using wrappers or adaptors -- see Basic Adaptors.
- Or, with future expansion in mind, either:
- Add a controlling Service Gateway.
- Or implement a sophisticated infrastructure -- such as a Web services-compliant Broker or Enterprise Application Integration Infrastructure for Service-Oriented Architecture (EAI Infrastructure for SOA).
|
Enable wider connectivity to one or more applications
|
Scenario
| |
Existing packaged or custom-developed applications, (for example, Customer Relationship Management (CRM), Enterprise Resource Planning (ERP), and so forth) perhaps implemented in the J2EE platform or other application server environments, provide functions that are useful beyond the applications themselves. Value exists in exposing these functions as services either to enable the applications to interoperate with each other, or to provide access to new channels or clients. The use of interoperable or open standard communication and service protocols seems the best way forward. | | Most relevant issues | Relevant solution patterns (see next article) | | 1,2,3,4,6,8,9,10,11,12,13,14 |
- Implement basic integration using wrappers or adaptors -- see Basic Adaptors.
- Add a controlling Service Gateway.
- If more sophisticated capabilities are required, implement a Web services-compliant Broker or an EAI Infrastructure for SOA.
- If Process Choreography is also required, implement a Service Choreographer or a Full Service-Oriented Architecture Infrastructure (Full SOA Infrastructure).
|
Enable wider connectivity to legacy systems
|
Scenario
| |
An organization has a large investment in legacy technologies (such as CICS, IMS, and so forth) supporting applications that provide their core business transactions and data access. Significant value exists in providing interoperable or open standard, service-based access to those transactions (for example, transactions that query account balance, create orders, schedule or track deliveries, query stock levels, and so forth). | | Most relevant issues | Relevant solution patterns (see next article) | | 1,2,3,4,7,8,9,10,11,13,14 |
- Implement basic integration using wrappers or adaptors -- see Basic Adaptors.
- Or, with future expansion in mind, either:
- Add a controlling Service Gateway.
- Or implement a sophisticated infrastructure with a Web services Compliant-Broker or an EAI Infrastructure for SOA.
|
Enable wider connectivity to an EAI infrastructure
|
Scenario
| |
There is an existing EAI infrastructure, such as IBM® WebSphere® Business Integration, to which extended access based on interoperable protocols or open standards is required. While the exposure of service interfaces defined in terms of XML business data through a highly interoperable protocol such as HTTP or WebSphere MQ might provide an appropriate level of interoperability in some scenarios, support for the WSDL and SOAP Web services standards might be required if neither a custom-developed, nor a proprietary extension to the existing scope of integration, are acceptable.
| | Most relevant issues | Relevant solution patterns (see next article) | | 3,4,5,8,9,11,13,14 |
- Extend the EAI infrastructure using open data formats with an EAI Infrastructure for SOA.
- Add a controlling Service Gateway.
- Or add open standard support to the infrastructure with a Web services-compliant Broker.
|
Implement controlled integration of services or systems between organizations
|
Scenario
| |
An organization wishes to enable its customers, suppliers, or other partners to integrate directly with functions provided by one or more applications, legacy or otherwise. A point of control is required to provide secure, manageable access from external parties to those applications. The use of open standards is preferred as the organization has no direct control over the technologies used by its partners. This scenario might apply either between separate organizations, or between units of a larger distributed organization.
| | Most relevant issues | Relevant solution patterns (see next article) | | 1,2,3,4,6,8,9,10,11,13,14 |
- Add a Service Gateway.
- Or if more sophisticated capabilities are required implement a Web services-compliant Broker.
|
Implement controlled integration of services or systems between organizations
|
Scenario
| |
An organization wishes to enable its customers, suppliers, or other partners to integrate directly with functions provided by one or more applications, legacy or otherwise. A point of control is required to provide secure, manageable access from external parties to those applications. The use of open standards is preferred as the organization has no direct control over the technologies used by its partners. This scenario might apply either between separate organizations, or between units of a larger distributed organization.
| | Most relevant issues | Relevant solution patterns (see next article) | | 1,2,3,4,6,8,9,10,11,13,14 |
- Add a Service Gateway.
- Or if more sophisticated capabilities are required implement a Web services-compliant Broker.
|
Automate processes by choreographing services
|
Scenario (Note: this scenario can be considered an evolution of the Enable wider connectivity to one or more applications scenario. It is not considered an ESB scenario, as service choreography is usually implemented separately to the ESB, as described in the first article in this series. However, I include it here as it is a scenario that often drives requirements for an ESB as well as for a service choreography component.)
| |
Existing packaged (for example, CRM, ERP, and so forth) or custom-developed applications, perhaps implemented in the J2EE platform or other application server environments, provide functions that are useful beyond the applications themselves. These functions can be exposed as services using interoperable or open standard communication and service protocols so that the applications can interact. The interactions combine at some level to form business processes. These processes should be explicitly modeled and executed using appropriate modeling and process execution technology, possibly in compliance with appropriate open standards. | | Most relevant issues | Relevant solution patterns (see next article) | | 1,2,3,4,6,10,11,12,13,14 |
- If the direct connection of services is possible, implement a Service Choreographer.
- If more sophisticated integration or control is required, implement a Full SOA Infrastructure.
|
Implement an SOA infrastructure with high quality of service and Web services standards support
|
Scenario
| |
This scenario is a composite of the preceding scenarios. It represents the need to enable widespread internal or external access to services provided by multiple applications, legacy or otherwise. Various security, aggregation, transformation, routing, and service choreography capabilities are required. An IT organization often drives this scenario in response to increasing demands from across the business it supports to allow more widespread and flexible integration between business systems. | | Most relevant issues | Relevant solution patterns (see next article) | | All |
- Implement a Full SOA Infrastructure.
|
Issues driving ESB architecture and design decisions
In order to identify a suitable solution pattern and implementation technology for an ESB, requirements for specific ESB capabilities will need to be analyzed in detail. The intention of the following questions is to aid this process, and the preceding section indicates the specific questions relevant to each scenario.
- Are the existing functions and their data interfaces good matches to the services you want to provide, or can you modify or aggregate the applications?
- o If not, transformation or aggregation capability will be required either in adaptors or the ESB infrastructure, or will have to be performed by service clients.
- Should the services be exposed in the form of some common business data model? If so, do the systems implementing those services already support that model, or can they be made to do so?
- If not, transformation or aggregation capability will be required either in adaptors or the ESB infrastructure.
- Are open standards required, or can appropriate interoperability be achieved through EAI middleware? If open standards are required, which ones are appropriate?
- While the use of open standards is one way to achieve interoperability, proprietary EAI middleware is also highly interoperable and often significantly more mature. Many organizations also have extensive existing infrastructures that can, in some scenarios, minimize the benefits of open standards.
- In scenarios where open standards are required, Web services are perhaps the most obvious choice in this context. However, you can also apply Java Messaging Service (JMS), JDBC, basic XML, or several other technologies, such as EDI or industry XML formats.
- In practice, you cannot always assume interoperability between different implementations of the same standards, particularly if the standards are recent or emerging. In the case of Web services, the Web Services Interoperability Organization has published the Basic Profile for interoperability using SOAP and WSDL, and other profiles for more advanced standards will follow (for example, WS-Security, WS-Transaction, and so forth). Until such profiles are comprehensive, established, and widely supported by products, the use of open standards will not guarantee and might not always facilitate interoperability.
- Is support for basic communication protocols and standards (for example, WebSphere MQ, SOAP, WSDL) required, or are more advanced capabilities (for instance, WS-Security, WS-Transaction, and so forth) needed?
- Requirements to support more sophisticated standards will impose more stringent constraints on the options for implementation technologies, and might imply the use of less mature technologies.
- When considering changes to the message formats and protocols used by an existing infrastructure, including the possibility of adoption of open standards, are these changes required throughout the existing infrastructure, or can they be applied at the edges? If EAI technology is in use or under consideration, does that technology have its own internal format, or can it process open standards as an internal format?
- Any use of open standards is likely to be driven by the need to extend access. As such, it is more important that they are available at the interfaces to existing infrastructure than to be used internally.
- If internal use of specific formats, technologies, or standards is required, this will place constraints on the choice of implementation technology.
- Do the systems implementing functions that will be exposed as services, support the required technologies or open standards, such as SOAP, JMS, or XML?
- If not, either the ESB infrastructure or adaptors will need the capability to transform between the required open standards and the formats supported by the service providers.
- Where access to legacy systems is required, and with the use of more recent XML-based technologies (including SOAP, but also basic XML with EAI middleware), is direct support (for example, CICS SOAP support) available for that legacy system, or are separate adaptors required? Does the legacy platform support XML processing, and, if so, is this type of processing a sensible use of the platform capabilities?
- If, for any of these reasons, a required SOAP or XML capability will not be made available on a legacy platform, appropriate transformation capability will be required either in adaptors (such as J2C Connector Architecture (JCA) or WebSphere Business Integration Adaptors), in an integration tier, or in the ESB infrastructure.
- If an EAI technology is already available, does it implement services as message flows with appropriate function and interface granularity? What connectivity protocols does it support (for example, JCA, SOAP, WebSphere MQ, and Java Remote Method Invocation)?
- If existing message flows do not provide the required services, then additional flows will be needed to perform transformations. If the EAI technology does not directly support the required standards, a Service Gateway component can be added.
- What measure of protection should be afforded to the service provider systems from service client channels in the form of workload buffering, security, logging, and so forth?
- Such buffering will often be a role of the ESB infrastructure, and define some of the capabilities it requires. If specific service provider systems (for example, legacy transactional systems) have additional needs for protection, a dedicated integration tier could be used.
- How many services will be enabled? What aspects of enablement should be consistent across the services, and how can consistency be enforced, perhaps across multiple platforms and applications?
- If very few services are involved, a simple point-to-point integration model might be appropriate. However, if more are involved or likely to become so over time, the addition of a control point, such as that provided by an ESB, becomes increasingly beneficial.
- Are the service interactions contained within the organization, or are some interactions external?
- If external access is required, a Service Gateway component can be used to provide additional control. This is often the case in addition to an ESB infrastructure implementation within a single organization, as the requirements for security and service routing might differ for services made available externally.
- Are there requirements for service choreography, and do they involve short-lived or long-lived (in other words, stateful) processes, or both? Do they include manual activities?
- Where these requirements constitute business function, the choreography should be implemented in a Service Choreographer component separate from the ESB. Requirements to support long-lived stateful processes or manual activities will place constraints on the choice of implementation technology.
- What service level requirements should the infrastructure support (for example, service response time, throughput, availability, and so forth), and how is it required to scale over time?
- Some of the candidate technologies for ESB implementation are relatively new and might only have been tested against limited service levels. Similarly, because the relevant open standards are either recent or emerging, support for them in more established products and technologies is also new.
- For the foreseeable future, critical architectural decisions will be concerned with balancing the benefits of specific open standards, supported by emerging or mature product technologies against service level requirements. These point-in-time decisions will need to recognize that some standards, and product support for them, are relatively mature (for example, XML, SOAP, and so forth), others (for example, WS-Security) are newer, while others (for example, WS-Transaction) are still emerging.
- The trade-off between the benefits of standards and proven service level characteristics will often drive a mixed approach combining standards-compliant, and proprietary, or custom technologies in an ESB and SOA architecture.
- Is a point-to-point or end-to-end security model required (for example, should the ESB simply authorize service requests, or should it pass the requestor identity or other credentials through to the service provider)? Is there a need to integrate the service security model with application or legacy security systems?
- If point-to-point security is acceptable, a number of existing solutions (for example, SSL, J2EE security for database access, adaptor security models, and so forth) can be applied. If end-to-end security is required, the WS-Security standard is a possibility, providing all the systems involved support it. Alternatively, you could use a custom model with custom message headers or pass security information as application data.
 |
Summary
In this article, I have established some of the most common scenarios for ESB implementation and the issues imposing direct forces on corresponding solutions. While the issues covered here might not be all-inclusive, they are amongst those most commonly encountered.
The common scenarios outlined range from the basic integration of two systems, to implementing an SOA infrastructure with high quality-of-service, and Web services standard support. The fourteen different issues described take into account:
- Existing data interfaces
- Business data models
- The use of open standards
- Support for basic or advanced communications protocols
- Changes to messaging formats from existing systems
- Exposing existing services through new technologies
- Access to legacy systems
- Existing EAI technologies
- The measures of protection needed
- How many services to enable and the level of consistency needed
- Interacting intra-company and with other companies
- The need for service choreography
- Infrastructure-level support for service level requirements
- The use of point-to-point or end-to-end security models
Understanding these basic scenarios and issues gives a strong basis for the potential solutions that you can develop. In Part 3 of this series, I will discuss the actual solutions as mentioned in this article, namely:
- Basic Adaptors
- Service Gateway
- Web services-compliant Broker
- Service Choreographer
- EAI Infrastructure for an SOA
- Full SOA Infrastructure
Finally, I will discuss some options available to organizations considering how to evolve their infrastructure in a controlled and incremental fashion. I will also explain the issues that can guide you in developing your own ESB roadmap.
Acknowledgements
This article would not exist without the discussions the author has been involved in with the following people, all of whom contributed to the ideas and thinking behind it:
Beth Hutchison, Rachel Reinitz, Olaf Zimmerman, Helen Wylie, Kyle Brown, Mark Colan, Jonathan Adams, Paul Fremantle, Keith Jones, Paul Verschueren, Daniel Sturman, Scott Cosby, Dave Clarke, Ben Mann, Louisa Gillies, Eric Herness, Bill Hassell, Guru Vasudeva, Kareem Yusuf, Ken Wilson, Mark Endrei, Norbert Bieberstein, Chris Nott, Alan Hopkins, and Yaroslav Dunchych.
Resources - For more details on Enterprise Service Bus, see "Understand Enterprise Service Bus scenarios and solutions in service-oriented architecture" Part 1 and Part 3 (developerWorks, 2004).
-
To learn how Web Services technologies provide the compelling foundation for implementing Web Service-oriented architectures, read Web Service-Oriented Architecture - The Best Solution to Business Integration by Annrai OToole.
- Check out the article SOA - Save Our Assets by Lawrence Wilkes at CBDI Forum (subscription required).
-
Patterns: Service-oriented Architecture and Web Services Redbook, SG24-6303-00, April 2004, Endrei M., et al.
- Read about "Migrating to a service-oriented architecture, Part 1" (developerWorks, December 2003) and "Migrating to a service-oriented architecture, Part 2" (developerWorks, December 2003).
- Read the specification Web Services Description Language (WSDL) 1.1.
- Check out the Business Process Execution Language for Web Services Version 1.1
specification .
- Visit LooselyCoupled.com for a list of Enterprise Service Bus links.
- The original Gartner article defining the Enterprise Service Bus requires a subscription, but can be found by searching their site, and is entitled Predicts 2003: Enterprise Service Buses Emerge, published, 9th December 2002, by Roy W. Schulte.
- Examine the IBM Patterns for e-business website.
- Check out the WS-Transaction specification.
- For more information on Web Sevices Gateway, read "An introduction to Web Services Gateway" (developerWorks, May 2002).
- The material used in this article is from the forthcoming Redbook, Patterns: Implementing an SOA using an Enterprise Service Bus, SG24-6346-00, Draft, Martin Keen, et al.
- Using Web Services for Business Integration Redbook, SG24-6583-00, April 2004, Geert Van de Putte, et al.
- WebSphere Version 5.1 Application Developer 5.1.1 Web Services Handbook Redbook, SG246891-01, February 2004, Ueli Wahli, et al.
- See the WS-Security specification.
- Read the specification WS-Policy.
- Find additional SOA and Web services technology resources on the developerWorks SOA and Web services technology zone.
- Browse for books on these and other technical topics.
About the author | | | Rick Robinson works as an IT Architect for IBM, having joined the company in March 1997 as a developer. He is a member of the Architecture Services group in EMEA WebSphere Lab Services, having focused on the WebSphere software platform since its origins in 1999. Rick has a technology, rather than industry, focus, but has spent much of the last three years working with companies in the Financial sector. Rick is an accredited member of the IT Architect Profession within IBM. Prior to joining IBM, Rick studied for a Ph.D. in Physics. |
Rate this page
| |
