Architecture in practice, Part 5: SOA Scenario 2: Service connectivity options

Document options Document options requiring JavaScript are not displayed Rate this page Help us improve this content

Level: Advanced

Tilak Mitra (tmitra@us.ibm.com), Executive IT Architect, IBM　

18 Dec 2007

View SOA within a software development lifecycle context using the IBM® Service-Oriented Architecture (SOA) Foundation Lifecycle. This installment in the Architecture in practice series focuses on the Service Connectivity scenario, the second of the SOA scenarios. Explore four ways to realize connectivity between service producers and consumers that foster reuse of services across multiple delivery channels. Learn about three Enterprise Service Bus (ESB) topologies that enable service connectivity, and associated products that provide advanced, end-to-end ESB-based solutions. Get an overview on developing mediation modules and flows using four common patterns.

Introduction

The first installment of the Architecture in practice series, Realizing Service-Oriented Architecture, discusses IBM's Service-Oriented Architecture (SOA) Foundation Lifecycle (or SOA Lifecycle) and how it allows IBM customers to think of SOA in terms of a software development lifecycle. The four phases of the IBM SOA Lifecycle — Model, Assemble, Deploy, and Manage — are covered in detail.

This Architecture in practice series covers a wide range of topics. For more information about the IBM SOA scenarios, read Part 2, An introduction to SOA solution scenarios, which introduces the eight most common scenarios in a typical SOA-based IT project. It explains the philosophy behind them, the broad context in which they can be used, and a methodology for applying a particular SOA scenario to service-enable typical legacy projects. Part 3, Top ten tips for writing great IT project proposals, highlights the major steps you should follow to ensure the development of a high-quality proposal. Part 4, Service creation options in a real-world SOA scenario, focuses on the Service Creation scenario. It explores three main sources for services in SOA, and the architectural patterns that provide guidance for properly using the services.

This article focuses on the second of the eight SOA solution scenarios: the Service Connectivity scenario. Part 4 discussed techniques for creating services, and the motivation for reusing existing functions and exposing them as services. The Service Connectivity scenario takes this idea one step further, and builds upon the pivotal theme that it's better to reuse assets than it is to build them from scratch. In this article, learn about the various mechanisms that services both inside and outside the enterprise perimeter that can be connected to realize a service-oriented ecosystem. Learn about four different realizations of SOA connectivity, and how their implementations can be mapped to the phases of the SOA lifecycle.

For each example, IBM recommends tools and products that support its implementation. The main architectural construct that enables service connectivity is an Enterprise Service Bus (ESB). This article also examines some of the most commonly encountered topologies of an ESB, and how they may be used to define an end-to-end service connectivity solution.

Though many tools and products are mentioned, it's outside the scope of this article to explain all of them. See the Resources to further explore topics of interest.

Back to top

Addressing the business challenge

To become a service-oriented business and a participant in an SOA ecosystem, today's enterprise must figure out how to integrate existing and new services into a well-orchestrated business process. The heterogeneity of multiple platforms, multiple transmission protocols, and diverse data formats in the modern IT environment make it imperative to address the seamless flow of information from anywhere at any time using anything.

The ability to build business processes that transcend traditional enterprise boundaries, and to build trusted relationships with business partners, is becoming a mainstream requirement. The challenge is to:

• Homogenize the connectivity of assets and services to support the business processes, and
• Simplify the connectivity between services and the SOA infrastructures to build a seamlessly connected platform that fosters a secure, reliable, and scalable flow of information in a federated environment.

Back to top

The Service Connectivity scenario

The Service Connectivity scenario demonstrates the connectivity between service producers and consumers, fostering reuse of new and existing services across multiple delivery channels. For this article, we assume that the SOA solution scenarios were evaluated in the context of the business problem, and the Service Connectivity scenario was chosen for implementation. Briefly, the decision-making process relied on:

• Part 2 of this series, which provides a process for using the SOA scenarios. A critical part of the process is the criteria for selection. In Table 1 of Part 2, there are four generic use cases that fall under the Service Connectivity scenario. The first step is to map the functional requirements of a given problem domain to one of these four generic use cases.
• Realizations that were developed, which show how the scenario can be realized using the selected products. Realizations combine a customer scenario along with its solutions. Typical realizations have been provided that help customers jumpstart their SOA implementation.

The next section introduces the realizations of the generic use cases.

Back to top

The four common realizations

At the heart of the Service Connectivity scenario are the concept, application, and implementation of the ESB. The features of the ESB are mapped against the requirements of a given problem, which in turn yields one or more ESB products that may be required to implement the solution. The four common realizations are:

• Secure connectivity to external third parties and trading partners
• Connect internal business systems based on open standards
• Deliver existing business processes through new business channels
• Adapt from third-party systems to Web services

Secure connectivity to external third parties and trading partners

The requirement to extend the enterprise business process outside the enterprise boundaries, by integrating services provided by third-party vendors or business partners, warrants a common solution that can be used by any client. An ESB gateway is usually recommended, to be used either standalone or in conjunction with an intra-enterprise ESB to provide secure service interaction between internal and external domain boundaries.

An ESB gateway can be used standalone when the sole requirement is secure access to resources. An ESB gateway can play a dual role by also providing some basic mediation and routing capabilities, based on the middleware and integration requirements. In such situations, the recommended product to use is the IBM Datapower XML Security Gateway XS40 (henceforth called the Datapower appliance). The Datapower appliance is a hardware appliance designed to provide: XML transformation, a secure access channel to invoke enterprise services, and basic mediation flow capabilities. They are chiefly designed to be message-aware network proxies for Web services traffic. The devices have extremely high performance in processing XML messages. They are also a message control point in an application's security architecture.

Usage of a gateway ESB for this generic connectivity use case should also follow the SOA lifecycle. The phases of a lifecycle are modular in nature; the execution of each phase provides its own benefits and is not dependent on the execution of the other phases to realize those benefits. Figure 1 maps the high-level implementation steps of this use case to the SOA lifecycle.

Table 1 explains the activities for this use case during each phase of the SOA lifecycle.

Connect internal business systems based on open standards

Sometimes an enterprise has a business and IT prerogative to use standards-based technology for connectivity between various services that orchestrate business processes with a WebSphere® Application Server-based middleware infrastructure. When the enterprise wants to reach out to multiple clients that support a multitude of standards-based service invocation mechanisms, the recommended ESB is the IBM WebSphere ESB.

WebSphere ESB provides a full-featured departmental solution for standards-based ESB implementations. It supports publish/subscribe and one to many flows, is fully transactional and exactly once delivery of critical business functions, and arbitrary integration logic wherever necessary. The mediation capabilities assist in transparent support for message formats and transport protocols required to support the service provider.

Figure 2 maps the high-level implementation steps to the SOA lifecycle.

In the Service Connectivity scenario, the focus is mainly on the assembly and deployment of services and business processes, which may be followed by the standard mode of practice for their management and monitoring.

Table 2 explains the activities for this use case during each phase of the SOA lifecycle.

Deliver existing business processes through new business channels

An essential tenet of an SOA ecosystem is to foster a seamless and transparent connectivity between service consumers and service providers. The heterogeneity of a typical business and IT environment poses a distinct set of challenges for integrating consumers and providers of services. The message format sent by the service consumer:

• Might not be standardized
• May vary between one consumer and another
• May be too coarse grained and need decomposing

Message decomposition, routing, and mediation are key requirements.

Even if message formats are resolved, and routing based on message information is implemented, protocols used by service consumers to initiate a service request may be different from the protocol expected by the provider. For example, a service consumer may initiate an HTTP-based service request, while the actual service may expect a JMS-based request invocation.

Security around service invocations is also a challenging endeavor. A service may require proper security context when invoked from outside the enterprise perimeter, but might not require any security when invoked by applications and systems inside the corporate firewall.

A service requester may request a business function that might be supported by one or more services that are composed together to realize the function. Such compositions need to be designed, implemented, and deployed on a middleware layer to facilitate seamless connectivity.

An enterprise with core business functions implemented in legacy systems and applications may need to unlock its business potential and open up. Both external customers and the enterprise can benefit from reuse across multiple applications in the IT portfolio. The ability to connect to standard and non-standard back-end systems through specific technology adapters becomes an imperative requirement.

An advanced ESB that performs, at a minimum, the following functions would be an ideal choice in this scenario:

• Conversion between communication protocols
• Conversion of the heterogeneous data formats into a canonical message model, with mediation capabilities
• Routing of service request to the proper service provider
• Composition of services to fulfill a service request
• Transaction management
• Messaging backbone

IBM WebSphere Message Broker (Message Broker) is suitable where advanced ESB functions are required. Message Broker is an option when Web services support is not critical, and quality-of-service requirements demand the use of mature middleware. Message Broker can support all the ESB capabilities above, and is not limited to interoperability based on open standards.

Table 3 explains the activities for this use case during each phase of the SOA lifecycle.

Adapt from third-party systems to Web services

Adherence to open standards, such as SOAP/HTTP and JMS, is increasingly becoming a corporate mandate, yet the majority of enterprises have a medium-to-heavy IT portfolio based on legacy applications and systems. A fundamental principle of SOA is to unlock the business value embedded in old, existing, and third-party-based applications, systems, and technologies by adapting them to open standards based on Web service technology.

Reusing key functions in existing systems and applications, in a mechanism that realizes multiple business processes and reaches different business channels, is a fundamental value proposition of SOA. You can reduce costs of new application development, and costs throughout the development lifecycle, by reusing existing functions adapted by modern Web services technology. Adapting third-party and legacy systems using Web services technology is a relevant and important option during any application modernization initiative.

You can use an ESB to provide the connectivity to enterprise information systems (EIS) and legacy systems using the adapter framework. There are two main categories of adapters to consider:

• The WebSphere Business Integration (WBI) adapters allow heterogeneous business applications to exchange data between ERP, HR, CRM, and supply chain systems through the coordinated transfer of information using business objects.
• The WebSphere JCA adapter provides a mechanism to store and retrieve enterprise data in J2EE™. It supports enterprise connectivity to JDBC, flat files, and systems such as SAP, PeopleSoft, or Siebel.

Usage of the WBI and WebSphere JCA adapters assumes:

• The customer has, or is considering having, a WebSphere Application Server infrastructure.
• There are no WS-Security or other complex security considerations.
• There aren't many data synchronization requirements between the client and the EIS systems.
• The volume of requests is moderate.

Whether to use WBI adapters or WebSphere JCA adapters is an architectural decision that needs to be specific to customer requirements and the existing third-party systems. Some of the WebSphere adapters include:

• IBM WebSphere Adapter For Flat Files, Version 6.0
• IBM WebSphere Adapter for JDBC, Version 6.0
• IBM WebSphere Adapter for PeopleSoft Enterprise, Version 6.0
• IBM WebSphere Adapter for Siebel Business Applications, Version 6.0
• IBM WebSphere Adapter for SAP Applications, Version 6.0

Table 4 explains the activities for adapting from third-party systems to Web Services in relation to the SOA lifecycle phases.

Back to top

Overview of mediations

Mediations, in the context of connectivity between services in an SOA, refers to:

• Ways in which the incoming service request message is transformed, routed, and augmented before it reaches the service provider.
• How the same techniques are applied to the response from the provider before it is finally sent back to the originating requestor.

WebSphere ESB is a run time environment capable of running message mediations. The technology used to define the message mediations is based on Service Component Architecture (SCA). A mediation module is a basic unit of deployment for message mediation. It contains mediation flow components that operate on the message requests and responses. In the final packaging of a mediation module, there is at least one export (defining the interface through which service consumers invoke the mediation) and one import (defining the connectivity to a service provider). The mediation flows and the modules are all designed and packaged using the mediation flow editor in WebSphere Integration Developer.

Mediations are developed following four commonly used patterns:

Message protocol transformation

The mediation module converts any standards-based protocol (HTTPS, HTTP, SOAP/JMS) used by the service requestor to any of the other standards-based protocol formats, thereby letting the service provider support any standards-based protocol used by service consumers.

Content-based message routing

The mediation module encapsulates the logic and intelligence to dynamically choose one of multiple service providers to fulfill the service request. The logic is primarily based on filtering rules that may be applied on the message content, message header, service requester, and so on to decide the best service provider at run time. This is a form of service virtualization that keeps the requestor completely ignorant of which provider is actually servicing the requests.

A Transfer Funds service is an example of this form of virtualization based on business rules and non-functional requirements. A premium customer might have funds readily available, but typical customers require a two-day period before the funds are available.

Message format transformation

This method comes in particularly handy when the service consumer uses a message structure that is not compliant with the structure the service provider expects for its incoming request messages. In such cases, the mediation module mediates the incoming request, detects the differences, and translates the request message format into the response message format before forwarding it on to the service provider. It also does the reverse transformation from the service response message format to the format of the originating request.

An example implementation of this type of mediation module is an XSLT transformation from the requestor message schema to the provider's message schema.

Message augmentation

This type is used to add new parameters to an incoming request before it is sent to the service. Sometimes the incoming messages don't provide all the necessary information. The mediation module intercepts the incoming request message, performs functions such as database lookup, then augments the message with the required parameters before passing it on to the service provider. This pattern may also be used when the addition of optional parameters can speed up the processing of the service request.

The message protocol transformation and message format transformation patterns are used to resolve differences between the service consumer and service provider. The content-based routing and message augmentation patterns are used to implement a form of service virtualization, and to encapsulate the addition of intelligent logic to facilitate a more seamless connectivity and communication.

Back to top

Choosing the right ESB

Three different ESB products have been discussed so far: WebSphere DataPower Appliances, WebSphere ESB, and WebSphere Message Broker. This section provides some guidance on choosing the right product to meet your ESB requirements.

The WebSphere DataPower Appliances is a set of three different hardware appliances for SOA solutions that redefine the boundaries of middleware. They extend the SOA foundation with specialized, consumable, dedicated SOA appliances that combine superior performance and hardened security for SOA implementations. This appliance may be used at the enterprise perimeter, providing accelerated (at wire speed) XML, and enhanced SOA security processing with XML-level protection and structured data handling. It also provides very basic mediation capabilities, and integrates seamlessly with the security infrastructure based on IBM TFIM, Access Manager, and so on. If you are looking for an ESB gateway that also performs some basic message mediation, then the WebSphere DataPower Appliances is recommended.

WebSphere ESB will connect applications with standards-based interfaces to power your SOA. Building an ESB that is solely based on WebSphere ESB is an option when Web services support is absolutely critical, and the service consumer and provider communicate only with open standards. Since WebSphere ESB is built on top of WebSphere Application Server, it leverages all the transport protocols that the latter supports. WebSphere ESB supplements Application Server with mediation capabilities that provide intelligent service connectivity options.

WebSphere Message Broker is the most advanced ESB solution in the WebSphere suite of products. It provides any-any connectivity for both standards and non-standards-based technologies and applications. Message Broker also facilitates an any-any message and data transformation framework, realizing an SOA solution in the most complex and heterogeneous environment. This product offers transactional processing, near-universal connectivity over multiple transports (including WebSphere MQ, JMS, and HTTP), and reliability and performance unmatched by others in the market. It supports a broader set of transformation and mediation features, and supports integration assets that may or may not be Web services-enabled. When fast throughput and quality-of-service requirements are critical, Message Broker is a far better choice than WebSphere ESB.

Coexistence of multiple ESB products

A WebSphere DataPower appliance is complimentary with either WebSphere ESB or Message Broker. Figure 4 shows how the different ESBs may be used together.

DataPower and WebSphere ESB can coexist in an ESB infrastructure because they provide complimentary ESB features. While WebSphere ESB provides message mediation and transformation features within the enterprise firewall, a DataPower appliance can be deployed at the enterprise perimeter and configured to provide message security (denial of service, authentication, and authorization). DataPower can also provide basic message routing capabilities. In configurations with multiple departmental WebSphere ESBs providing services specific to line-of-businesses, the DataPower appliance can route messages to the correct WebSphere ESB run time, working as a federation hub for a multi-WebSphere ESB run time.

In situations where higher processing throughput is required in a WebSphere ESB-only run time, the power of DataPower appliances can be harnessed by offloading the XML processing overhead from the WebSphere ESB to the appliance, freeing the WebSphere ESB platform to provide higher service throughput. WebSphere ESB can also bolster a DataPower-only infrastructure. It can add full transaction support, complex message flow implementations, a persistent JMS messaging server, publish/subscribe support, and a wide range of application and technology adapters, including a robust support for IBM transaction processing environments.

DataPower bolsters a Message Broker solution the same way it does for WebSphere ESB. Because Message Broker is the most advanced ESB in the WebSphere suite, it adds more features to a DataPower-only solution than WebSphere ESB. Message Broker provides, among other things: a broader protocol support (any third-party JMS 1.1 support, telemetry, device integration); a tight integration with IBM transaction processing systems such as CICS and IMS; advanced messaging and complex event processing; a sophisticated scheduler and timer service; and a general purpose programming environment (Java, ESQL, C, and so on) to define arbitrary programming logic.

Each of the three ESB products can provide a targeted middleware solution. They can also work together to create a more advanced and end-to-end ESB based service connectivity solution. The configuration to choose is dictated by the requirements of a given problem domain.

Back to top

Conclusion

Share this... Digg this story Post to del.icio.us Slashdot it!

There are eight different SOA scenarios that IBM identifies as the most commonly occurring in typical SOA-based IT projects. To help customers get started with SOA, IBM offers business-centric and IT-centric SOA entry points and scenarios, and provides comprehensive guidance on how each scenario should be modeled, designed, and implemented using IBM tools, products, and methodologies.

In this article you learned about Service Connectivity, the second SOA scenario. The Service Connectivity scenario is a realization of the connectivity entry point. The connectivity entry point allows you to effectively connect your infrastructure, integrating all of the people, processes, and information in your company. With flexible SOA connections between services and throughout your environment, without much effort you can take an existing business process and deliver it through a different business channel. You can even connect to external partners outside your firewall in a secure way. You can also adapt legacy and third party applications using Web services technology to unlock their business value and foster their reuse.

This article also explained how the SOA lifecycle can be applied to the realizations, and how the IBM product suite can address the specific design, development, and run time requirements for service enablement.

Resources

• Read the other parts in this series:
  • Part 1, Realizing Service-Oriented Architecture, discusses IBM's SOA Foundation Lifecycle (or SOA Lifecycle) and how it allows IBM customers to think of SOA in terms of a software development lifecycle.
  • Part 2, An introduction to SOA solution scenarios, introduces the eight most common scenarios in a typical SOA-based IT project.
  • Part 3, Top ten tips for writing great IT project proposals, highlights the major steps you should follow to ensure the development of a high-quality proposal.
  • Part 4, Service creation options in a real-world SOA scenario, focuses on the Service Creation scenario.
  
  
• Learn more about IBM's SOA foundation.
  
  
• "Integrating WebSphere DataPower XML Security Gateway XS40 with WebSphere Message Broker" (developerWorks, Oct 2007) shows how the DataPower XS40 enhances a Message Broker flow with WS-Security, with security of the connection to Message Broker provided by HTTPS.
  
  
• The following IBM Redbooks are helpful:
  • WebSphere Message Broker Basics provides an overview of the latest release of WebSphere Message Broker and the Message Brokers Toolkit.
  • Implementing IBM Tivoli OMEGAMON XE for WebSphere Business Integration V1.1 covers the installation, configuration, and troubleshooting of IBM Tivoli OMEGAMON XE for WebSphere Business Integration on Windows, AIX, Linux, and z/OS platforms, with real life scenarios.
  • Patterns: SOA Design Using WebSphere Message Broker and WebSphere ESB discusses patterns for integrating WebSphere Enterprise Service Bus and WebSphere Message Broker, and includes a scenario to help you design, develop, and deploy these products.
  
  
• "Twelve great things about the WebSphere Message Broker V6.0.2 Toolkit" (developerWorks, Jan 2007) explains how to take advantage of new features in the WebSphere Message Broker V6.0.2 Toolkit.
  
  
• Read "Getting started with WebSphere Enterprise Service Bus and WebSphere Integration Developer" (developerWorks, Jan 2006) to learn how WebSphere Integration Developer is used to build mediation flows for the WebSphere ESB.
  
  
• Learn more about:
  • ITCAM for SOA
  • IBM Tivoli Federated Identity Manager
  • WebSphere Adapters
  • WebSphere DataPower SOA Appliances
  
  
• "Simplify integration architectures with an Enterprise Service Bus" (developerWorks, Aug 2005) dispels the myths of an Enterprise Service Bus and explains how you can apply this architectural style to the implementation of Service-Oriented Architecture-based applications.
  
  
• "A case for SOA governance" (developerWorks, Aug 2005) explains responsibilities of a governance body, and shows how you can effectively implement SOA governance.
  
  
• In the Architecture area on developerWorks, get the resources you need to advance your skills in the architecture arena.
  
  
• Browse the technology bookstore for books on these and other technical topics.
  
  

• Get an RSS feed for this series (Find out more about RSS feeds).
  
  

• Check out developerWorks blogs and get involved in the developerWorks community.

About the author

		Tilak Mitra is a Senior Certified Executive IT Architect in IBM. He specializes in SOAs, helping IBM in its business strategy and direction in SOA. He also works as an SOA subject matter expert, helping clients in their SOA-based business transformation, with a focus on complex and large-scale enterprise architectures. His current focus is on building reusable assets around Composite Business Services (CBS) that has the ability to run on multiple platforms like the SOA stacks for IBM, SAP and so on. He lives in sunny South Florida and, while not at work, is engrossed in the games of cricket and table tennis. Tilak did his Bachelors in Physics from Presidency College, Calcutta, India, and has an Integrated Bachelors and Masters in EE from Indian Institute of Science, Bangalore, India. Find out more about SOA at Tilak's blog. View Tilak Mitra's profile on LinkedIn or e-mail him with your suggestions at tmitra@us.ibm.com.

Rate this page

Please take a moment to complete this form to help us better serve you. Did the information help you to achieve your goal? Yes No Don't know   Please provide us with comments to help improve this page:   How useful is the information? 1 2 3 4 5 Not useful Extremely useful

Share this.... Digg this story del.icio.us Slashdot it!

Back to top