 | Level: Intermediate Greg Flurry (flurry@us.ibm.com), Senior Technical Staff Member, IBM Software Group
Andre Tost (andretost@us.ibm.com), Senior Technical Staff Member, IBM
02 Apr 2008 
The transformation to a service-oriented architecture (SOA) includes aspects that cover the entire lifecycle of a solution, from inception, to design and development, to its ultimate deployment and management. IBM® published an SOA Reference Architecture that helps structure and position these aspects into a number of different components, and the IBM SOA Foundation includes a set of products that address specific components within the overall architecture. This article is the first of several that will discuss how products that are part of the IBM SOA Foundation can be used together. First up: how to add advanced transformation capabilities to IBM's set of Enterprise Service Bus (ESB) products: WebSphere® Message Broker, WebSphere ESB, and WebSphere DataPower®.
From IBM WebSphere Developer Technical Journal.
Introduction
If you've seen other articles and material about the IBM SOA Foundation, you've probably seen the Reference Model diagram in Figure 1. This model serves as a reference point to position both products and functionality within a service-oriented architecture (and will be used in this and upcoming articles to point out where in the overall architecture our discussion takes place).
Figure 1. The IBM SOA Foundation Reference Model
One of the core features of an Enterprise Service Bus (ESB) is the ability to transform data. Within the IBM SOA Foundation, the IBM WebSphere Transformation Extender product offers advanced transformation functionality. Hence, it is positioned to be tightly integrated with an ESB and, specifically, be consistently reusable across all ESB products within the IBM SOA Foundation. The remainder of this article describes this integration.
One product, many facets
You can consider IBM WebSphere Transformation Extender (hereafter referred to as Transformation Extender) as a "universal transformation engine" for the IBM SOA Foundation that enables delivery of trustworthy information for critical business initiatives and compliance with regulatory requirements. Transformation Extender supports validation, transformation, and enrichment of complex documents and messages in any format, and provides support for COBOL copybook, database schema, Java™, plain text, WSDL, XML DTD, XML schema, and other code formats. Available Transformation Extender Industry Accelerator Packs enable you to accelerate your development when using industry formats such as X12, EDIFACT, SWIFT, ACORD, HIPAA and HL7. Available Transformation Extender Enterprise Application Accelerator Packs enable you to accelerate your development when using formats from popular applications such as SAP, Siebel, and PeopleSoft.
Transformation Extender has multiple editions that offer different deployment options. Transformation Extender can:
- Run integrated with various application programs on various operating systems.
- Run in standalone or batch processing environments; particularly in CICS®, MVS™, and IMS™.
- Enhance event processing environments, database systems, WebSphere MQ messaging, and more.
- Enhance WebSphere integration environments, particularly those using WebSphere Process Server, WebSphere Message Broker, and WebSphere ESB.
The final point in the list above is the subject of this article.
The Transformation Extender transformation engine is complemented by advanced tooling called the Design Studio. This Eclipse-based tool enables you to define formats (or import predefined formats), then create and test format maps that transform from one format to another by using drag and drop techniques, all without writing any code. Design Studio then deploys the maps to the Transformation Extender runtime engine in any run time environment. Design Studio is relevant to the WebSphere integration environments listed above, as well as to WebSphere DataPower, also a subject of this article.
The general procedure for leveraging Transformation Extender consist of these steps:
- Install and configure the Transformation Extender engine as appropriate for the chosen run time deployment option.
- Use Design Studio to define the source and target formats; this can be done for custom formats or using formats available with the Accelerator Packs.
- Use Design Studio to define the map that transforms from the source to the target format.
- Use Design Studio to test the map.
- Use Design Studio to deploy the map to the Transformation Extender engine (or to WebSphere DataPower).
- Use the Transformation Extender engine (or WebSphere DataPower) in service interactions.
IBM's ESB products
IBM considers an ESB to be an architectural pattern that can be implemented using a variety of products, either individually or in concert. They all support a number of core functional features that are expected of any ESB product. These features are described in detail in the article Exploring the Enterprise Service Bus, Part 1: Discover how an ESB can help you meet the requirements for your SOA solution.
The three ESB products that are part of the IBM SOA Foundation are:
-
WebSphere Message Broker focuses on enterprise-level, universal connectivity between heterogeneous environments, supporting a large number of protocols and data formats, with specifically strong support for WebSphere MQ as the messaging backbone.
-
WebSphere ESB emphasizes support for advanced Web services interactions. It is based on the Service Component Architecture (SCA) and leverages the WebSphere Application Server Network Deployment J2EE™ run time environment.
-
WebSphere DataPower is an SOA appliance that offers quick deployment and accelerated handling of XML and Web services interactions, including support for a rich set of security standards.
See Resources for detailed information about these products.
Putting Transformation Extender and the ESB together
Let's look at how Transformation Extender and the ESB products can actually be leveraged together, starting out with a discussion on where both pieces fit into the overall architecture, followed by descriptions of each specific product combination.
Architectural perspective
Figure 2 shows a low level architectural view of a service interaction in SOA. This scenario uses an ESB in the role of intermediary between the roles of the service consumer and the service provider:
- A consumer instance issues a request for (from the consumer perspective) an abstract service provider.
- The request is intercepted by a mediation instance that is part of the ESB.
- The ESB can process the request message to deal with any syntactic mismatches between the consumer instance and the actual provider instance that processes the request.
- The mediation instance can also process any response message returned from the service instance.
Figure 2. Service interaction with mediation
Figure 3 shows some internal detail of what takes place in the mediation instance:
- The incoming request message is processed by a request flow. The request flow comprises a number of steps. The individual steps, sometimes called mediation patterns or primitives, can receive or send messages using various communication protocols, parse or serialize the message, and can manipulate the message in many ways. One or more steps can be forms of transformation that are part of the standard capabilities of a particular ESB product.
Figure 3. Mediation internals
- It is possible for a primitive to in fact simply delegate the needed activity to an external ancillary service, as shown in Figure 4. In such a situation, the processing is taking place architecturally in the mediation instance, while physically it is actually done outside of the ESB product.
Figure 4. Ancillary processing in a mediation
The Transformation Extender transformation engine can be used to provide such ancillary transformation processing when, for example, the messages nicely match built-in Transformation Extender formats, match formats available in the Accelerator Packs, or would require overly complex development in the environment of the associated ESB product.
WebSphere Transformation Extender V8.2 and WebSphere Message Broker V6.1
In the WebSphere Message Broker (hereafter referred to as Message Broker) environment, a message flow receives data for processing via various input nodes and sends data out via various output nodes. A number of different types of Message Broker nodes can be used to process the data, including sophisticated forms of transformation. However, the addition of Transformation Extender brings support for additional, complex message formats using the Transformation Extender Accelerator Packs; this could simplify development with complex customer data formats. Transformation Extender can also support advanced transformation requirements, such as efficient processing of large data records or messages, and advanced data validation without complex coding.
Transformation Extender is not included as part of Message Broker; both are licensed separately. You must install the IBM WebSphere Transformation Extender for Integration Servers edition in your Message Broker. This installs a Transformation Extender node for processing data in message flows. You must also install, as needed, Transformation Extender Design Studio plug-ins into the WebSphere Message Broker Toolkit.
Figure 5 shows the general form of a message flow in Message Broker:
- An input node (primitive) parses an incoming message into an internal form that other Message Broker nodes can process.
- Various built-in nodes can perform transformations.
- Finally, an output node serializes the transformed message and sends it to the service instance.
Figure 5. WebSphere Message Broker
Figure 6 shows the usage of Transformation Extender:
- The input node parses an incoming message and makes it available to the Transformation Extender node that appears on the node palette after Transformation Extender is installed.
- The Transformation Extender node sends the message and the relevant transformation map to the Transformation Extender engine for complex processing.
- The Transformation Extender engine runs in the same process, enabling very efficient message processing.
- The mapped message is then returned to the message flow, and processing continues with the next step (node) in the flow.
Figure 6. WebSphere Message Broker with WebSphere Transformation Extender
Thus, the integration between Message Broker and Transformation Extender is completely handled via the Transformation Extender node that can be embedded in any Message Broker flow. The message is automatically passed into the Transformation Extender engine and back, and no further coding is required.
WebSphere Transformation Extender V8.2 and WebSphere ESB V6.1
In the WebSphere ESB SCA environment, a mediation module receives data for processing in a mediation flow component via an export, or sends data out via an import, as shown in Figure 7.
Figure 7. Request and Response flow in WebSphere ESB
The incoming or outgoing data can be in a wide range of formats, such as COBOL or EDI. The data must be converted into a data object before the mediation flow component can process it. The exports and imports at the edges of the module are responsible for converting external data to data objects, and vice versa. Exports and imports contain data bindings for this purpose. All exports and imports have data bindings that convert XML data to and from data objects, but if the external format is something other than XML, you can configure a custom data binding for exports and imports with JMS, EIS, HTTP, and native MQ protocol bindings.
Figure 8. Exports and imports with data bindings
To transform external data in the exports and imports, you can write custom data bindings, or you can use the built-in Transformation Extender data binding provided by the WebSphere ESB product to convert external data to a data object, and vice versa. Transformation Extender is not included as part of WebSphere ESB; both are licensed separately. You must install the IBM WebSphere Transformation Extender for Integration Servers edition in your WebSphere ESB server. The two-step installation process lets you to install the run time Transformation Extender engine in WebSphere ESB, and then install, as needed, Transformation Extender Design Studio plug-ins into the WebSphere Integration Developer tooling for WebSphere ESB.
Figure 9 shows the data flow when using Transformation Extender with the Transformation Extender data binding to convert incoming data to a data object:
- The client delivers the data to the export in its format.
- The export then passes the data in that format to Transformation Extender via the Transformation Extender data binding.
- The Transformation Extender transformation engine converts the data to data object format and returns it to the export via the Transformation Extender data binding.
- The Transformation Extender transformation engine runs using the same process as the mediation flow for efficient data processing.
- The export then passes the data object on to the mediation component for processing.
Figure 9. WebSphere ESB and WebSphere Transformation Extender
Figure 9 also shows the data flow when using Transformation Extender with the Transformation Extender data binding to convert outgoing data from data object format into the native format required by the client:
- The mediation component sends a data object to the import.
- The import passes the data object to Transformation Extender via the Transformation Extender data binding.
- Transformation Extender converts the information to the native format of the client and this information is returned to the import via the Transformation Extender data binding.
- The import sends the native data to the client.
When using Transformation Extender with WebSphere ESB, you might, of course, need to use the Transformation Extender Design Studio to define the external formats and create the appropriate mappings between such formats and the data objects used in WebSphere ESB.
There is a fundamental difference in the way you integrate the Transformation Extender engine with Message Broker and how you do it in WebSphere ESB. In Message Broker, you can invoke Transformation Extender anywhere in a message flow. In WebSphere ESB, you can invoke Transformation Extender at the "edge," namely right before data goes in or out of the mediation flow. Both methods are essentially equivalent, and neither is necessarily better than the other.
WebSphere Transformation Extender V8.2 and WebSphere DataPower
WebSphere DataPower follows the general mediation model, again with product specific details, as shown in Figure 10.
Figure 10. WebSphere DataPower
Like the other products, WebSphere DataPower has its own built-in specialized transformation engine, based on extended XSLT. Furthermore, the transformation engine can leverage hardware capabilities that are built into the product. Using the external Transformation Extender engine would not leverage such hardware, and thus the usage model for DataPower is quite different than Message Broker and WebSphere ESB. In fact, Transformation Extender is not used, but the Transformation Extender Design Studio is.
The Design Studio provides a highly productive way of defining formats accepted by the DataPower appliance and, similarly, for defining the maps between those formats. However, as of this writing, the Accelerator Packs cannot be used with DataPower.
With DataPower, you use the Design Studio to create the necessary maps and to load the maps into DataPower. The internal transformation engine then uses the maps to effect the proper transformations during mediation of service interactions.
Figure 11. WebSphere DataPower and Design Studio
Conclusion
The ability to transform data from one format into another is one of the core functional features of an Enterprise Service Bus. This article explained how you can leverage the WebSphere Transformation Extender product as a complex transformation engine in each of IBM's ESB products. The level of integration, however, differs between them:
- With WebSphere Message Broker, you can insert a node that represents the Transformation Extender engine directly into the flow, and transform data where needed within the flow processing.
- With WebSphere ESB, you insert the transformation engine at the edge of the mediation to transform data either on its way into or out of the mediation.
- With WebSphere DataPower, you don't use the actual Transformation Extender runtime engine, but instead you leverage its tooling (Transformation Extender Design Studio) to create the transformation maps, which are then executed using native DataPower capabilities.
Resources
About the authors | 
| |
Greg Flurry is a Senior Technical Staff Member in the IBM SOA Advanced Technology group. His responsibilities include working with customers on service-oriented solutions and advancing the IBM service-oriented products. |
| 
| |
Andre Tost works as a Senior Technical Staff Member in the IBM Software Services for WebSphere organization, where he helps IBM customers establishing Service-Oriented Architectures. His special focus is on Web services and Enterprise Service Bus technology. Before his current assignment, he spent ten years in various partner enablement, development and architecture roles in IBM software development, most recently for the WebSphere Business Development group. |
Rate this page
| |
