Integrate remote enterprise information systems with JCA, JMS, and Web services

Mikhail Genkin (genkin@ca.ibm.com), Scenario Designer, IBM

Mikhail Genkin is a Scenario Designer working with the IBM SWG System House group at the IBM Toronto Software Development lab. His work involves conducting expert reviews of IBM and competitor tools for Java and Web service-based enterprise access. He advises several IBM development teams building WebSphere tools for J2EE, Web services, RDBMS, and EIS access about the usability, performance, and interoperability aspects of their products. He has contributed to several releases of VisualAge® for Java, Enterprise Edition; WebSphere Application Developer; and WebSphere Application Developer Integration Edition. Mikhail holds a bachelor's degree in Geophysics from Carleton University, Ottawa, Canada, and an M.Sc. in Earth Sciences from University of Ottawa. He is a Sun Certified Java Programmer, and an IBM Certified Developer Associate - IBM VisualAge for Java.

Summary: This article shows you how to integrate legacy transactions running on a geographically remote enterprise information systems (EIS) into your business processes. It discusses possible quality of service (QoS) issues and how to overcome them using a combination of Java™ Connector Architecture (JCA), Java Messaging Service (JMS), and Web services. The article also covers how to use WebSphere® Studio Application Developer Integration Edition tools, WebSphere MQ, and WebSphere Application Server to enable the required levels of QoS and boost developer productivity.

Date: 05 Mar 2004
Level: Introductory
Activity: 625 views
Comments:

Introduction

Many real-world solutions require integrating geographically remote enterprise information systems (EIS) into business processes. One example of such a solution is described in the article "Integrating business processes to streamline the supply chain, Part 1: Use global repositories to manage synchronization of item data." The supplier application described in that article needs to be able to aggregate data from several EISs:

EIS 1, which belongs to corporate headquarters and contains product description information
EIS 2, which belongs to the marketing or sales divisions and contains up-to-date pricing information
EIS 3, which belongs to the manufacturing division

In this article I focus on the case of EIS 3. Although part of the same WAN, this EIS is geographically remote with respect to the application server that executes key middle-tier business logic: the description handler and catalog aggregation processes. It is owned and operated by a different division of the company, and serves many different applications operated by other divisions.

Variable network and server load conditions lead to unpredictable response times during calls to transactions executing on this EIS. Business processes that require access to transactions running on this remote EIS may end up being blocked for long periods. The next section discusses how to overcome this problem by using JMS to achieve reliable network transport and JCA to enable access to EIS.

Accessing EIS transactions with JCA

The J2EE platform includes a specification that provides a standard interface for accessing EIS transactions and data: the Java Connector Architecture (JCA) specification. The JCA specification is a guide that explains how to develop and use a pluggable component called a resource adapter. The JCA-compliant resource adapter is a set of Java classes that comes packaged in a RAR file, along with an XML deployment descriptor that details the resource adapter's capabilities.

The JCA specification defines two sets of contracts:

System-level: Define the standards for connection management, transaction management, and security to ensure that the resource adapter can be plugged into an application server.
Application: Define the standard manner in which applications can interact with the resource adapter.

The application contract defines the standard API, or Common Client Interface (CCI), used by application components for EIS access. Client applications can use CCI to talk directly to the EIS using Java. Web service invocation frameworks, such as the IBM Web Service Invocation Framework (WSIF), also use CCI to expose EIS transactions as Web services.

See Resources for a link to an article with more details about JCA resource adapters and how to leverage the IBM WebSphere Studio Application Developer (Application Developer) Integration Edition tools for JCA development.

To use JCA to access EIS transactions with Application Developer Integration Edition, you need to do the following high-level steps:

Deploy the resource adapter to the integrated development environment (IDE).
Use business integration tools, which are part of the Business Integration perspective, to generate an enterprise service for the required EIS transaction (more details below).
Use the Application Developer Integration Edition IDE to unit test and integration test the new enterprise service.
Deploy the resource adapter and the generated enterprise service, which is essentially a J2EE application, to the WebSphere production application server.
Configure EJB deployment descriptor settings to achieve the desired level of transactional support and security.

Application Developer Integration Edition generates a stateless session bean that uses WSIF to invoke the resource adapter with the input data, and returns the output data back to the calling code. When the resource adapter is deployed on the same application server as the generated session bean, and it can establish a direct communication channel with the target EIS (using an EIS-specific protocol, such as CICS® ECI), then the above steps are all that's required to integrate the legacy transaction with your middle-tier application logic.

But what happens if the desired EIS is geographically remote with respect to the application server hosting the middle-tier business logic, and belongs to a different branch of the organization? It may not be feasible to communicate with the EIS using EIS-specific protocol. Owners of the EIS might not be willing to expose the EIS directly. Synchronous calls to the EIS could block the middle-tier logic because of slow network conditions. The following sections discuss how to overcome these problems by combining JCA with messaging technology.

Using JMS for cross-system communication

JMS messaging

The messaging programming paradigm has become popular for building business integration solutions. With JMS messaging, you can construct distributed applications whose components are loosely bound to each other. Different components don't need to be aware of each other's interfaces, just the format of the messages that they exchange. Portions of the overall application can remain operational, while others may be down, because of the asynchronous nature of messaging communication.

The Java Messaging Service (JMS) specification does for messaging what the JCA specification does for enterprise application integration. The JMS specification defines a set of contracts that describe how a particular messaging engine can be plugged into a J2EE application server. Manufacturers of messaging software supply JMS providers that can be plugged into any application server. The JMS API also defines the programming model (standardized API) that describes how one application component can send messages to another application.

JMS gives you powerful controls over QoS characteristics associated with calls between components of a distributed application. Of particular importance to this solution are:

Assured delivery for messages, even in case of system outage
Unit-of-work semantics for calls between application components

The next sections discuss how these aspects of JMS can be combined with JCA to address problems related to accessing remote EIS.

Integrating remote EIS with JCA and JMS

Figure 1 below shows the logical architecture of the item synchronization application. A business process executing on the middle tier needs to access data from three EISs. EIS3 is a CICS system that belongs to the manufacturing division, is located overseas, and contains up-to-date product availability data.

Figure 1. Logical architecture of the item synchronization application
Logical architecture of the item synchronization application

The Catalog Aggregation business process accesses three EISs. EIS1 contains product description data, EIS2 contains product price data, and EIS3 contains product availability data. The business process transforms and aggregates the data into a single catalog database.

The manufacturing division does not expose this EIS directly to other organizations. It operates a cluster of WebSphere Application Servers that host adapters and other key business logic related to the manufacturing division's business functions. The manufacturing division's application server can access EIS3 using the CICS ECI JCA resource adapter that was deployed on the manufacturing division's application server together with the CICS Transaction Gateway.

The Catalog Aggregation business process that needs to access EIS3 executes on the corporate WebSphere Application Server. This application server is not part of the same application server cell as the one that belongs to the manufacturing division of the company. The two application servers communicate with each other using messaging technology. The resulting topology is shown in Figure 2.

Figure 2. Accessing EIS3

EIS3 is owned by the manufacturing division of the company, which operates a cluster of WebSphere Application Servers. These servers host services that serve as adapters and enable connectivity to the manufacturing division's EISs. The services, including the product availability service, are accessible by a messaging channel provided by the WebSphere MQ JMS provider.

To build this solution the manufacturing division and corporate headquarters need to collaborate. Developers building the manufacturing division's applications need to expose the EIS3 transactions as enterprise services. Developers building the item synchronization application for corporate headquarters need to integrate these enterprise services into their solution. Application Developer Integration Edition's business integration tools provide very good support for these tasks. Developers working for the manufacturing division first need to do the steps below.

EIS3 is a CICS system that, among others, runs the COBOL transaction called CHKSTCK, which we need to call to get the most up-to-date product availability information. The input to this transaction is the product identification number. The return value is a data structure describing the product packaging unit type, and how many units are currently in stock. In this article I use CICS as an example for EIS3, but the steps and discussion below could apply equally well to any EIS.

To expose this transaction as a service, developers working for the manufacturing division will use Application Developer Integration Edition tools to create a service with a CICS ECI binding, as follows:

Import the CICS ECI JCA resource adapter provided with Integration Edition into the Integration Edition IDE using File > Import > RAR File
Create a Service Project (File > New > Service Project) and a package (File > New > Package) to store the artifacts that will be generated by Business Integration tooling.
Generate a service for the CHKSTCK transaction by invoking the Create Service wizard (File > New > Service build from…), and follow prompts from the wizard. See Figure 3 below for an example of excellent samples that Application Developer Integration Edition provides. These examples explain exactly how to expose CICS COBOL transactions with CICS ECI adapter. I omit the details here and encourage you to consult the Application Developer Integration Edition documentation.
Figure 3. Generating service definitions for a CICS system by CICS ECI protocol

Application Developer Integration Edition generates three WSDL files that contain the port type definition (abstract interface, message formats, and operation signatures), operation bindings, and service definitions for the EIS3 service, as shown in Figure 4. The AvailabilityCICSECIService can talk to the CICS system using the CICS ECI protocol, using the CICS Transaction Gateway running on the same machine as the host application server.

Figure 4. Generating three WSDL files
Generate a service deployment with an inbound JMS binding type. Select the Service.wsdl file in the Services view. Click the right mouse button to bring up the pop-up menu, and select Enterprise Services > Generate Deploy Code. On the first page of the Generate Deploy Code wizard, be sure to select JMS as the inbound binding type, as shown in Figures 5 and 6 below.
Figure 5. Generating an enterprise service deployment with a JMS inbound binding

The Generate Deploy Code wizard generates two additional WSDL files: a J2EE application and an MDB that listens to incoming messages and dispatches reply messages.

Figure 6. WSDL files
Unit test the adapter. Use the facilities provided with Application Developer Integration Edition, which provides an MQ simulator that lets you configure request and response queues in the local unit test environment (UTE). You can use this feature to unit test the generated adapter code prior to production deployment.
Deploy the generated code to the production application server. You'll need to export the generated J2EE enterprise application (plus the WSDL artifacts) from the Application Developer Integration Edition's IDE by:
1. Selecting the generated enterprise application in the J2EE view
2. Clicking the right mouse button to bring up the pop-up menu
3. Selecting Export
Then the application server administrator, or whoever is performing the deployer function, needs to deploy the generated EAR to the production server using the WebSphere Application Server Administration Console.
Export the generated JMS-bound service definition to a local UDDI server with the Web services explorer in Application Developer Integration Edition, as shown in Figure 7. Now developers working on the item synchronization application can import the generated JMS service.
Figure 7. Exporting generated JMS-bound service to a local UDDI registry

Now that you've seen how to create the adapter for the product availability system and expose it as a service, take a look at what the developers working on the corporate item synchronization application must do to enable access to that system:

Import the service definition for the product availability service from a local UDDI. This can be done using the Web services explorer in Application Developer Integration Edition.
Generate a proxy for the service by selecting the service in the Services view, clicking the right mouse button and selecting Enterprise Services > Generate Service Proxy. This brings up a wizard, shown in Figure 8, that guides you through the necessary steps to generate a Java bean that encapsulates and simplifies service invocation calls. The proxy is a Java bean that implements the getProductAvailability method defined in our WSDL file.
Figure 8. Generate Service Proxy wizard
Use the proxy in the implementation of the Catalog Aggregation component.

This section described how to construct the desired EIS integration topology. The next section discusses key aspects of messaging system setup required to achieve the desired topology and necessary QoS level.

Achieving the desired physical topology

The physical topology shown in Figure 1 can be readily constructed using WebSphere software. The WebSphere Application Server ships with a built in JMS provider called WebSphere embedded messaging, which allows for messaging between processes running on different application servers. To use this provider, application servers have to be part of a single cell (cluster instance). To exchange messages, both application servers would have to communicate synchronously with one instance of the embedded messaging server.

Given our solution constraints, this is not feasible. Application servers belonging to the manufacturing division and those hosting the item synchronization application typically belong to different cells. Establishing synchronous communication between a messaging client and a messaging server across less-than-ideal network conditions would leave us with the same problem we initially set out to solve, so we need to look for a different solution.

The WebSphere Application Server can be configured to use the WebSphere MQ JMS provider instead of embedded messaging. Essentially, this means using the highly popular WebSphere MQ (formerly MQSeries®) as the JMS provider for both the manufacturing division's application servers and corporate servers hosting the item synchronization application. WebSphere MQ provides features that will let the item synchronization application send messages even if the destination is temporarily unavailable because of poor network conditions. (See Resources for more information about WebSphere MQ.)

To do the above, the system administrators would need to:

Install WebSphere MQ on the physical machine that hosts the WebSphere Application server (for both the manufacturing division's server and the item synchronization application server).
Configure WebSphere Application Servers (both manufacturing and corporate) to use WebSphere MQ as the JMS provider. (See Resources for a link to detailed instructions.)
Configure WebSphere MQ queue managers, queues, and channels to enable communication between the two systems.

WebSphere MQ queue managers need to be running on both the manufacturing division's server and the server hosting the item synchronization application. The item synchronization application's catalog aggregation process makes calls to the AvailabilityProxy (which we generated in previous section). The AvailabilityProxy uses WSIF to post messages to the request queue defined on the local JMS provider. The local WebSphere MQ queue manager, which underlies the local JMS provider implementation, uses a transmission queue and channel to enable guaranteed delivery of messages to the remote application server belonging to the manufacturing division. If the remote system is temporarily unavailable, the queue manager stores the request message persistently and retransmits it later.

On the manufacturing division's application server the MDB that implements the AvailabilityService (generated earlier) listens to a local queue, receives the request message, and uses WSIF to invoke JCA calls to EIS 3 and retrieve the availability data. It then posts the response message to a queue defined on the local JMS provider. Again, as with the request call, the underlying WebSphere MQ queue manager uses a transmission queue and a channel to return the response message to the application server that hosts the item synchronization application. The AvailabilityProxy retrieves the response message from the response queue defined on the local JMS provider and returns the availability data to the catalog aggregation process.

Conclusions

Many projects require integrating remote EISs into new applications. You can solve problems that may be associated with these systems by combining JCA and JMS technologies. WebSphere Studio Application Developer Integration Edition provides excellent tooling support for building applications with the required topology. WebSphere MQ JMS providers running on the two remote systems with intercommunication enabled provide location transparency and guaranteed message delivery, and shield calling application code from having to deal with transport reliability issues.

Resources