Back-end Integration Services

Several key services are required to create a robust reliable Back-end Integration solution. Although not all present in every Back-end Integration deployment, these services are the core building blocks of an EAI solution.

Adapters

Adapters encapsulate the transport and target application interface to enable loose couple of applications. Without adapters, each application would need to implement the specific interface of the target application. Adapters free applications from implementation of the following:

• Abstraction of remote interface
• Connection protocol
• Messaging interface
• Data marshalling of the interface parameters or message
• Mapping to standard message formats

Protocol specific adapters

Protocol adapters transform the protocol used by a communicating partner application. They are particularly important for allowing integration with legacy systems since legacy systems typically cannot be changed to adopt a new, common protocol. Instead legacy systems may require terminal emulators, for example. Software components, which implement protocols native to an integration hub, are also included in this service, for example, messaging, IIOP, and so on.

Transformation

In almost all integration activities, transformation from source format to target format is required. For process integration this requires transformation between different API or message formats whereas in data integration it is transformations between different data repository schema and semantic definitions. The base transformation process involves format modifications (both syntax and semantic mapping). Often more sophisticated transformation processes like message augmentation (or boosting) and data cleansing are required to handle the idiosyncrasies of application implementations.

Message handlers

Message handlers deal with the discovery (parsing) of the in-bound message content, the optional conversion to a common internal format and the building of the format expected by the target systems. Current implementations use XML as the internal format and increasingly also for the out-bound message format.

Message Routing

Except for the basic 1:1 integration projects, routing of messages is a key component of the EAI solution. This includes:

• managing the delivery of the messages over communication connections, including protocol conversion, flow control, and guaranteed delivery, and connection optimization (for example, connection pooling)
• multi-point message decomposition/recomposition to enable one to many and many to one message routing
• routing definition for content-based routing often with associated directory-based or rules-based routing capabilities.

Guaranteed Delivery

Provide an infrastructure that guarantees the message is delivered once and once only to the target system. Transactional ACID properties bound the PUT and GET of the messages onto the messaging queue. This ensures that the process is not left in an undetermined state and that the operation is now lost. Target application failure or system down conditions result in either the message remaining on the queue for a later update or onto an error queue for handling by systems management.

Message Repository

A catalog of messages for process integration or metadata definitions for data integration provide an integrated interface warehouse. This system of record of the interfaces not only captures the enterprise architecture but fuels the transformation and routing process.

Transactional Control

This enables the coordination of multiple application operations to be coordinated to provide an atomic deterministic end result. Often known as two-phase commit this service ensures ACID properties (Atomic, Consistent, Isolated, and Durable) are preserved. Transactional services provides a a commit and rollback facility that works with compliant XA resources to ensure that either all the updates are done or all the updates are rolled back.

Business Process Management

Business Process Management is used to drive an extended business process. It is used for long running business processes that often involve complex high-latency applications where the process can extend anywhere from minutes to days in duration. It is also useful where process execution requires interaction with users.

Job Scheduling

EAI activities are often initiated by an application event which is detected through an adapter or database trigger. When integration activities do not need to be executed immediately, a simpler approach for scheduled EAI activity enables integration without the need for application event monitoring or triggers. The EAI solution provides job scheduling to meet this requirement.

Audit Log

To establish non-repudiation of processing operations and for systems management, the EAI Infrastructure provides an audit log capacity where messages and operations are logged to allow for later analysis and reporting.

Systems Management

Systems Management ensures that the EAI solution is easily controlled by the IT staff. This includes:

• Deployment of the EAI solution
• Managed daily by operations staff
• Detection of error conditions
• Enhancing and extending the system for new solutions and integration
• Monitor and report on the integration activities including Audit Reports

Workload Management

The EAI infrastructure must provide a clustering solution where multiple instances of the integration flow can be instantiated both for fail-over and for load balancing. Support for multiple hardware platforms and exploitation by workload management products are key to making this happen.

Security

Permission to access the participating applications may be associated with the requesting application itself or this application may carry the credentials of a user initiating the actions. Consequently access control can be applied as far as the requesting application (a transit of trust) or only from the Integration Hub (a trusted source) that authenticated the original request. Securing messages transported and ensuring integration is achieved only with authorized applications under the correct user credentials is a must. The EAI solution needs to provide:

• data protection through encryption
• authentication of users - either subscribing applications and in cases where non-repudiation of the end-user is required, authentication of the end-user
• authorization of the user for participation in this specific integration activity.

Directory

A centralized directory for the sharing of user and application information. This centralized directory ensures easy extensibility of the EAI solution by removing the need to explicitly define participants (users or applications) in the EAI implementation.

Data transformation

A service, frequently referencing a repository of common data formats, to resolve data representation (schema) differences between the applications to be integrated. In simpler cases this service can be performed by the message handlers--especially if using the power of XML.

Decomposition/Recomposition

The attraction of Back-end Integration is, amongst others, the ability to access multiple applications with a single request. To make this possible a service is required to turn such a complex, compound request into a series of simple distinct requests (decomposition) and then to compile the results into a single response (recomposition). This service is particularly important if the source of the request is an end-user.

Routing/ Navigation

This is also referred to as micro workflow or system workflow. The multiple requests, generated by the Decomposition/Recomposition service can be executed in an arbitrary sequence and with parallelism. The ordering of the execution is sometimes referred to as "message choreography" and essentially represents the navigation among the target systems, driven by business rules. This navigation frequently records the degree of completeness of the request and the cases when error messages must be generated. In its primitive form navigation merely performs the routing of a request to the target adapter/connection.

State management

This is an optional facility supporting the preservation of state information between multiple application message exchanges. It can also support work-in-progress data, obviating the need for repeated access to target systems.

Local business logic

This service provides an opportunity to augment the results of compound requests or in a more complex case to generate additional requests. The business logic frequently drives the navigation as well.

(Business) unit-of-work management

A number of Application patterns involved in Back-end Integration need to execute under transactional control such that a unit-of-work completes entirely or not at all. The unit-of-work here is defined in a broad sense, which may span multiple, independent changes occurring within independent applications, at times during an extended time period.

If you have determined that the Back-end Integration pattern is appropriate for use in your solution, the next step is to select an Application pattern.

If the Back-end Integration pattern is not appropriate for your development efforts, review the Business patterns to determine which pattern best addresses your e-business needs.