Integrate with SAP using WebSphere Message Broker
Are you thinking about integrating your applications with SAP using IBM® WebSphere® Message Broker (referred to hereafter as Message Broker) and the WebSphere Adapter for SAP? The first thing you need to do is consider the SAP interfaces you are going to use. Are you going to be calling into SAP from Message Broker (inbound calls), or pushing data from SAP into Message Broker (outbound calls)? Will you be dealing with BAPI, ALE, QISS, or AEP? These decisions have probably already been made for you by the SAP implementation team. As middleware, Message Broker has been designed to fit in with SAP architecture.
Introduction to SAP for Message Broker users
If you don't understand some of the terms in that last paragraph, then you probably haven't had your discussion with the SAP team yet. The two main interfaces with SAP are Application Link Enabling (ALE) and Business Application Programming Interface (BAPI). Message Broker also supports Advanced Event Processing (AEP) and Query Interface for SAP Software (QISS), but they are less common, so the article will just touch on them and focus mainly on ALE (and IDoc) and BAPI.
This article is not a comprehensive introduction to SAP. It focuses only on those aspects of SAP that are relevant to integration with Message Broker, and does not cover other aspects of SAP.
Outbound calls from Message Broker to SAP
Outbound adapters allow data to be passed from Message Broker to the SAP system.
Outbound calls are supported with the BAPI, ALE, and AEP interfaces using Advanced Business Application Programming (ABAP) handlers and QISS. With BAPI, the calls can be simple BAPI calls; BAPI can use Remote Function Calls (RFCs) or multiple BAPI calls in a single interaction (referred to as a BAPI unit of work). BAPI outbound calls have a request-response interaction style. The ALE interface lets you pass single or multiple intermediate documents (IDocs). These outbound calls are one-way, with IDocs passed to the SAP application. With the AEP interface, the adapter uses the ABAP handlers, and with the QISS interface, you can directly query SAP application tables. Figure 1 shows how these calls work:
Figure 1. Outbound calls propagated from Message Broker to SAP
Inbound calls from SAP to Message Broker
An inbound adapter enables data to be passed from the SAP system to Message Broker. Inbound calls are supported by the ALE interface using asynchronous event notification. For the inbound call, the adapter acts as an RFC server and listens for ALE events from the SAP application. The adapter uses an event recovery table to manage the inbound events.
For ALE outbound and inbound operations, queued RFC (qRFC) support is enabled for the ALE interface in Message Broker V7. Client applications can specify the queue to which IDocs are delivered in order to ensure the order in which those IDocs are delivered and processed by an SAP application. The BAPI callout and synchronous RFC callout introduced in Message Broker V7 are used to monitor any events related to the invocation of a BAPI call. With the AEP interface, the adapter processes any events related to custom IDocs using the ABAP handlers, as shown in Figure 2:
Figure 2. Inbound calls from SAP to Message Broker
A BAPI is a standardized business API that enables vendor systems to
interact with SAP systems. A BAPI is implemented as an RFC-enabled
function, so the adapter's BAPI interface can support any such function.
The BAPI interface APIs enable vendor systems to interact with SAP.
Adapters provide local transaction support for the BAPI interface using
the BAPI calls
operations are all supported, except in the case of the BAPI result set,
RetrieveAll is the only supported operation.
BAPIs are business functions provided by SAP. Typically, BAPIs provide applications with access to the business objects stored in the SAP database without the applications needing to know the details of how the tables are implemented. However, you can define and implement custom BAPIs to provide other functions that may or may not access the SAP database. BAPIs are RFC-enabled so that they can be accessed from applications external to SAP, such as a Message Broker message flow. There are two parts to a BAPI: the definition, and the implementation.
Programs running inside SAP can invoke BAPIs that are implemented and that execute on that system, or that execute on an external RFC server. The external RFC server might be another SAP system or an external application, such as a Message Broker message flow. In short, BAPIs are defined in SAP. If they are also implemented in SAP, then you can invoke them from a message flow and execute them on the SAP system, potentially accessing data in the SAP database tables, as shown in Figure 3:
Figure 3. Outbound calls using BAPI interface
Whether they are actually implemented in SAP or just defined there, they can be invoked by programs running on the SAP system. If they are not actually implemented in that SAP system, they can still be invoked from that SAP system across RFC to be executed in a Message Broker message flow. In that scenario, the message flow plays the role of an RFC server in the SAP model, as described later in this article.
BAPIs are defined in terms of their input parameters (imports) and output parameters (exports). BAPIs can be invoked synchronously or asynchronously. You will often hear the term transactional used in place of asynchronous when talking about SAP interactions. BAPIs are invoked over RFC, and you can use an SAP layer named tRFC to make these calls asynchronously and transitionally. In this sense, transactional means that the caller is given a promise that the call will be made at some point in the future, even if the system where the BAPI is implemented is not reachable when the call is made. This behavior should be familiar to anyone with experience with Message Broker or WebSphere MQ. As you might expect, when a BAPI is invoked asynchronously, the output parameters (exports) are never received by the caller, so this model really makes sense only for BAPIs that have only input parameters (imports).
IDocs are like envelopes that are used to send data between SAP applications. The data within these envelopes can have different structures and can be sent for different reasons. The IDoc type defines the structure of the data. All IDocs have a control record and one or more data records (the latter are also referred to as segments). The control record has the same structure in all IDocs, while segment structure is defined by IDoc type.
IDocs of the same IDoc type can have different message types. While the IDoc type reflects the structure of the data, the message type reflects the business reason for sending the data. Furthermore, an IDoc can be sent with a message code and/or a message function. The control record includes Message Type, Message Function, and Message Code fields.
IDocs are transmitted via tRFC. In fact, you may hear people use the term tRFC interchangeably with IDoc. This is not strictly accurate because as described above, BAPIs can also be sent over tRFC.
When we talk about senders and receivers of IDocs, we talk about logical systems or partners. These entities controlled by the ALE layer, which is responsible for routing IDocs between logical systems. The configuration of ALE lets the SAP administrator determine which logical systems can send which message types to which other logical systems.
The ALE interface provides an asynchronous interface for sending batch data represented by IDocs to SAP applications. The adapter extracts the IDoc information from the message and then uses SAP Java™ Connector function calls to convert the message representing the IDoc object to a table format that is compatible with the IDoc format. The adapter then uses RFCs in the SAP RFC library to establish an RFC connection to the ALE interface, pass the IDoc data to the SAP system, and then release the connection to SAP. Because the ALE interface is asynchronous, SAP returns only a return code and a null object to the caller. If no exceptions are raised, the outbound transaction is considered successful.
The success of the transaction can be verified by inspecting the IDocs that
have been generated in SAP. Only a return code (success or failure) is
returned, and no return data is sent back to the adapter. For the ALE
outbound call, all the information is in the IDoc and the only supported
Execute on the IDoc.
Delete operations do not make sense
for ALE because it is a batch-oriented asynchronous interface. The
receiving SAP application processes the data and performs appropriate
operations based on the data contained in the control record child object
of the IDoc, as shown in Figure 4:
Figure 4. Outbound operations using the ALE interface
The AEP interface of the WebSphere SAP adapter is used for both inbound and outbound processing. For inbound processing, it polls for events in SAP, converts them into business objects, and sends the event data as business objects to Message Broker. For outbound processing, the adapter processes events sent from an application to retrieve data from or update data in the SAP server.
Business objects for the AEP interface are defined in SAP as IDocs; either
standard IDocs delivered with SAP or custom-built IDocs can be used. The
External Service wizard is used to generate the business object definition
based on an IDoc. Business object development for the ABAP extension
module consists of creating an application-specific business object
definition and an associated ABAP handler for each operation that you want
to support. ABAP handlers reside in the SAP application as ABAP function
modules, communicate with the connector, and are needed to get business
object data into or out of the SAP application database. ABAP handlers are
responsible for adding business object data into the SAP application
operations) or for retrieving data from the SAP application database
Retrieve operations). You must develop operation-specific
ABAP handlers for each supported hierarchical business object.
The application-specific business object and the ABAP handler rely on each other's consistency to pass data into and out of the SAP application. If you change the business object definition, you must also change the ABAP handler.
In outbound processing, the adapter receives the AEP record as a business object that contains the business data along with the metadata. Depending on the operation set, the adapter looks up the ABAP handler to be invoked from the operation application-specific information. The adapter converts the business data to handler data format and passes it along with the ABAP handler information to the SAP application. After the object-specific ABAP handler finishes processing the business object data, it returns the business object data in IDoc format to the ABAP component, which converts the business object data back to its original format and returns it to the adapter. The adapter component in the SAP application invokes the appropriate ABAP handler and returns the results back to the adapter, which converts the results to a business object and returns it to the caller. With the AEP interface, the adapter makes use of the ABAP handlers.
QISS provides you with the means to retrieve data from application tables on an SAP server or to query SAP application tables for the existence of data. The adapter can perform hierarchical data retrieval from the SAP application tables.
QISS supports outbound interactions for read operations only
Exists). You can use this
interface in local transactions to look up records before write operations
example, you can use the interface as part of a local transaction to do an
existence check on a customer before creating a sales order. You can also
use the interface in non-transaction scenarios.
QISS supports data retrieval from SAP application tables, including
hierarchical data retrieval from multiple tables. The interface supports
static as well as dynamic specification of
WHERE clauses for
The Adapter Connection wizard finds the application data tables in SAP,
interprets the hierarchical relationship between tables, and constructs a
representation of the tables and their relationship in the form of a
business object. The wizard also builds a default WHERE clause for the
query. You can control the depth of the data retrieval, as well as the
amount of information, by using the
rowsSkip properties. All of these outbound QISS operations
are illustrated in Figure 5:
Figure 5. Outbound operations supported by QISS
SAP-specific enhancements in Message Broker V7
Message Broker V7 includes enhancements in SAP support and operations. The new SAPReply node provides a new way to interact with SAP, by using a message flow to implement a synchronous call out from SAP via the BAPI interface. Other enhancements make the administration model more flexible to allow for extension, reconfiguration, and horizontal scaling. New features include:
- A single RFC program ID now allows multiple different kinds of IDocs to be handled by different flows without having to redeploy or rediscover.
- In WebSphere Adapter for SAP Software V7, inbound synchronous processing of RFC-enabled functions is known as the BAPI callout or synchronous RFC callout. A BAPI callout or synchronous RFC callout supports the SAPReply node, which can therefore be used to send a reply to an SAP synchronous callout.
- High availability (HA) for SAP input nodes allows multi-broker failover using a WebSphere MQ HA multi-instance queue manager as the TID store.
- By using the
Waitparameter before calling the BAPI
Commitparameter, you can control whether the SAP Adapter waits for SAP to commit updates synchronously, or issues a commit and then returns while the SAP commit occurs asynchronously.
- Several SAP-specific features have been added, including the ability to specify a single program ID (enabling multiple IDocs to be handled by different message flows without disruption), support for SCI using a new SAPReply node, and BAPI commit-wait processing.
- Three adapter nodes are provided with Message Broker: the SAPInput
node, the SAPReply node, and the SAPRequest node. The SAPReply node is
the newest addition, proving BAPI callout or synchronous RFC callout.
Figure 6 shows how these nodes are represented in the Message Broker
Figure 6. SAP nodes in Message Broker
Let's take a look at some of these features in more detail.
You use the SAPReply node on an SAP program to call functions on remote RFC servers. These functions can be called asynchronously or synchronously; in the latter case, they must provide a reply. A message flow can act as an RFC destination and receive the function call through the SAPInput node. When the function is called synchronously, you use the SAPReply node to send a reply back to an SAP synchronous callout.
SAP BAPI callout or synchronous RFC callout
Message flows can implement both one-way and synchronous BAPIs that can be called from an SAP system. One-way operations are implemented with an SAPInput node, which can represent one or more BAPIs; synchronous operations are implemented with the combination of an SAPInput node and an SAPReply node, which can reside on different message flows. Synchronous and asynchronous calls are illustrated in Figures 7 and 8, respectively:
Figure 7. Synchronous call using SAPReply node
Figure 8. Asynchronous call using SAPReply node
Iterative discovery and iterative deployment
Message Broker V7 introduces two new features for using WebSphere Adapters: iterative discovery and iterative deployment. Before V7, to use an adapter with Message Broker, you needed to:
- Use the Adapter Connection Wizard to discover objects or operations from your SAP system and create your adapter component.
- Create a message flow with at least one adapter node.
- Create a BAR file and add your message flow, adapter component, and message set to it.
- Deploy your BAR file.
If at a later date you wanted to support additional objects or operations from your SAP system, you needed to repeat these steps, rerunning the Adapter Connection Wizard to create a new adapter component and message set. This procedure created problems, because previously discovered objects and operations had to be discovered again, even if you just added a single operation or object. In addition, previously deployed message flows, adapters, and message sets had to be redeployed -- which meant that they also had to be retested.
Iterative discovery and iterative deployment address these problems. Iterative discovery lets you add objects and operations to an existing adapter and message set, removing the need to completely replace your resources. Iterative deployment lets a single adapter node use multiple adapter components and message sets, which means that the Adapter Connection wizard can discover new objects and operations, the adapter component and message set can be deployed, and existing adapter nodes can start using the newly deployed adapters and message sets -- all without a restart. Only the new operations and objects need to be tested, and existing resources are left untouched.
Figures 9 and 10 illustrate these new features. These screenshots are taken from the Message Broker Toolkit under the Iterative discovery feature:
Figure 9. Iterative discovery
Figure 10. Configuring settings for the discovery agent
There are a number of scenarios in which iterative discovery and deployment can make your life easier:
- A single message flow acting as a generic gateway to SAP
- If a message flow is agnostic about the data being transferred to and from an SAP system, then nodes in that message flow are not sensitive to the kinds of objects or operations it receives or sends to the SAP system. However, the flow still needs a message set to serialize the message. So, to support more objects or operations, you need to discover them from your SAP system. Iterative deployment can be used to support additional objects or operations without re-deploying any of the adapter components or message sets, as you can deploy additional adapter components and message sets as and when you want to support them.
- Switching or migrating between two different SAP systems
- Iterative deployment allows secondary adapters and message sets for adapter nodes that contain new objects or operations. The connection details for the SAP system are held on the primary adapter, which is explicitly referenced by the node. It is possible to put all of your objects and operations in secondary adapters and message sets, so that when you want to switch between SAP systems, you can slowly replace the primary adapters of each of your adapter nodes, enabling BAPIs to move from one SAP system to another.
- A single SAP system, with many flows calling different BAPIs
- If you have multiple flows, all of which do different things but use the same SAP system, you will probably have the connection information for that SAP system duplicated across multiple adapters. If any of the details change, then your message broker administrator must change many adapters or many configurable services. But with iterative deployment, all of the adapter nodes can use a single primary adapter that contains the connection information, with their objects and operations in secondary adapters, minimizing the administrative overhead for changing connection details.
With iterative deployment, each enabled adapter node has access to additional message sets and to the types within these message sets. It is therefore important to ensure that the types do not clash, which you can easily do by using namespaces. When objects or operations share types, use a different namespace for your message set. If in the third scenario above, you have different flows doing different things, and these flows are developed by different developers, some communication is required to ensure that different namespaces are used.
Message Broker V7 adds support for making message flows that include SAPInput nodes highly available when using assured delivery. Previously, assured delivery relied on a transaction store kept on the broker's queue manager, in order to keep track of incoming requests in case a broker crashed. But having the same adapter deployed to two different execution groups or brokers can be a problem: if one of the brokers crashes, SAP will try to continue any conversations with the other broker, as it has the same RFC program ID. The other broker will have no record of this conversation in its transaction store, and so it will be unable to continue any conversations started by the crashed broker.
In Message Broker V7, this transaction store can be put on any queue manager and shared among execution groups and brokers on different machines. Therefore an active-active scenario can be set up; if one broker goes down, the other can continue its work by looking at the state in the transaction store, as shown in Figure 11:
Figure 11. High availability for SAP input nodes
A common requirement in building systems with SAP is to take IDocs from SAP and route them to different applications to process. Imagine that you are integrating SAP with another system that does not understand the physical format of the data that SAP sends out. The data is sent from SAP to the other system via ALE and tRFC layers in the form of an IDoc, with many different types of IDocs being sent. The SAP administrator has assigned you a single RFC program ID to receive these IDocs -- in other words, SAP sees your side of the interface as a single system. In addition, your system has at least one of the following requirements:
- The solution must be maintainable: The components and artifacts used to process different types of IDocs need to be modular in nature and isolated from each other.
- The solution must be extensible: New IDoc types may be added in the future and should not have any effect on the existing applications.
- The solution must be scalable: The processing of the IDocs must be distributable across many physical processors.
Prior to Message Broker V7, you could meet these requirements by using the
IDoc pass-though mechanism, which let you receive IDocs from SAP without
fully parsing them. The Message Broker tree would contain only a few
control fields, such as the IDoc name and a
containing the IDoc. You could use the IDoc name or a different field to
route the IDoc to another application (which could be another message
flow). If you wanted to route IDocs to different message flows, then you
were left with the problem of parsing the IDoc, as you now had the IDoc on
a queue or some other transport in binary form. You could achieve this
result by exporting a C header file from SAP and importing it into Message
Broker and using the MRM parser to parse the IDoc, but in doing so you
would lose all the benefits of the Adapter Connection wizard and the
message sets that it generates.
Message Broker V7 lets you run the Adapter Connection Wizard and generate a message set, and then associate it with an MQInput node, along with the DataObject domain. You can then parse the binary IDoc that has been routed using the IDoc passthrough mode. An integration pattern has been created to use this new function and solve the IDoc routing problem. The solution has two parts:
- A connection to SAP that receives the unparsed IDoc. It needs to be a generic component that is not sensitive to the exact type of IDoc being received, and it is implemented using the passthrough mode described above. This function already existed and is represented in Figure 12 as the flow on the left.
- The parsing and transformation of the IDoc, which is distributed
across a number of components (one per IDoc). It can be a message flow
that uses the new function described above or another application
altogether. Figure 12 shows how message flows can be used, one per
IDoc, to parse and process the IDocs:
Figure 12. Parsing and processing IDocs
This article described the important features of the SAP Adapter for Message Broker V7. You should now be able to discuss the interactions possible between Message Broker and SAP with the SAP team in your organization, and be able to plan an implementation of the Message Broker components needed for those interactions. Topics included:
- Outbound and inbound interfaces using WebSphere Adapter for SAP with Message Broker
- The two main interfaces with SAP: ALEs and BAPIs
- Synchronous and asynchronous calls using the SAPReply node
- Iterative discovery and iterative deployment
- High availability for SAPInput nodes
- IDoc routing
- IBM WebSphere Message Broker
- What's new in WebSphere Message Broker V7
by Matt Lucas (developerWorks, December 2009). WebSphere Message Broker V7 is a major release, and this article describes some of its significant improvements.
- Getting connected with WebSphere Integration Developer
Adapters, Part 4: An introduction to the WebSphere Adapter for
by Greg Adams, John Green, Richard Gregory, and David Van (developerWorks, August 2007). SAP is an enterprise resource planning (ERP) system that combines financial, human resource, operations, and corporate service systems. This article shows you how to integrate it with WebSphere products.
- Connecting your business using IBM WebSphere Message Broker
V7 as an ESB
by Darrell Bleakley, Jenny Chow, Kirstine Clapperton, Sowmya Hebbur Dayananda, Vivek Grover, and Sunho Sung (May 2010). This IBM Redbook describes the key features that make WebSphere Message Broker a powerful enterprise service bus solution in an SOA environment, illustrating the interoperability between WebSphere Message Broker and SOA applications.
- WebSphere Message Broker developer resources
Technical resources to help you use WebSphere Message Broker for connectivity, universal data transformation, and enterprise-level integration of disparate services, applications, and platforms to power your SOA.
- WebSphere Message Broker product page
Product descriptions, product news, training information, support information, and more.
- WebSphere Message Broker V7 information center
A single Web portal to all WebSphere Message Broker V7 documentation, with conceptual, task, and reference information on installing, configuring, and using your WebSphere Message Broker environment.
- What's new in WebSphere Message Broker V7
WebSphere Message Broker V7 provides universal connectivity with its ability to route and transform messages from anywhere to anywhere. Through its simple programming model and a powerful operational management interface, it makes complex application integration solutions much easier to develop, deploy, and maintain. This article describes the major enhancements in V7.
- Download free trial version of WebSphere Message Broker
WebSphere Message Broker V7 is an ESB built for universal connectivity and transformation in heterogeneous IT environments. It distributes information and data generated by business events in real time to people, applications, and devices throughout your extended enterprise and beyond.
- WebSphere Message Broker documentation
WebSphere Message Broker specifications and manuals.
- WebSphere Message Broker forum
Get answers to your technical questions and share your expertise with other WebSphere Message Broker users.
- WebSphere Message Broker support page
A searchable database of support problems and their solutions, plus downloads, fixes, and problem tracking.
- Redbook: Patterns: SOA design using WebSphere Message Broker
and WebSphere ESB
Patterns for e-business are a group of proven, reusable assets that you can use to more quickly develop and deploy e-business applications. This Redbook shows you how to use WebSphere Message Broker with WebSphere ESB to implement an ESB within an SOA. Includes a scenario to demonstrate design, development, and deployment.
- What's new in WebSphere Message Broker V7
- IBM WebSphere resources
- developerWorks WebSphere developer
Technical information and resources for developers who use WebSphere products. developerWorks WebSphere provides product downloads, how-to information, support resources, and a free technical library of more than 2000 technical articles, tutorials, best practices, IBM Redbooks, and online product manuals. Whether you're a beginner, an expert, or somewhere in between, you'll find what you need to build enterprise-scale SOA solutions using the open-standards-based WebSphere software platform.
- developerWorks WebSphere application connectivity developer
How-to articles, downloads, tutorials, education, product info, and other resources to help you build WebSphere application connectivity and business integration solutions.
- developerWorks WebSphere SOA and Web services developer
How-to articles, downloads, tutorials, education, product info, and other resources to help you design and build WebSphere SOA and Web services solutions.
- Most popular WebSphere trial downloads
No-charge trial downloads for key WebSphere products.
- WebSphere forums
Product-specific forums where you can get answers to your technical questions and share your expertise with other WebSphere users.
- WebSphere on-demand demos
Download, watch, and learn what WebSphere products and WebSphere-related technologies can do for your company.
- developerWorks WebSphere weekly newsletter
The developerWorks newsletter gives you the latest articles and information only on those topics that interest you. In addition to WebSphere, you can select from Java, Linux, Open source, Rational, SOA, Web services, and other topics. Subscribe now and design your custom mailing.
- WebSphere-related books from IBM Press
Convenient online ordering through Barnes & Noble.
- WebSphere-related events
Conferences, trade shows, Webcasts, and other events around the world of interest to WebSphere developers.
- developerWorks WebSphere developer resources
- IBM developerWorks
Join a conversation with developerWorks users and authors, and IBM editors and developers.
- developerWorks Webcasts
Free technical sessions by IBM experts that can accelerate your learning curve and help you succeed in your most difficult software projects. Sessions range from one-hour Webcasts to half-day and full-day live sessions in cities worldwide.
- developerWorks podcasts
Listen to interesting and offbeat interviews and discussions with software innovators.
- IBM Education Assistant
A collection of multimedia educational modules that will help you better understand IBM software products and use them more effectively to meet your business requirements.
- developerWorks blogs