Integrate with SAP using WebSphere Message Broker

This article describes the WebSphere Adapter for SAP in WebSphere Message Broker V7. Topics include the new SAPReply node for BAPI callout or synchronous RFC callout, iterative discovery and iterative deployment, high availability (HA) for SAPInput nodes, and multi-broker failover using a WebSphere MQ HA multi-instance queue manager.

Christopher Batey, Software Engineer, IBM

Photo of Christopher BateyChristopher Batey is a Software Engineer on the WebSphere Message Broker Development team at the IBM Hursley Software Lab in the UK. You can contact Christopher at christopher.batey@uk.ibm.com.



John Hosie (jhosie@uk.ibm.com), Software Developer, IBM

Photo of John HosieJohn Hosie is a software developer on the WebSphere Message Broker Development team at the IBM Hursley Software Lab in the UK. He was responsible for the development of the WebSphere Message Broker SAP Adapter synchronous RFC callout support with the SAPReply node. You can contact John at jhosie@uk.ibm.com.



Ernese Norelus (ernese@sg.ibm.com), Consulting IT Specialist, IBM

Photo of Ernese NorelusErnese Norelus is an IBM Certified Consulting IT Specialist and WebSphere Services Practice Leader with the IBM ASEAN Software Services team in Singapore. In this role, he consults with clients to help define IT solutions to complex business problems. He has more than 13 years of IT experience. You can contact Ernese at ernese@sg.ibm.com.



12 January 2011

Also available in Chinese

Introduction

Are you thinking about integrating your applications with SAP using IBM® WebSphere® Message Broker (referred to hereafter as Message Broker) and the WebSphere Adapter for SAP? The first thing you need to do is consider the SAP interfaces you are going to use. Are you going to be calling into SAP from Message Broker (inbound calls), or pushing data from SAP into Message Broker (outbound calls)? Will you be dealing with BAPI, ALE, QISS, or AEP? These decisions have probably already been made for you by the SAP implementation team. As middleware, Message Broker has been designed to fit in with SAP architecture.

Introduction to SAP for Message Broker users

If you don't understand some of the terms in that last paragraph, then you probably haven't had your discussion with the SAP team yet. The two main interfaces with SAP are Application Link Enabling (ALE) and Business Application Programming Interface (BAPI). Message Broker also supports Advanced Event Processing (AEP) and Query Interface for SAP Software (QISS), but they are less common, so the article will just touch on them and focus mainly on ALE (and IDoc) and BAPI.

This article is not a comprehensive introduction to SAP. It focuses only on those aspects of SAP that are relevant to integration with Message Broker, and does not cover other aspects of SAP.

Outbound calls from Message Broker to SAP

Outbound adapters allow data to be passed from Message Broker to the SAP system.

Outbound calls are supported with the BAPI, ALE, and AEP interfaces using Advanced Business Application Programming (ABAP) handlers and QISS. With BAPI, the calls can be simple BAPI calls; BAPI can use Remote Function Calls (RFCs) or multiple BAPI calls in a single interaction (referred to as a BAPI unit of work). BAPI outbound calls have a request-response interaction style. The ALE interface lets you pass single or multiple intermediate documents (IDocs). These outbound calls are one-way, with IDocs passed to the SAP application. With the AEP interface, the adapter uses the ABAP handlers, and with the QISS interface, you can directly query SAP application tables. Figure 1 shows how these calls work:

Figure 1. Outbound calls propagated from Message Broker to SAP
Outbound calls propogated from Message Broker to SAP

Inbound calls from SAP to Message Broker

An inbound adapter enables data to be passed from the SAP system to Message Broker. Inbound calls are supported by the ALE interface using asynchronous event notification. For the inbound call, the adapter acts as an RFC server and listens for ALE events from the SAP application. The adapter uses an event recovery table to manage the inbound events.

For ALE outbound and inbound operations, queued RFC (qRFC) support is enabled for the ALE interface in Message Broker V7. Client applications can specify the queue to which IDocs are delivered in order to ensure the order in which those IDocs are delivered and processed by an SAP application. The BAPI callout and synchronous RFC callout introduced in Message Broker V7 are used to monitor any events related to the invocation of a BAPI call. With the AEP interface, the adapter processes any events related to custom IDocs using the ABAP handlers, as shown in Figure 2:

Figure 2. Inbound calls from SAP to Message Broker
Inbound calls from SAP to Message Broker

BAPI

A BAPI is a standardized business API that enables vendor systems to interact with SAP systems. A BAPI is implemented as an RFC-enabled function, so the adapter's BAPI interface can support any such function. The BAPI interface APIs enable vendor systems to interact with SAP. Adapters provide local transaction support for the BAPI interface using the BAPI calls BAPI_TRANSACTION_COMMIT and BAPI_TRANSACTION_ROLLBACK. Create, Update, Delete, and Retrieve operations are all supported, except in the case of the BAPI result set, where RetrieveAll is the only supported operation.

BAPIs are business functions provided by SAP. Typically, BAPIs provide applications with access to the business objects stored in the SAP database without the applications needing to know the details of how the tables are implemented. However, you can define and implement custom BAPIs to provide other functions that may or may not access the SAP database. BAPIs are RFC-enabled so that they can be accessed from applications external to SAP, such as a Message Broker message flow. There are two parts to a BAPI: the definition, and the implementation.

Programs running inside SAP can invoke BAPIs that are implemented and that execute on that system, or that execute on an external RFC server. The external RFC server might be another SAP system or an external application, such as a Message Broker message flow. In short, BAPIs are defined in SAP. If they are also implemented in SAP, then you can invoke them from a message flow and execute them on the SAP system, potentially accessing data in the SAP database tables, as shown in Figure 3:

Figure 3. Outbound calls using BAPI interface
Ootbound calls using BAPI interface

Whether they are actually implemented in SAP or just defined there, they can be invoked by programs running on the SAP system. If they are not actually implemented in that SAP system, they can still be invoked from that SAP system across RFC to be executed in a Message Broker message flow. In that scenario, the message flow plays the role of an RFC server in the SAP model, as described later in this article.

BAPIs are defined in terms of their input parameters (imports) and output parameters (exports). BAPIs can be invoked synchronously or asynchronously. You will often hear the term transactional used in place of asynchronous when talking about SAP interactions. BAPIs are invoked over RFC, and you can use an SAP layer named tRFC to make these calls asynchronously and transitionally. In this sense, transactional means that the caller is given a promise that the call will be made at some point in the future, even if the system where the BAPI is implemented is not reachable when the call is made. This behavior should be familiar to anyone with experience with Message Broker or WebSphere MQ. As you might expect, when a BAPI is invoked asynchronously, the output parameters (exports) are never received by the caller, so this model really makes sense only for BAPIs that have only input parameters (imports).

IDocs

IDocs are like envelopes that are used to send data between SAP applications. The data within these envelopes can have different structures and can be sent for different reasons. The IDoc type defines the structure of the data. All IDocs have a control record and one or more data records (the latter are also referred to as segments). The control record has the same structure in all IDocs, while segment structure is defined by IDoc type.

IDocs of the same IDoc type can have different message types. While the IDoc type reflects the structure of the data, the message type reflects the business reason for sending the data. Furthermore, an IDoc can be sent with a message code and/or a message function. The control record includes Message Type, Message Function, and Message Code fields.

IDocs are transmitted via tRFC. In fact, you may hear people use the term tRFC interchangeably with IDoc. This is not strictly accurate because as described above, BAPIs can also be sent over tRFC.

When we talk about senders and receivers of IDocs, we talk about logical systems or partners. These entities controlled by the ALE layer, which is responsible for routing IDocs between logical systems. The configuration of ALE lets the SAP administrator determine which logical systems can send which message types to which other logical systems.

ALE

The ALE interface provides an asynchronous interface for sending batch data represented by IDocs to SAP applications. The adapter extracts the IDoc information from the message and then uses SAP Java™ Connector function calls to convert the message representing the IDoc object to a table format that is compatible with the IDoc format. The adapter then uses RFCs in the SAP RFC library to establish an RFC connection to the ALE interface, pass the IDoc data to the SAP system, and then release the connection to SAP. Because the ALE interface is asynchronous, SAP returns only a return code and a null object to the caller. If no exceptions are raised, the outbound transaction is considered successful.

The success of the transaction can be verified by inspecting the IDocs that have been generated in SAP. Only a return code (success or failure) is returned, and no return data is sent back to the adapter. For the ALE outbound call, all the information is in the IDoc and the only supported operation is Execute on the IDoc. Create, Update, and Delete operations do not make sense for ALE because it is a batch-oriented asynchronous interface. The receiving SAP application processes the data and performs appropriate operations based on the data contained in the control record child object of the IDoc, as shown in Figure 4:

Figure 4. Outbound operations using the ALE interface
Outbound operations using the ALE interface

AEP

The AEP interface of the WebSphere SAP adapter is used for both inbound and outbound processing. For inbound processing, it polls for events in SAP, converts them into business objects, and sends the event data as business objects to Message Broker. For outbound processing, the adapter processes events sent from an application to retrieve data from or update data in the SAP server.

Business objects for the AEP interface are defined in SAP as IDocs; either standard IDocs delivered with SAP or custom-built IDocs can be used. The External Service wizard is used to generate the business object definition based on an IDoc. Business object development for the ABAP extension module consists of creating an application-specific business object definition and an associated ABAP handler for each operation that you want to support. ABAP handlers reside in the SAP application as ABAP function modules, communicate with the connector, and are needed to get business object data into or out of the SAP application database. ABAP handlers are responsible for adding business object data into the SAP application database (Create, Update, or Delete operations) or for retrieving data from the SAP application database (Retrieve operations). You must develop operation-specific ABAP handlers for each supported hierarchical business object.

The application-specific business object and the ABAP handler rely on each other's consistency to pass data into and out of the SAP application. If you change the business object definition, you must also change the ABAP handler.

In outbound processing, the adapter receives the AEP record as a business object that contains the business data along with the metadata. Depending on the operation set, the adapter looks up the ABAP handler to be invoked from the operation application-specific information. The adapter converts the business data to handler data format and passes it along with the ABAP handler information to the SAP application. After the object-specific ABAP handler finishes processing the business object data, it returns the business object data in IDoc format to the ABAP component, which converts the business object data back to its original format and returns it to the adapter. The adapter component in the SAP application invokes the appropriate ABAP handler and returns the results back to the adapter, which converts the results to a business object and returns it to the caller. With the AEP interface, the adapter makes use of the ABAP handlers.

QISS

QISS provides you with the means to retrieve data from application tables on an SAP server or to query SAP application tables for the existence of data. The adapter can perform hierarchical data retrieval from the SAP application tables.

QISS supports outbound interactions for read operations only (RetrieveAll and Exists). You can use this interface in local transactions to look up records before write operations (Create, Update, or Delete). For example, you can use the interface as part of a local transaction to do an existence check on a customer before creating a sales order. You can also use the interface in non-transaction scenarios.

QISS supports data retrieval from SAP application tables, including hierarchical data retrieval from multiple tables. The interface supports static as well as dynamic specification of WHERE clauses for the queries.

The Adapter Connection wizard finds the application data tables in SAP, interprets the hierarchical relationship between tables, and constructs a representation of the tables and their relationship in the form of a business object. The wizard also builds a default WHERE clause for the query. You can control the depth of the data retrieval, as well as the amount of information, by using the maxRow and rowsSkip properties. All of these outbound QISS operations are illustrated in Figure 5:

Figure 5. Outbound operations supported by QISS
Outbound operations supported by QISS

SAP-specific enhancements in Message Broker V7

Message Broker V7 includes enhancements in SAP support and operations. The new SAPReply node provides a new way to interact with SAP, by using a message flow to implement a synchronous call out from SAP via the BAPI interface. Other enhancements make the administration model more flexible to allow for extension, reconfiguration, and horizontal scaling. New features include:

  • A single RFC program ID now allows multiple different kinds of IDocs to be handled by different flows without having to redeploy or rediscover.
  • In WebSphere Adapter for SAP Software V7, inbound synchronous processing of RFC-enabled functions is known as the BAPI callout or synchronous RFC callout. A BAPI callout or synchronous RFC callout supports the SAPReply node, which can therefore be used to send a reply to an SAP synchronous callout.
  • High availability (HA) for SAP input nodes allows multi-broker failover using a WebSphere MQ HA multi-instance queue manager as the TID store.
  • By using the Wait parameter before calling the BAPI Commit parameter, you can control whether the SAP Adapter waits for SAP to commit updates synchronously, or issues a commit and then returns while the SAP commit occurs asynchronously.
  • Several SAP-specific features have been added, including the ability to specify a single program ID (enabling multiple IDocs to be handled by different message flows without disruption), support for SCI using a new SAPReply node, and BAPI commit-wait processing.
  • Three adapter nodes are provided with Message Broker: the SAPInput node, the SAPReply node, and the SAPRequest node. The SAPReply node is the newest addition, proving BAPI callout or synchronous RFC callout. Figure 6 shows how these nodes are represented in the Message Broker GUI:
    Figure 6. SAP nodes in Message Broker
    SAP nodes in Message Broker

Let's take a look at some of these features in more detail.

SAPReply node

You use the SAPReply node on an SAP program to call functions on remote RFC servers. These functions can be called asynchronously or synchronously; in the latter case, they must provide a reply. A message flow can act as an RFC destination and receive the function call through the SAPInput node. When the function is called synchronously, you use the SAPReply node to send a reply back to an SAP synchronous callout.

SAP BAPI callout or synchronous RFC callout

Message flows can implement both one-way and synchronous BAPIs that can be called from an SAP system. One-way operations are implemented with an SAPInput node, which can represent one or more BAPIs; synchronous operations are implemented with the combination of an SAPInput node and an SAPReply node, which can reside on different message flows. Synchronous and asynchronous calls are illustrated in Figures 7 and 8, respectively:

Figure 7. Synchronous call using SAPReply node
Synchronous call using SAPReply node
Figure 8. Asynchronous call using SAPReply node
Asynchronous call using SAPReply node

Iterative discovery and iterative deployment

Message Broker V7 introduces two new features for using WebSphere Adapters: iterative discovery and iterative deployment. Before V7, to use an adapter with Message Broker, you needed to:

  1. Use the Adapter Connection Wizard to discover objects or operations from your SAP system and create your adapter component.
  2. Create a message flow with at least one adapter node.
  3. Create a BAR file and add your message flow, adapter component, and message set to it.
  4. Deploy your BAR file.

If at a later date you wanted to support additional objects or operations from your SAP system, you needed to repeat these steps, rerunning the Adapter Connection Wizard to create a new adapter component and message set. This procedure created problems, because previously discovered objects and operations had to be discovered again, even if you just added a single operation or object. In addition, previously deployed message flows, adapters, and message sets had to be redeployed -- which meant that they also had to be retested.

Iterative discovery and iterative deployment address these problems. Iterative discovery lets you add objects and operations to an existing adapter and message set, removing the need to completely replace your resources. Iterative deployment lets a single adapter node use multiple adapter components and message sets, which means that the Adapter Connection wizard can discover new objects and operations, the adapter component and message set can be deployed, and existing adapter nodes can start using the newly deployed adapters and message sets -- all without a restart. Only the new operations and objects need to be tested, and existing resources are left untouched.

Figures 9 and 10 illustrate these new features. These screenshots are taken from the Message Broker Toolkit under the Iterative discovery feature:

Figure 9. Iterative discovery
Iterative discovery
Figure 10. Configuring settings for the discovery agent
Configuring settings for the discovery agent

There are a number of scenarios in which iterative discovery and deployment can make your life easier:

A single message flow acting as a generic gateway to SAP
If a message flow is agnostic about the data being transferred to and from an SAP system, then nodes in that message flow are not sensitive to the kinds of objects or operations it receives or sends to the SAP system. However, the flow still needs a message set to serialize the message. So, to support more objects or operations, you need to discover them from your SAP system. Iterative deployment can be used to support additional objects or operations without re-deploying any of the adapter components or message sets, as you can deploy additional adapter components and message sets as and when you want to support them.
Switching or migrating between two different SAP systems
Iterative deployment allows secondary adapters and message sets for adapter nodes that contain new objects or operations. The connection details for the SAP system are held on the primary adapter, which is explicitly referenced by the node. It is possible to put all of your objects and operations in secondary adapters and message sets, so that when you want to switch between SAP systems, you can slowly replace the primary adapters of each of your adapter nodes, enabling BAPIs to move from one SAP system to another.
A single SAP system, with many flows calling different BAPIs
If you have multiple flows, all of which do different things but use the same SAP system, you will probably have the connection information for that SAP system duplicated across multiple adapters. If any of the details change, then your message broker administrator must change many adapters or many configurable services. But with iterative deployment, all of the adapter nodes can use a single primary adapter that contains the connection information, with their objects and operations in secondary adapters, minimizing the administrative overhead for changing connection details.

With iterative deployment, each enabled adapter node has access to additional message sets and to the types within these message sets. It is therefore important to ensure that the types do not clash, which you can easily do by using namespaces. When objects or operations share types, use a different namespace for your message set. If in the third scenario above, you have different flows doing different things, and these flows are developed by different developers, some communication is required to ensure that different namespaces are used.

High availability

Message Broker V7 adds support for making message flows that include SAPInput nodes highly available when using assured delivery. Previously, assured delivery relied on a transaction store kept on the broker's queue manager, in order to keep track of incoming requests in case a broker crashed. But having the same adapter deployed to two different execution groups or brokers can be a problem: if one of the brokers crashes, SAP will try to continue any conversations with the other broker, as it has the same RFC program ID. The other broker will have no record of this conversation in its transaction store, and so it will be unable to continue any conversations started by the crashed broker.

In Message Broker V7, this transaction store can be put on any queue manager and shared among execution groups and brokers on different machines. Therefore an active-active scenario can be set up; if one broker goes down, the other can continue its work by looking at the state in the transaction store, as shown in Figure 11:

Figure 11. High availability for SAP input nodes
High availability for SAP input nodes works

IDoc routing

A common requirement in building systems with SAP is to take IDocs from SAP and route them to different applications to process. Imagine that you are integrating SAP with another system that does not understand the physical format of the data that SAP sends out. The data is sent from SAP to the other system via ALE and tRFC layers in the form of an IDoc, with many different types of IDocs being sent. The SAP administrator has assigned you a single RFC program ID to receive these IDocs -- in other words, SAP sees your side of the interface as a single system. In addition, your system has at least one of the following requirements:

  • The solution must be maintainable: The components and artifacts used to process different types of IDocs need to be modular in nature and isolated from each other.
  • The solution must be extensible: New IDoc types may be added in the future and should not have any effect on the existing applications.
  • The solution must be scalable: The processing of the IDocs must be distributable across many physical processors.

Prior to Message Broker V7, you could meet these requirements by using the IDoc pass-though mechanism, which let you receive IDocs from SAP without fully parsing them. The Message Broker tree would contain only a few control fields, such as the IDoc name and a hexBinary field containing the IDoc. You could use the IDoc name or a different field to route the IDoc to another application (which could be another message flow). If you wanted to route IDocs to different message flows, then you were left with the problem of parsing the IDoc, as you now had the IDoc on a queue or some other transport in binary form. You could achieve this result by exporting a C header file from SAP and importing it into Message Broker and using the MRM parser to parse the IDoc, but in doing so you would lose all the benefits of the Adapter Connection wizard and the message sets that it generates.

Message Broker V7 lets you run the Adapter Connection Wizard and generate a message set, and then associate it with an MQInput node, along with the DataObject domain. You can then parse the binary IDoc that has been routed using the IDoc passthrough mode. An integration pattern has been created to use this new function and solve the IDoc routing problem. The solution has two parts:

  • A connection to SAP that receives the unparsed IDoc. It needs to be a generic component that is not sensitive to the exact type of IDoc being received, and it is implemented using the passthrough mode described above. This function already existed and is represented in Figure 12 as the flow on the left.
  • The parsing and transformation of the IDoc, which is distributed across a number of components (one per IDoc). It can be a message flow that uses the new function described above or another application altogether. Figure 12 shows how message flows can be used, one per IDoc, to parse and process the IDocs:
    Figure 12. Parsing and processing IDocs
    Diagram showing how IDocs are parsed and processed

Conclusion

This article described the important features of the SAP Adapter for Message Broker V7. You should now be able to discuss the interactions possible between Message Broker and SAP with the SAP team in your organization, and be able to plan an implementation of the Message Broker components needed for those interactions. Topics included:

  • Outbound and inbound interfaces using WebSphere Adapter for SAP with Message Broker
  • The two main interfaces with SAP: ALEs and BAPIs
  • Synchronous and asynchronous calls using the SAPReply node
  • Iterative discovery and iterative deployment
  • High availability for SAPInput nodes
  • IDoc routing

Resources

  • IBM WebSphere Message Broker resources
  • IBM WebSphere resources
    • developerWorks WebSphere developer resources
      Technical information and resources for developers who use WebSphere products. developerWorks WebSphere provides product downloads, how-to information, support resources, and a free technical library of more than 2000 technical articles, tutorials, best practices, IBM Redbooks, and online product manuals. Whether you're a beginner, an expert, or somewhere in between, you'll find what you need to build enterprise-scale SOA solutions using the open-standards-based WebSphere software platform.
    • developerWorks WebSphere application connectivity developer resources
      How-to articles, downloads, tutorials, education, product info, and other resources to help you build WebSphere application connectivity and business integration solutions.
    • developerWorks WebSphere SOA and Web services developer resources
      How-to articles, downloads, tutorials, education, product info, and other resources to help you design and build WebSphere SOA and Web services solutions.
    • Most popular WebSphere trial downloads
      No-charge trial downloads for key WebSphere products.
    • WebSphere forums
      Product-specific forums where you can get answers to your technical questions and share your expertise with other WebSphere users.
    • WebSphere on-demand demos
      Download, watch, and learn what WebSphere products and WebSphere-related technologies can do for your company.
    • developerWorks WebSphere weekly newsletter
      The developerWorks newsletter gives you the latest articles and information only on those topics that interest you. In addition to WebSphere, you can select from Java, Linux, Open source, Rational, SOA, Web services, and other topics. Subscribe now and design your custom mailing.
    • WebSphere-related books from IBM Press
      Convenient online ordering through Barnes & Noble.
    • WebSphere-related events
      Conferences, trade shows, Webcasts, and other events around the world of interest to WebSphere developers.
  • IBM developerWorks resources
    • developerWorks blogs
      Join a conversation with developerWorks users and authors, and IBM editors and developers.
    • developerWorks Webcasts
      Free technical sessions by IBM experts that can accelerate your learning curve and help you succeed in your most difficult software projects. Sessions range from one-hour Webcasts to half-day and full-day live sessions in cities worldwide.
    • developerWorks podcasts
      Listen to interesting and offbeat interviews and discussions with software innovators.
    • IBM Education Assistant
      A collection of multimedia educational modules that will help you better understand IBM software products and use them more effectively to meet your business requirements.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=607881
ArticleTitle=Integrate with SAP using WebSphere Message Broker
publish-date=01122011