Life just got a little easier for IT architects and business analysts who want to use web services to integrate business processes between their enterprise and their partners. In the first article of this series, I suggested that "as the industry further develops specifications that determine how to coordinate flows between web services and how to describe dependencies between web services that realize business processes, support for asynchronous operations will be simplified." Well, guess what just happened? On August 9, IBM, Microsoft, and BEA announced a set of jointly developed specifications for adding business semantics to web services. Three specifications were released to the industry for further development, along with itemized plans for addressing additional capabilities with the intent to jump-start the standardization process required to address a critical set of customer process and transactional requirements. The specifications are Business Process Execution Language for Web Services (BPEL4WS or BPEL), Web Services Coordination (WS-Coordination or WS-C), and Web Services Transaction (WS-Transaction or WS-T). These three specifications truly enable enterprises to describe their business processes using a comprehensive model and provide an implementation framework for executing that model and for coordinating business activities and transactional behavior. This article will outline how the BPEL, WS-C, and WS-T specifications simplify support for asynchronous operations. More importantly, you'll also learn how these specifications can describe and implement enterprise business interactions that typically assume sequences of peer-to-peer message exchanges within stateful long-running interactions involving two or more parties.
In previous articles in this series, I introduced the concepts of asynchronous web services operations, along with various integration patterns for implementing individual asynchronous operations between two parties. These patterns were essentially modeled as stateless independent interactions. Within the patterns, I outlined requirements for the participating applications or the messaging transports to create and manage correlators in mapping requests to responses. Likewise, the solution designers need to provide a means by which the reply-to address to which responses are to be delivered are provided to the service provider; that way, when responses are available, they can be sent to the requesting clients on separate threads of execution. All of these requirements, plus much, much more significant ones (such as the coordination of a set of stateful interactions and routing of requests to stateful process instances), are being addressed in the new specifications. As a result, complex, real-world enterprise processes can be modeled and directly executed within a business process engine (more on which momentarily).
Today, version 1.1 of the Web Services Description Language (WSDL) specification by itself, with no extensions, only supports an interaction model that is stateless, with synchronous or noncorrelated asynchronous message exchanges between two parties. The release of the three new specifications makes possible support for enterprise business processes that require an interaction model involving sequences of peer-to-peer message exchanges, synchronous and asynchronous, within stateful, long-running interactions between two or more parties.
Before you go any further into this article, you should have a basic understanding of the BPEL, WS-C, and WS-T specifications, as many of the concepts from these specifications will be explored here. You can find links to these specifications, along with previous articles in this series and other useful data, in the Resources section below.
Asynchronous operations requirements: A review
Supporting real-world enterprise business processes inherently involves asynchronous operations, as the processes are long in duration. Individual activities of each process need to be decoupled from the initial request in order to optimize the use of system resources and to break the processing into a recoverable set of transactions. As outlined in the first article of this series, several tasks need to be addressed in supporting asynchronous operations:
- Defining a correlator or set of correlators and a mechanism for their exchange.
- Defining a reply-to address specifying where the response should be sent, and ensuring that the service provider is informed of this destination.
- The generation of a response by a service provider as a transaction separate from the request.
- The receipt of an asynchronous response by the client.
- The correlation of a response with its request by both the client and service provider.
In addition, since I'll be discussing asynchronous operations within the scope of a larger stateful business transaction potentially involving many interactions between the process and its partners, the messages exchanged during asynchronous operations will need to be routed to the same business process instance. Thus, I need to include one more requirement that was not previously itemized in the initial article of this series:
- The routing of requests to stateful business process instances.
Elements of the BPEL XML grammar
This article will outline how BPEL simplifies the integration of business processes between partners for long-running business transactions, where a business transaction consists of multiple interactions between the process and its partners. In the following sections, I'll be providing specific examples of the BPEL XML grammar that can be used to address the requirements identified above for asynchronous operations.
BPEL XML flow language has grammar for describing activities and mechanisms for controlling process behavior. I'll be using a subset of the grammar in my examples, which includes the following activity tokens and elements:
- Basic activities: For handling reception of inbound requests and responses, data assignments into global containers, and the generation of outbound web service requests. Examples of these activities that will be illustrated in this article include
- Structured activities: For managing the overall process flow for sequencing of activities, parallel processing of activities, and adding conditional logic in controlling the process flow. Examples of these activities that will be illustrated in this article include
- Other BPEL XML elements help define relationships and dependencies required to support asynchronous operations in business transactions. Examples of these elements that will be illustrated in this article include
For a detailed description of BPEL XML grammar, review the BPEL specification (see Resources).
What does it mean to apply business semantics to web services?
Applying business semantics to web services is really about leveraging web services in a way that allows an enterprise to replicate in an operational programming environment the roles and relationships that the enterprise has with its partners in real life. In this environment, manual tasks can be automated by integrating business applications from different vendors used by the enterprise's partners. Mirroring real-world relationships and dependencies of participants in a business interaction to web service clients and service providers is fundamental to exploiting the value proposition of dynamic e-business. When real-world processes are initiated, a set of tasks are started, with some tasks being done in parallel and others needing to be executed in sequence. During the process, other business partners may need to be involved, with their input being incorporated into the business logic in determining the overall outcome of the process. This type of scenario is very typical and can easily be described and implemented using BPEL.
Figure 1. Scope of a business transaction
A model of a business process described using the BPEL XML grammar can be interpreted by a business process engine and executed. The business process engine is usually part of a Web application server's runtime; it manages the flow and control of a process that is being executed. Each step of the process will be represented by an activity; the activity is implemented as either an outbound web service invocation on an external partner service or as the processing of an inbound web service request. The execution of the process model by a business process engine creates an operational instance of the process with its web service interfaces. The process interfaces are used for initiating the process itself or for controlling aspects of the process (for example, wait, terminate, compensate). The web service operations exposed by the process can aggregate other web services in performing the business tasks associated with the process. Thus, the flow of the process will include a set of web service interactions between the process itself, internal web services, and external partner web services.
Long-running business transactions require asynchronous web service support
Enterprise business processes operating across a distributed environment are generally long in duration, due to several factors:
- Interactions with partners are slowed by Internet latency.
- Batch processing is commonly offloaded to legacy backend systems.
- General data and transport conversions are often required for B2B solutions.
As a result of the processing duration, the web service requests and responses associated with the interactions will rely on asynchronous operations to decouple the two events.
BPEL supports both synchronous and asynchronous operations. For the former, the response to a request can be defined as being returned on the same thread of execution using the
<reply> activity token. However, for long-running business processes where many activities may be run in parallel with nonblocking threads for efficiency, BPEL enables the process to be described so that the responses are generated on a separate threads of execution using the
<invoke> activity token.
Coordination of process activities
In addition to mapping roles and responsibilities of participating partners into the business process description, the operational characteristics of activity coordination and transactional support required in a distributed environment need to be applied to the process implementation using a common set of infrastructure services, as outlined by the WS-C and WS-T specifications. However, the use of the coordination and transaction facilities will be managed by the business process engine based on the structure of the BPEL process description. Thus, the process description itself will not explicitly utilize the coordination and transaction services outlined within the specifications.
The facilities outlined by the WS-C and WS-T specifications will enable participants' business process engines to register for coordination protocols that ensure a reliable, mutually agreed upon outcome for the business processes modeled. Because of this, I won't address the specific facilities to provide coordination and transactional support for business processes in this article, as the implementation and administration of such facilities will be dependent on specific business process engines.
Currently, BPEL supports the Business Activity coordination type, which allows for coordination of long-lived activities and the use of business logic to handle business exceptions using compensation-based recovery. This level of support is critical when interoperability across different vendor implementations is required and when a simple process abort is not appropriate for a business exception. Typically, when an activity is long running and the operations that are performed by the activity need to be recoverable, the
scope activity token is used to encompass the activity so that, if necessary, compensation can be applied to reverse the results of the operations within the activity using the compensation handlers mechanisms supported in BPEL. This methodology applies for both synchronous and asynchronous operations. Business process engines are expected to utilize asynchronous operations when implementing the facilities associated with WS-C and WS-T to support the process behaviors that can be described with BPEL.
For a detailed description of the Business Activity coordination protocols, review the WS-T specification (see Resources).
Modeling asynchronous interactions between business partners
As part of its XML grammar, the BPEL specification includes mechanisms for defining activities, partners, peer-to-peer roles, message exchange protocols, synchronization dependencies of activities, message properties for identifying correlators, and data containers for sharing business protocol data and other context data during the execution of the business process. BPEL enables an enterprise to model behaviors between the process itself and participating partners for each of the business interactions that make up the flow of the process. Interactions are considered peer-to-peer message exchanges (that is, a request with an explicit response, a request with no response, or a single one-way notification). The behaviors are characterized for interactions based on roles assigned to each partner, with the messages sent and received for each role identified based on operations of
portTypes associated with each role. In some cases, only a single message may be sent (that is, a request without a response, or a notification).
You may find the description in the last paragraph a little overwhelming at this stage, so I'll review some concrete examples of how BPEL can be used to address the six requirements I've outlined above for the support of asynchronous operations.
Defining the callback mechanism
A peer-to-peer message exchange between a process and a given partner (that is, an individual web service) is considered an interaction between the two parties that is based on their relationship. The relationship between the two parties is modeled as a
serviceLinkType, where up to two roles are defined. Each role has at least one
portType that it must support for the interaction to be successful.
Listing 1. serviceLinkType example
<serviceLinkType name="BuyerSellerLink" xmlns="http://schemas.xmlsoap.org/ws/2002/07/service-link/"> <role name="ServiceProvider"> <portType name="Serviceprovider:ServicePortType"/> </role> <role name="Client"> <portType name="Client:CallbackmechanismPortType"/> </role> </serviceLinkType>
When an asynchronous response is expected as part of the interaction, two roles will be identified for the
serviceLinkType, as shown in Listing 1 above: one for the service provider, identifying the
portType of the service being offered, and one for the requesting client, to identify the
portType for a callback mechanism service that will handle the asynchronous response associated with the request.
The web services that a business process utilizes are modeled as
partners; thus, both internal services and external business partner services apply.
partners define the participants of an interaction, where the role of each participant is identified based on the roles defined for the
serviceLinkType that models the relationship.
Listing 2. partners example
<partners> <partner name="RequestingClient" serviceLinkType="BuyerSellerLink" myRole="ServiceProvider"? partnerRole="Client"?>+ </partner> </partners>
These mechanisms enable an IT architect and business analyst to address asynchronous requirements 2 and 4 in my list at the beginning of this article.
In most cases, a single service will be used by each of the participating partners for the callback mechanism; thus, the use of a correlator will need to be defined as part of the business process definition. Typically, business protocol information such as customer ID, order number, vendor ID, and invoice number will be used to correlate requests with responses; at times, it's necessary to combine individual correlators when several interactions need to be associated with one another. The initial interaction may use a combination of customer ID and order number correlators, while the response and follow-up interactions may include an invoice number correlator as well.
Since business protocol data is used in real-world interactions for correlation of business tasks, it's natural to use such data to correlate web service requests and responses. To use business protocol data as correlators, you need to use the process definition to define message properties as named data types, thus allowing you to assign a more relevant meaning to an existing type (for example, defining an
OrderNumber property as an
integer type or defining a
VendorID property as a
String type). Once a property has been defined, you can specify an
propertyAlias to identify where in the application message the specific business protocol data is located based on the structure of the message (for example, identifying WSDL
<part> elements) and an XPATH query string. In essence, this defines fields within the message where the values of the correlators can be found.
Listing 3. property and propertyAlias example
targetNamespace="http://example.com/supplyCorrelation.wsdl" xmlns:tns="http://example.com/supplyMessages.wsdl" xmlns:bpws=http://schemas.xmlsoap.org/ws/2002/07/business-process/ <!-- define correlation properties --> <bpws:property name="customerID" type="xsd:string"/> <bpws:property name="orderNumber" type="xsd:int"/> <bpws:property name="vendorID" type="xsd:string"/> <bpws:property name="invoiceNumber" type="xsd:int"/> <bpws:propertyAlias propertyName="customerID" messageType="tns:POMessage" part="PO" query="/CID"/> <bpws:propertyAlias propertyName="orderNumber" messageType="tns:POMessage" part="PO" query="/Order"/> <bpws:propertyAlias propertyName="vendorID" messageType="tns:InvMessage" part="IVC" query="/VID"/> <bpws:propertyAlias propertyName="invoiceNumber" messageType="tns:InvMessage" part="IVC" query="/InvNum"/>
It goes without saying that the fields that are being used to locate correlators must be defined within each abstract WSDL message definition as formal message parts. Likewise, the same message parts must be defined in both the request message and the response message, and the operations of the
portTypes associated with the roles of the
serviceLinkType must use the message types for which the
propertyAlias has been defined.
When relying on more than one piece of business protocol data to correlate requests with responses, BPEL enables you to group message properties into
correlationSets and use the named
correlationSets to identify application-level conversations within a process instance.
Listing 4. correlationSet example
<correlationSets xmlns:cor="http://example.com/supplyCorrelation.wsdl"> <!-- Order numbers are particular to a customer, this set is carried in application data --> <correlationSet name="PurchaseOrder" properties="cor:customerID cor:orderNumber"/> <!-- Invoice numbers are particular to a vendor, this set is carried in application data --> <correlationSet name="Invoice" properties="cor:vendorID cor:customerID cor:orderNumber"/> </correlationSets>
These mechanisms enable an IT architect and business analyst to address asynchronous requirements 1, 5, and 6 in the list above.
Synchronizing response messages
When a service provider -- in this case, the business process engine that is executing the BPEL-described business process -- receives a request (for example, one interaction in a multistep business transaction), one or more functions need to be executed in performing the business operation that is exposed as a web service by the business process. Upon completion of the processing, a response containing either the results of the business operation or an acknowledgment of the completion of the operation is sent to the requesting client. Assuming that the operation performed is part of a long-running business transaction, the response for this interaction will be provided asynchronously on a separate thread of execution.
Managing the reception of the request, routing the request to the appropriate process instance, executing process logic to perform one or more tasks, ensuring that flow dependencies are met and the response is generated once all processing is complete -- all this can be easily described by BPEL and handled by a compliant business process engine without IT programming staff having to develop complex synchronization flow-control logic or manage process state data.
Using correlators, BPEL enables a business process engine to create new instances of a process for handling inbound requests or to route inbound requests to existing process instances based on the values of message properties within the requests. Within a process flow in which multiple activities can be executed in parallel to optimize performance, follow-on dependent activities need to be synchronized. To identify synchronization dependencies, BPEL uses
links within activity definitions to control synchronization between activities. The outcome of an activity that is needed as an input for another activity (or, alternately, an activity that must simply be performed prior to another activity) will identify itself as a
source to a link, while the dependent activity will identify itself as a
target. With this information, a business process engine can manage the interdependencies of the process flow to control the overall outcome, thus ensuring a consistent behavior for the generation of responses communicated to the requesting clients.
Listing 5. link, source, and target example
<flow> <!--Three synchronization links are defined --> <links> <link name="ReceiveRequest"/> <link name="XtoY"/> <link name="CtoD"/> </links> <receive partner="RequestingClient"> <source linkName="ReceiveRequest"/> </receive> <sequence name="X"> <target linkName="ReceiveRequest"/> <source linkName="XtoY"/> <invoke name="A" .../> <invoke name="B" .../> </sequence> <!--Sequence Y is dependent on Sequence X completing --> <sequence name"Y"> <target linkName="XtoY"/> <receive name="C"/> <source linkName="CtoD"/> </receive> <invoke name="E" .../> </sequence> <!--Invoke partner Requesting Client is dependent on Sequence X and Sequence Y completing --> <sequence name"D"> <target linkName="CtoD"/> <invoke partner="RequestingClient"/> <invoke name="F" .../> </sequence> </flow>
Since the response is being sent on a separate thread of execution, the
invoke activity token is used in the process description to invoke the callback mechanism service of the client. To summarize, the response is being sent to a callback mechanism service as the result of the
invoke activity utilizing the named partner of
RequestingClient that was defined as a participant (
partnerRole set to
Client) in a
BuyerSellerLink relationship (
serviceLinkType), where the
Client role specifies support for the
These mechanisms enable an IT architect and business analyst to address asynchronous requirements 3 and 5.
Realizing the potential of BPEL
The development of the Business Process Execution Language for Web Services, Web Services Coordination, and Web Services Transaction specifications is a critical step toward enabling enterprises to leverage web services in modeling their real-world processes. With BPEL, you can describe a long-running, multistep process of a business transaction using activities to encompass atomic transactions for individual process steps.
Once a process is described using the BPEL XML flow language, a business process engine must interpret the process description in order to execute and manage the business transaction. The business process engine will provide the infrastructure services and resources for enabling long-running stateful business transactions that require interactions between the process, internal services, and external partner services (which themselves could be individual services exposed by the partners' process flows).
However, at this time, before enterprise processes can be fully realized in an operational programming environment, IBM and other vendors will need to update their business process engines to interpret the BPEL XML flow language and to support the coordination and transactional facilities outlined in the WS-C and WS-T specifications. Likewise, these three critical specifications will need to be moved into a standards organization and approved as open standards. Currently, IBM, Microsoft, and BEA are working towards that goal.
Once business process engines generally support BPEL, WS-C, and WS-T, the IT staff of any enterprise that seeks to automate steps in its business processes by leveraging web services technology and the new specifications will need to:
- Describe the enterprise's processes using the BPEL XML flow language.
- Provide the business logic for the activities.
- Provide the web services interfaces for legacy applications (for example, transaction resource managers) that will be utilized in the process flow activities for realizing the business transaction.
- Inform the business process engine how to locate the web services utilized within the process flow activities.
Before too long, enterprises will be taking advantage of their existing business assets, integrating their mission-critical processes with those of their partners, and executing long-running interruptible process flows with mutually agreed upon outcomes that mirror today's real-life experiences.
- Read the previous articles in Holt Adams' series, "Asynchronous operations and web services," on IBM developerWorks:
- "A primer on asynchronous transactions" (April 2002)
- "Programming patterns to build asynchronous web services" (June 2002)
- Check out the following specifications, hosted by developerWorks:
- Download IBM Business Process Execution Language for Web Services Java Runtime (BPWS4J), a business process engine available from IBM alphaWorks.
- Read version 1.1 of the Web Service Description Language (WSDL) specification, available on the W3C's Web site.
- See the product page for IBM WebSphere Application Server.
- See the product page for IBM WebSphere Studio Application Developer.