GPASS: A generalized publish and subscribe solution using WS-Notification standards, Part 1 : An introduction

Generalized Publish and Subscribe Services (GPASS), built on the implementation of Web Services-Notification in WebSphere's® Application Server V6.1, extends ESB connectivity to simplify data synchronization, thus helping you dramatically cut the cost of information distribution in your enterprise. Part 1 of this series describes GPASS architecture and implementation and highlights SOA case studies from IBM that use the GPASS framework.

Akram Bou-Ghannam, Ph.D., Senior Certified IT Architect, IBM

Akram Bou-Ghannam photoDr. Bou-Ghannam is an Executive IT Architect (Senior Certified) leading efforts to transform IBM into a globally integrated on-demand enterprise through SOA and dynamically reconfigurable services and architectures. His expertise in complex event processing and intelligent agents is helping IBM to realize the smart and agile enterprise – an enterprise that is capable of real-time discovery and response to actionable business situations. His SOA enterprise information architecture extends SOA into the data layer through the provision of information services, and helps achieve the enterprise vision of becoming information-centric in an SOA world. Dr. Bou-Ghannam is an IBM Master Inventor, and an accomplished author with multiple internal and external publications to his name.

You can reach Dr. Bou-Ghannam at akram@us.ibm.com.



Matt Roberts (matt.roberts@uk.ibm.com), WS-Notification Architect and Development Lead, IBM

Matt RobertsMatt Roberts is the architect and development team lead for support of WS-Notification in WebSphere Application Server. He is an active member of the OASIS WS-Notification technical committee responsible for producing the WS-Notification specifications, and works both inside and outside IBM to promote adoption of WS-Notification. Matt has worked on the Service Integration Bus development team for the past five years, with responsibility for design and implementation of a range of components including the Publish/Subscribe Bridge (for pub/sub interoperability between SIB and WebSphere Businesss Integration Message Broker), Java™ Message Service (JMS), and topology and routing management. Before joining the SIB team, Matt worked in the WebSphere MQ JMS development team. You can contact Matt at matt.roberts@uk.ibm.com.



22 August 2007

Introduction

Information is the lifeblood of an enterprise – it must be distributed and synchronized across multiple line-of-business (hereafter called LOB) systems in order to run the business of the enterprise. Enterprises implement many information distribution scenarios, including master data updates to LOB systems, and are constantly faced with implementing costly new requirements for information distribution. The Generalized Publish and Subscribe Services (hereafter called GPASS) framework offers a solution that extends an enterprise service bus (hereafter called ESB) engine with features that greatly simplify data synchronization, and helps to dramatically cut the cost of information distribution in an enterprise, enabling enterprises to implement new business requirements quicker and cheaper. GPASS is an adaptive policy-driven notification framework that helps enterprises meet the flexibility and responsiveness requirements of the enterprise. With GPASS information consumers can dynamically and declaratively create and configure entities on their behalf to manage their distribution requirements.

GPASS enhances ESB connectivity by providing functions that support a "generalized" publish/subscribe interaction pattern. GPASS serves as the foundation for common patterns to help developers leverage ESB more easily. The GPASS implementation uses and extends Web Services-Notification (hereafter called WSN) standards, and is built on the WSN implementation in WebSphere® Application Server V6.1.

LOB applications are critical computer applications that are vital to running an enterprise, such as accounting, Enterprise Resource Planning (ERP), Customer Relationship Management (CRM), and Supply Chain Management (SCM). LOB applications are usually large programs that contain a number of integrated capabilities and tie into databases and database management systems. Increasingly, LOB applications are being connected and integrated. Because LOB applications often use much of the same data, they end up duplicating the data. Thus when one of these applications updates the data, it must propagate the updates to the others so that the duplicated data remains synchronized across all of them.

You can use an ESB to integrate systems, but today's ESB engines provide only minimal out-of-the-box support for this kind of data synchronization. The GPASS framework extends an ESB engine with features that greatly simplify data synchronization. The GPASS framework has been used repeatedly and successfully in IBM's internal IT systems, and can be readily applied to the IT systems of customers who need this kind of data synchronization.

This article describes real-world scenarios that highlight the need for functions that support a generalized publish and subscribe distribution pattern as part of the ESB in the enterprise SOA architecture. This generalized publish and subscribe pattern is necessary to accommodate enterprise distribution scenarios, including the distribution of master data updates to LOB systems. For such scenarios, an adaptive policy-driven notification framework can meet the flexibility and responsiveness needs of the enterprise. With this type of framework, information consumers can dynamically and declaratively create and configure entities on their behalf to manage their distribution requirements. This paper describes such a framework based on the WSN standards.

Part 1 of this series explains the following aspects of the GPASS framework:

  • The problems the GPASS framework solves
  • The architecture GPASS uses to solve these problems
  • How to extend WSN to add the GPASS feature

After reading this article, you'll understand why GPASS is useful, how it works, and how you can implement it.

Part 2 of the series provides an overview of the WSN specification and its implementation in WebSphere Application Server V6.1, and describes how it has been used to solve the SAP Ledger scenario introduced in Part 1.

The problem

Information consumers, such as LOB systems performing fulfillment, and other enterprise business applications, need information and updates from information producers like enterprise master data systems. Consumers need to be able to subscribe to information updates, and to receive selected business information when changes occur. In real-world scenarios, enterprise information consumers and producers are typically legacy applications with different information models and different interfaces, and thus have distribution requirements beyond simple pub/sub distribution, or notification brokering, requirements. These additional distribution requirements include specifying the output message format (or transformation of the message before reaching the consumer), the output message size, the delivery schedule, and so on. What is needed is a solution that extends basic pub/sub pattern into a generalized pattern that satisfies these additional requirements, and captures the generalized pub/sub integration pattern and packages it as reusable integration services, such as a notification broker service, a transformation broker service, a distribution service, and a scheduling service. These services can then be reused in other scenarios that integrate information consumers with information producers to help accelerate the development of business initiatives in the enterprise.

A framework is needed to handle the real-world information integration scenario by allowing a subscriber to specify on behalf of an information consumer filtering rules and policy constraints, not only to select what types of messages or content the subscriber wants the consumer to receive, but also to specify transformation, scheduling, distribution, and other constraints to be applied to selected messages before they reach the consumer. The architecture must enable the generation (on behalf of a consumer) of a proxy service, or agent, that includes the engines to enforce and manage these constraints at run-time. The proxy service can receive the messages on behalf of the information consumer and apply the specified constraints to the message before delivering it to its consumer.

The GPASS solution

The GPASS framework satisfies all of these requirements, and provides an adaptive, policy-driven notification framework to implement the real-world distribution scenarios. Figure 1 illustrates the concepts of GPASS:

Figure 1. GPASS – an adaptive policy-driven notification framework
Figure 1. GPASS – an adaptive policy-driven notification framework

Using GPASS a consumer can dynamically create and configure a "smart" subscription (the Information Consumer Proxy Service in Figure 1) on its behalf to satisfy its information integration policies, or constraints. For example, a subscriber, on behalf of the consumer, can send a subscribe request message to GPASS that specifies the following:

  • Consumer policies, including filtering policies to indicate what messages the consumer is interested in
  • Transformation policies to specify the transformation flows, that is, to specify filtering rules or conditions that determine when to apply certain transformation services, and the sequence of applying these services
  • Scheduling policies to indicate when the information consumer is interested in receiving the messages
  • Distribution policies to specify distribution constrains such as message size and message resending policies

You can extend the GPASS architecture to include other consumer-specific custom mediations. The smart subscription created in GPASS on behalf of the information consumer turns the messages that are being published into the messages the consumer wants to receive. The consumer is able to declaratively configure policies through an XML subscription request, and can dynamically change these configurations at run-time.

In this article, we'll walk through some real-world scenarios that highlight the challenges of enterprise flexibility and responsiveness. We'll also describe solutions that extend enterprise capabilities with reusable integration services that facilitate and enable the integration of business services and applications across the enterprise. In particular, we'll focus on scenarios from IBM, like the Order Status scenario and the SAP Ledger scenario, that present real-world requirements for a generalized publish and subscribe pattern. To address these requirements, we'll implement a solution composed of a set of integration, or GPASS, services that provide notification brokering, scheduling, distribution, transformation, and logging functionality. You can reuse the resulting common integration services in business scenarios other than the Order Status and SAP Ledger scenarios.

GPASS scenarios and requirements

Publish and subscribe is an interaction pattern that involves registering consumers and disseminating events to them. Enterprise information sources contain business information for an enterprise and often include master data stores for customer, product, order information, and so on, in addition to operational data stores. Many enterprise business applications depend on and need data from business information sources. These business applications are thus considered information consumers. In many situations, the information consumers need to be able to subscribe to changes or updates in the information sources, and receive selected business information when the changes or updates happen. The consumer requirements for subscribing to information or information changes in an information source, and the subsequent receipt of the specified information at a later time (asynchronously) when the changes occur to the information source, are typically handled by a basic publish and subscribe interaction pattern. This basic pattern focuses on the distribution of notifications and associated aspects like subscription management. However, as described earlier, real world scenarios may have additional requirements beyond the basic pub/sub or notification brokering requirements, such as transformation, distribution, and scheduling. To help explain the requirements for GPASS, we'll describe two real-world IBM scenarios that use GPASS. Each of these scenarios requires functions beyond simple pub/sub functions.

Order status scenario

IBM B2B partners want to receive information on the status of their IBM orders, as well as receive notifications if the status of their order changes. They need to be able to specify the following:

  • Type of order status changes
  • Output message format
  • Delivery schedule
  • Output order message size
  • Message resending and persistence rules

Specify the type of order status changes

This requirement implies that partners (information consumers) want to specify the kind of notifications they require. Through subscribing, or sending a subscribe message to the Order Status application, partners can record that they are interested in some or all of the notifications that the Order Status application, the notification producer, can provide. The subscribe request message may include filter expressions that constrain the kind of notifications the subscription is to cover. The Order Status application must send a notification to each partner that has registered a subscription with it whenever it has a message to deliver and the filter conditions expressed in the subscription are satisfied.

Subscription filtering is needed because it is often inefficient to send messages the consumer is not interested in. WSN specs define three types of filters:

  • Topic filters exclude all notifications that do not correspond to the specified topic (for example, topic = stock/IBM).
  • Message filters define a Boolean expression evaluated over the content of the notification message (for example, /weatherReport/temperature > 35).
  • Producer state filters. In order to specify this filter expression the subscriber must know about the properties of the notification producer. That is, the filter expression is based on some state of the notification producer itself (for example, DebugMode=On) that is not carried in the message.

Each filter expression resolves to True or False. Notifications are sent to the subsribed consumer if all filter expressions resolve to True.

The Order Status scenario requirement to specify the type of order status changes is a filtering requirement that could be satisfied by topic filters. A topic is a concept used to categorize different kinds of notifications and their associated message types. Topics provide a way for a subscriber to indicate the kinds of notifications or the underlying situations it is interested in. The subscriber supplies a topic filter rather than a filter specified in terms of the message body. This allows for more flexibility because topic filters are not tied to the notification message.

Specify output message format

The Order Status scenario requires that business partners be able to specify the output message format they want to receive. For example, partners can subscribe and indicate that they wish to receive order data via a Rosetta-Net 3A61 PIP or some alternative proprietary format. Moreover, partners can request to receive maintenance data via a Rosetta-Net 4C62 PIP, and carrier data as an attachment on a 3A61 PIP.

These requirements mean that a partner subscription must contain filter expressions, policies, and context information. One policy option provides the ability for the subscriber to specify the format of the message that should be transmitted between the producer and the consumer. In the event that the original message is not in the correct format, GPASS must convert the message from the source format to the specified target format.

Specify delivery schedule

This requirement describes a policy that allows the subscriber to specify during which time intervals notifications are allowed to be transmitted from the producer (the Integrated Order Status system) to the consumer (the IBM business partner). This enables the subscriber to ensure that notifications are not transmitted during maintenance windows or down-time for the consumer application. There may be other reasons why partners need to specify a delivery schedule for receiving information. For example, some partners don't want to receive updates during peak load hours, and others want to minimize the cost of retrieving information, especially if they're receiving this information into an Electronic Data Interchange (EDI) mailbox where schedule is a factor in the usage cost. Notifications that can't be transmitted due to the conditions specified in this policy are queued by the producer for delivery during the next available window.

Partners may further want the ability to determine the types of data for which delivery scheduling can be specified. There are five different choices for the frequency with which the deliveries should take place (Immediate, Hourly, Daily, Weekly, and Monthly). Depending upon which frequency is selected, one or more additional parameters may be required to complete the definition, as specified in the description of each frequency option in the following table:

ATTRIBUTEVALUEDESCRIPTION
Type topic Type of message that adheres to the schedule (could be based on message Topic)
Frequency
I = ImmediateNotifications are sent to the consumer as soon as they occur
H = Hourly Notifications are sent once an hour, at the number of minutes past the hour specified in the startTime parameter (the hour value is ignored)
D = Daily Notifications are sent once a day, at the time specified in the startTime parameter
W = Weekly Notifications are sent once a week, on the day specified in the dayOfWeek parameter, and at the time specified in startTime
M = Monthly Notifications are sent once a month, on the day specified in the dayOfMonth parameter, and at the time specified in startTime
startTime hhmm Time of day (in 24-hour clock syntax) at which the notifications are sent
dayOfWeek 1-7 Number of the day of the week on which notifications should be sent (1 - Sunday, 7 - Saturday)
dayOfMonth 1-28 Number of the day of the month on which notifications should be sent (days 29, 30, 31 are not permitted because they do not occur in all months)

This requirement specifies a subscription policy element of the partner's subscription request.

Specify and control the message size

This requirement enables partners, or consumers, to limit the size of notification that will be transmitted from Order Status application (the producer). If a notification message that matches the conditions of this subscription exceeds this size, it will be broken into a sequence of pieces that are each smaller than the size limit and transmitted to the consumer application individually. It is then up to the consumer to reassemble the pieces of the message if required. A large message is converted to many small messages using the specific rendering rules. This requirement specifies another subscription policy element of the partner's subscription request.

SAP Ledger scenario

SAP Ledger is a large IBM business initiative. Part of the scope of the SAP Ledger project is to establish a SAP Enterprise Shared Services instance by integrating into a single SAP instance the following enterprise applications: Ledger, Fixed Assets, General Procurement, and Enterprise SAP (ESAP). The SAP Ledger project places requirements on Offering Information (OI) and Customer Information (CI). SAP Ledger requires:

  1. Customer information when info is created and updated in legacy CMR (Customer Master Record)
  2. Coverage information
  3. Product material information from OIM CDR: What is OIM? Please spell out.

Like the Order Status scenario described earlier, the SAP Ledger scenario requires notification brokering with filtering and message transformation filtering. However, in addition to the requirements of the Order Status scenario, the SAP Ledger scenario requires composite transformation. This enables a consumer to specify what transformations to apply to a notification message, and in what sequence these transformations are appied. For example, for SAP Ledger the notification messages are to be transformed into various types of SAP IDOC messages. However this transformation must take place in steps, depending on the message content. For certain types of notification messages, we want to first apply a base transform service (such as transformation to a basic type of IDOC like a Material Master IDOC). The second step in the transformation applies a segment filter transform on the IDOC that is specific to the consumer requirements. The final step might require a code page transformation if the consumer is not Unicode enabled.

Composite transformation requirement

The logic for the composite transformation requirement consists of multiple steps in sequence, and shown in Figure 2, and described below:

Figure 2. Composite transformation logic implemented in GPASS
Figure 2. Composite transformation logic implemented in GPASS
  1. Check content of message M1, and if condition (C1) is true, then apply Transformation T1 (Base Transform to transform the message to the base IDOC format) to M1 producing message M2. Some consumers are satisfied with this step while others require additional steps.
  2. Check content of message M1, and if condition (C2) is true, then apply Transformation T2 (for example, Segment Filter transform to fill default values in the IDOC and prune some segments) to M2 producing message M3. Some consumers are satisfied with this step while others require additional steps.
  3. Check content of message M1, and if condition (C3) is true, then apply Transformation T3 (for example, Code Page transform) to message M3 producing message M4. Some consumers are satisfied with this step while others require additional steps.

Notification brokering requirements

Relative to the notification brokering requirements for our scenarios, we discovered that the WSN standards more than adequately satisfy our requirements. WSN specifications support the dynamic registration of consumers, as well as subscription filtering with filter expressions including topic filters, content-based message filters, and producer state filters. WSN also supports both push (asynchronous) and pull (synchronous) consumers, and managing requests to query, delete, or renew subscriptions. Moreover, WSN supports a time-based expiration scheme for subscriptions, and multiple publisher-to-notification-broker interactions including simple publishing, registration of publishers, and demand-based publishing. GPASS is based on and extends the WSN specifications. Part 2 of this article describes the WSN specifications in more detail and provide an extensive overview of WSN.

Message size and additional distribution requirements

The information consumer may specify that it does not want to receive messages with payloads that are larger than a certain size. This constitutes the requirement for a policy option that provides the ability to limit the size of notification that will be transmitted between the producer and the consumer. If a notification message that matches the conditions of the subscription exceeds this size, it will be broken into a sequence of pieces which are each smaller than the size limit and transmitted to the consumer individually. Other distribution requirements include message resending rules, persistence rules (for example, persist message for 48 hours before discarding if unable to deliver), guaranteed delivery, and other quality of service rules.

GPASS architecture

As described earlier, the publish and subscribe notification pattern involves registration of consumers and subsequent dissemination of events. Current Web services standards include the OASIS Web Services-Notification (WSN) specifications that standardize the syntax and semantics of the message exchanges that distribute information to consumers. With WSN, an information provider (known as a notification producer) that conforms to WSN can be subscribed to by any WSN-compliant subscriber.

GPASS uses and extends the WSN specifications. In this article, we'll extend the WSN specifications to cover the real-world requirements described in the previous sections. These requirements include scenarios where information consumers have scheduling and distribution constraints beyond basic notification brokering. Moreover, information consumers in the real world may not be WSN-enabled, which means that there is a also a requirement for automatic generation of a WSN Notification Consumer proxy service that receives WSN notifications on behalf of an information consumer and disseminates them to the information consumer based on the transformation, scheduling, and distribution constraints.

GPASS architectural model

The GPASS architectural approach is as shown in Figure 3. The GPASS architecture extends the basic publish and subscribe pattern by extending the subscription capabilities to include the specification of transformation, distribution, and scheduling constraints as part of the publish and subscribe subscription. Additionally, this architecture enables non-pub/sub-enabled systems (that is, information consumers that are not able to consume notification messages of the pub/sub system) to participate in the pub/sub pattern by allowing GPASS to dynamically create a proxy service to receive pub/sub notifications on behalf of the consumer. This is the Information Consumer Proxy Service (ICPS) shown in Figure 3, which also manages the distribution of notifications to the consumer based on the transformation, distribution, and scheduling constraints specified by the consumer upon subscribing.

Figure 3. The GPASS architectural model
Figure 3. The GPASS architectural model

As shown, the model highlights the following components:

  • Information producer: Contains information of interest to a consumer. Good examples of information producers are systems that manage business information for an enterprise and include master data stores for customer, product, order information, and so on, in addition to enterprise operational data stores.
  • Information consumer: Depends on and must consume information from an information producer. For example, many enterprise business applications like order fulfillment systems depend on data from the business information sources.
  • Subscriber: Requests creation of a subscription. It sends a subscribe request message to a notification broker (pub/sub broker). The subscribe request message identifies a notification consumer. A subscription is an entity that represents the relationship between an information consumer and an information producer. It records the fact that the consumer is interested in some or all of the notifications that the producer can provide. It can contain filter expressions, and may be long-running or have a limited lifetime.
  • Publisher: Creates notification message instances. A publisher receives information from entities in the information producer that monitor and detect a situation. A situation is an occurrence that is noted by one party and is of interest to other parties. A notification is a one-way message that conveys information about a situation to other services.
  • Generalized Publish And Subscribe Service (GPASS): Performs a notification broker function between notification consumers and notification producers, and is responsible for sending notifications to the appropriate consumers. It also acts as a subscription manager and manages requests to query, delete, or renew subscriptions. GPASS also manages subscriptions with transformation, scheduling, and distribution constraints. GPASS includes transformation, scheduling, and distribution services that serve as an abstraction layer and hide the complexities of delivering messages to information consumers. The GPASS services manage rules and quality of service of information distribution including message size constraints for consumers, output message format, consumer schedule constraints, message resending rules, persistence rules (for example, persist message for 48 hours before discarding if unable to deliver), guaranteed delivery, and other quality of service rules.
  • Information Consumer Proxy Service (ICPS): Receives notifications from the notification broker (GPASS) on behalf of the information consumer. Typically, the consumer is not able to receive notification messages, hence the need for this service to act on its behalf, collect the notifications, perform some business logic (if the scenario calls for it), enforce the transformation, scheduling, and distribution constraints for the consumer, and then send the results to the consumer.
  • Adapter: An entity that enables the interaction with an information consumer.

GPASS pattern interactions

Following are the interaction steps of the GPASS pattern numbered in Figure 3:

  1. Subscribe: A subscriber, on behalf of the information consumer, sends an XML subscription request message to GPASS. This subscription message specifies transformation, distribution, and scheduling constraints for the information consumer, as well as the basic subscription constraints on information content from the producer that the consumer is interested in. Extending the WSN specifications to implement GPASS describes in detail a WSN subscribe message that is extended with the GPASS subscription policies.
  2. Update notification constraints: The notification broker is updated with the notification constraints specified in the notification subscription indicating the types of messages the consumer is interested in.
  3. Create/update ICPS: If the ICPS does not already exist, this step creates the ICPS and configures it with the appropriate subscription constraints. If the ICPS exists, this step merely updates the configuration constraints of the ICPS. GPASS creates the base template of the proxy service that includes transformation, scheduling, and distribution services. Developers can enhance this basic template to include additional business logic specific to the consumer using the mediation flow capabilities of the development tools. The ICPS design allows for mediation flows to be plugged into it at different points.
  4. Notify: The publisher entity creates a notification message when it receives information from entities in the information producer that monitor and detect a situation, such as information changes or updates that are of interest to consumers. The publisher sends the notification message to GPASS so it can be distributed to the appropriate consumers that have subscribed to that message.
  5. Notification brokering: The notify message sent by the publisher is routed by the notification broker service to the appropriate notification consumer proxy service. The notification broker matches notification messages to the consumers that are subscribed to these notifications. The information consumer proxy service receives the notification message and performs whatever business logic it needs to do in order to create and aggregate the appropriate response to the information consumer. For example, the consumer proxy service might need to access additional information from the information producers to get all the needed data associated with the change. As a result of applying its specific business logic, the notification consumer proxy service assembles a message or a set of messages to be sent to the consumer.
  6. Apply transformation constraints: The notification message is applied against the transformation constraints to determine what transformation module or modules to apply to the message.
  7. Invoke external transformation services: These transformation services and the conditions under which to invoke them are specified in the subscribe message. If transformation constraints are specified, these constrains are executed in this step.
  8. Apply scheduling constraints: The scheduling service applies the scheduling policy constraints if any were specified as part of the subscription. These constraints relate to the specified delivery schedule for the information consumer. Notifications that cannot be transmitted due to the conditions specified in the policy are queued by the scheduling service for delivery during the next available window.
  9. Apply distribution constraints: The distribution service applies the distribution policy constraints if any were specified as part of the subscription. One distribution policy specifies the size limit of a notification message transmitted to the consumer. If a notification message exceeds this size, it will be broken into a sequence of pieces that are each smaller than the size limit and transmitted to the consumer individually by the distribution service.
  10. Deliver: Once GPASS satisfies the scheduling and distribution constraints mentioned in the previous step, it sends the message to the information consumer through an adapter service.

Extending WSN specifications to implement GPASS

This section describes a method for extending the WSN specifications to implement additional GPASS requirements, including message size, schedule and hours of operation, output message format, and so on. We use the WSN subscription policy options to implement the desired extensions. For example, the XML code below denotes a subscription message using the WSN specifications. This code has been augmented with elements specifying additional tags under the SubscriptionPolicy of the WSN subscribe request XML. In these new tags, we specify the additional constraints of transformation rules, scheduling, and distribution. Each of these additional elements is described below.

Listing 1. A sample WSN subscribe message extended with GPASS subscription policy
<wsnt:Subscribe>
    <wsnt:ConsumerReference>
  	    <wsa:address> http://www.ibm.com:9080/Ledger/ICProxyServiceEndpointHere 
		</wsa:address>
    <wsnt:/ConsumerReference>
    <wsnt:Filter>
	    <wsnt:TopicExpression Dialect="http://docs.oasis-open.org/wsn/t-1/
		TopicExpression/Full"> 	
            npex: /TYPE/Customer//SEGMENT/IBM/GEO/897/ACCOUNTGROUP/ZS01/PARTNER/All//.
	    </wsnt:TopicExpression>
    </wsnt:Filter>
    <wsnt:InitialTerminationTime> 2008.12.25T00:00:00.00000Z  
	</wsnt:InitialTerminationTime>

    <wsnt:SubscriptionPolicy>
           <gpass:InformationConsumerReference> 
                     <wsa:Address> URI of Information Consumer </wsa:Address> 
                     <wsa:ReferenceParameters> ..  </wsa:ReferenceParameters>
     	     <wsa:Metadata> ... </wsa:Metadata>
          <gpass:InformationConsumerReference> 

          <gpass:NotificationTransform>
	   	 …
         </gpass:NotificationTransform>
    </wsnt:SubscriptionPolicy> 
</wsnt:Subscribe>

The WSN <ConsumerReference> element is an endpoint reference that specifies the address of a WSN notification consumer. An endpoint reference is defined in the WS-Addressing specifications. In the GPASS implementation, the resource address specified in the <wsa:address> tag (under <ConsumerReference>) represents the unique address (URI) of an Information Consumer Proxy Service (ICPS). If the ICPS already exists, then the subscribe request (from a subscriber on behalf of an information consumer) will include its URI in this tag to update the configuration parameters of the existing ICPS. If an information consumer does not have an ICPS (or a service that can receive WSN notifications) and wants to create an ICPS on its behalf, the URI is left blank. GPASS will create the base ICPS and configure it with the constraints specified in <SubscriptionPolicy>. The consumer, however, will have to specify an endpoint reference that can receive the notification messages it wants to receive in the format it specifies. This is specified under the <InformationConsumerReference> tag of the subscription policy. This endpoint is typically an adapter service, like a WebSphere SAP Adapter, or a message queue. An endpoint reference, as defined in the WS-Addressing specifications, can contain reference parameters and metadata in addition to the endpoint address. These features are used in the GPASS implementation as we will illustrate in the following sections. In Listing 1, you see that the GPASS extensions are created under the <SubscriptionPolicy> section of the subscribe message. One such extension is the GPASS notification transform <NotificationTransform>, which is described in more detail in the next section.

Specifying message transformation constraints

Earlier, we described the composite transformation logic implemented in GPASS as shown in Figure 3. In this section we describe the XML implementation that enables the specification of the transformation constraints. Each transformation applied to the message (the payload of the WSN <Notify> message) is modeled as a series of transformation steps, or sub-transformations. Each step is represented by an ACTION and a CONDITION. The ACTION specifies the transformation service to apply if the corresponding CONDITION is satisfied. A CONDITION is an XPath expression representing a content-based filter. The expression, or condition, must be true when matched with the Notify message for the action to take place. A subscription can contain specifications for 0 or more transforms. Each transform can have one or more rules, and each rule specifies one ACTION and a set of conditions specified in the CONDITION expression. The current implementation of GPASS implements Concrete XPath expressions (no wildcards) for conditions. Future implementations will probably include Full XPath expressions, including wildcards. Listing 2 illustrates these concepts.

Listing 2. Extending the WSN subscribe message to specify the GPASS transformation constraints
<wsnp:NotificationTransform dialect=”http://www.ibm.com/gpass”>
     <gpass:Transform>
          <gpass:TransformStep> 
               <gpass:TransformReference> 
                     <wsa:Address> 
                           http://www.ibm.com:9080/ossc/
				BaseTransformServiceEndpointHere    
                    </wsa:Address> 
                    <wsa:ReferenceParameters>
	            <gpass:Action> Apply_Base_Transform </gpass:Action>
    	            <gpass:Type> IDOC</gpass:Type>
            <gpass:Name> SAPLedger_ZS01  </gpass:name> 
                            <gpass:IDOCName>DEBMAS02</gpass:IDOCName>
                            <gpass:IDOCVersion>3.1</gpass:IDOCVersion>
                     </wsa:ReferenceParameters>
	                 <wsa:Metadata> ... </wsa:Metadata>
                  </gpass:TransformReference> 
             <gpass:TransformFilter>  
        <wsnt:MessageContent Dialect="http://www.w3.org/TR/1999/REC-xpath-19991116">
                                 /Group = 'ZS01‘       </wsnt:MessageContent>
            </gpass:TransformFilter>
            </gpass:TransformStep>
	     …..
<gpass:TransformStep>							
       <gpass:TransformReference> 
            <wsa:Address> http://www.ibm.com:9080/ossc/
		SegmentFilterTransformServiceEndpointHere     </wsa:Address> 
	        <wsa:ReferenceParameters> … </wsa:ReferenceParameters>
	        <wsa:Metadata> … 	</wsa:Metadata>
       </gpass:TransformReference> 
  </gpass:TransformStep>

  <gpass:TransformStep>							
       <gpass:TransformReference> 
            <wsa:Address> http://www.ibm.com:9080/ossc/
		CodePageTransformServiceEndpointHere  </wsa:Address> 
	        <wsa:ReferenceParameters>    ...  </wsa:ReferenceParameters>
	        <wsa:Metadata>  ... </wsa:Metadata>
        </gpass:TransformReference> 
	    <gpass:TransformFilter> 
		<wsnt:MessageContent Dialect="http://www.w3.org/TR/1999/REC-xpath-19991116"> 	
                            		/NativeCodePage = '943‘  </wsnt:MessageContent>
   	    </gpass:TransformFilter>
    </gpass:TransformStep>
</gpass:Transform>
       
<gpass:Transform> You can specify another transform here with multiple steps. 
	Note that this transform is applied only if the previous transform did not 
	take place (match). 
       </gpass:Transform>
</wsnp:NotificationTransforms>

The example represented in Listing 2 is taken from the SAP Ledger scenario implementation where a composite transformation is required. In this example, we specify three different transform steps to implement our transform. These transform steps are a Base transform, followed by a Segment Filter transform, followed by a Code Page transform. In the first transform step, the Base transform service is invoked only if the transform filter condition is True. In second transform step, no transform filter is specified, which indicates that the condition for this step is always true, and thus the Segment Filter transform service is always invoked after Step 1. In the third transform step, the Code Page transform service is invoked if the transform filter condition in step three is True.

Controlling message size

This policy option enables you to limit the size of notification that will be transmitted between the producer and the consumer. If a notification message that matches the conditions of this subscription exceeds this size, it will be broken into a sequence of pieces that are each smaller than the size limit and transmitted to the consumer application individually. It is then up to the consumer to reassemble the pieces of the message if required. A large message will be converted to many small messages using the specific rendering rules. The solution adds a maxTransmissionSize element to satisfy this requirement. This policy option is configured by use of the maxTransmissionSize element which has the following schema:

<xsd:element name="maxTransmissionSize" type="xsd:positiveInteger"/>

where maxTransmissionSize is the maximum size in kilobytes (KB) of application data that should be transmitted in a single piece between the producer and the consumer. Notifications larger than this size will be broken down into several pieces which are then transmitted individually. This policy element is specified in the SubscriptionPolicy element of the Subscribe operation exposed by the pub/sub or notification broker. The following example specifies that messages larger than 1 MB must be split into pieces that are each smaller than 1 MB in order to be transmitted to the consumer application;

<gpass:maxTransmissionSize>1024</gpass:maxTransmissionSize>

Delivery scheduling

This option is specified in the SubscriptionPolicy element of the subscribe request. The example shown below indicates that maintenance (MAINT) messages should be delivered once an hour at 40 minutes past the hour.

<gpass:DeliverySchedule type"MAINT" frequency="H" startTime="1040"/>

The next example indicates that the SCAC feed should be delivered once a month, on the 17th of each month, at 11 PM.

<gpass:DeliverySchedule type"SCAC" frequency="M" dayOfMonth="17" startTime="2300"/>

Summary

In this article, we described an adaptive, policy-driven notification framework that enhances ESB connectivity by providing functions to support a generalized pub/sub interaction pattern. This framework is based on GPASS, a set of reusable integration services that were developed for IBM and are extensively used and reused by IBM in real-world scenarios. The article highlighted an IBM SOA case study, described the scenarios that formulate the requirements for GPASS, derived the functional specifications, described the GPASS architecture, and designed a solution based on the WSN stanndards. In the process, you learned how to use and extend WSN standards to implement real-world scenarios, such as updating LOB systems from master data.

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere, SOA and web services
ArticleID=249266
ArticleTitle=GPASS: A generalized publish and subscribe solution using WS-Notification standards, Part 1 : An introduction
publish-date=08222007