Managing services dynamically using WebSphere DataPower SOA Appliances with WebSphere Service Registry and Repository

Content

Introduction

A key aim of SOA is to loosely couple service requestors and service providers. Decoupling the service implementation from the service interface is one step, so that changing the provider's service implementation does not require any changes to the service requestor. But what about the service endpoint -- using a static endpoint provided by the service description (WSDL) means coupling the requestor and provider at the transport level:

Figure 1. Transport level coupling

Every time the physical endpoint of the service provider changes, every service consumer must reflect the changes in its application logic. A better solution is to use an intermediary that intercepts a service call and routes it to the appropriate endpoint. Abstracting from transport-specific information to have a transparent intermediary layer between service consumers and providers is one aspect of the Enterprise Service Bus (ESB) pattern. The WebSphere DataPower SOA Appliance and its Web Service Proxy component can be part of an ESB implementation by acting as such an intermediary layer. Implementing this part of an ESB layer in a static way by referencing WSDL files in the WS-Proxy component and providing a static service endpoint to the consumer provides a single point of control when changes are made to the real service endpoint, thus simplifying management of the service endpoint. With this approach, the service consumer is decoupled from the provider at the transport level, but changes on the proxy side are still necessary when changes are made on the providers endpoint.

Figure 2. Decoupling with WS-Proxy

Using DataPower's WebSphere Service Registry and Repository subscription feature inside the service proxy instead of statically referencing the WSDL file is the way to get a more dynamic service intermediary. DataPower can subscribe to WSDL files or to other artifacts stored in WebSphere Service Registry and Repository, and synchronize with changes either automatically by polling or manually driven from outside. Form a service endpoint management point of view, this approach provides a flexible and dynamic ESB layer functionality that reflects changes to the endpoints automatically without having to make changes on the service consumer nor on the ESB side:

Figure 3. Dynamic decoupling with WS-Proxy

This article describes how to set up such an dynamic Web service proxy with WebSphere Service Registry and Repository and DataPower. The article first describes the basic configuration -- how to set up the WS-Proxy and connect it to WebSphere Service Registry and Repository to subscribe to service definitions. The article then describes fault handling, schema validation, and how they can be realized in the WS-Proxy component of DataPower.

Prerequisites

The following prerequisites are required to set up the described scenario:

WebSphere DataPower SOA appliance with firmware V3.6.0.8 or later

This article uses a DataPower XI50 Appliance with firmware V3.6.0.14. The WSRR subscription feature was introduced in V3.6.0.8.

WebSphere Service Registry and Repository V6.0 or later

The article uses WebSphere Service Registry and Repository V6.0.0.2, but the DataPower subscription works for earlier and later versions as well. WebSphere Service Registry and Repository V6.1 will available in 1Q 2008.

Web service requestor application

Either you have a Web service requestor application or you can use the Web service explorer feature of WebSphere Integration Developer V6 or Rational Application Developer V6 or V7.

Web service provider application

You can use any Web service as long you have the corresponding WSDL file. You can download a sample Web service application below and deploy it on WebSphere Application Server V6.0 or later. WebSphere Service Registry and Repository is running on top of a WebSphere Application Server V6.0.2.17, so if you have free access to this WebSphere Service Registry and Repository installation, you can use it to host the Web service application. The sample application contains a simple Web service called Order Service. The request data type is a order object, which consists of a customer object and 1:n item objects. When no error occurs, the Order Service responds with a order confirmation containing an order id. This order id is hard-coded in the service implementation so the value is always the same.

WebSphere Service Registry and Repository configuration

Before you can start to configure the WS-Proxy in DataPower, you must register the Web service you want to subscribe to in WebSphere Service Registry and Repository, as described below.

Upload files

To register the Web service in WSSR, upload the WSDL and all of its dependent XSDs via the administrative console, as shown below. The administrative console is provided via a Web interface -- to access it, use a browser and enter the URL of your WebSphere Service Registry and Repository installation. The URL of the administrative console has the following structure: http://<hostname>:<port_number>/ServiceRegistry. For example: http://localhost:9080/ServiceRegistry

.

Figure 4. XSD dependencies

The minOccurs attribute of the customer and the order element is set to 1, meaning that these objects are required in every order request. Later you will use that fact to cause a validation error in the Web service proxy.

1. Log in to the administrative console

   Enter the URL of your WebSphere Service Registry and Repository installation. When the login screen opens, log in with your credentials. If you don't have configured security, you will be forwarded to the administrative console:

   Figure 4a. WSRR administrative console

2. Upload the XSD files.

   Under Service Documents go to XSD Documents and upload Item.xsd, Customer.xsd, and Order.xsd by clicking Load Document and following the instructions. Beside giving a description of the datatype you are uploading, you can provide a version number such as 1.0 (or you can leave it blank). The XSD Documents page should look like this:

   Figure 5. XSD import

3. Upload the WSDL file.

   After you have successfully uploaded the XSD documents, you can upload the WSDL file. Under Service Documents go to WSDL Documents and upload the WSDL in the same manner as you did with the XSD documents. Your WSDL Document page should show the WSDL file:

   Figure 6. WSDL import

You have now registered the Order Service in WSRR, so you can now query this service definition using DataPower.

WS-Proxy configuration

In this step, you configure the DataPower device. First, log in to the device and your domain (Figure 6a). Before creating the Web service proxy, you must define a Http Front Side Handler object and a WSRR Server object, which will both be used by the Web service proxy. The Http Front Side Handler object will listen on a specific URL and will route the HTTP requests sent to this URL to the Web Service Proxy service. The WSRR server object is the definition of the WSRR you want to use for the subscription of service definitions.

Figure 6a. DataPower login

Preparing configuration objects

The following section shows you how to create and configure the objects used by a Web Service Proxy service:

1. WSRR Server Object

   Under the left navigation pane on the DataPower Web GUI, navigate to the Objects section and click WSRR Server in the Configuration section. A list of all WSRR server definitions opens. Click Add to define a new WSRR Server:

   Figure 7. WSRR server configuration

   Assign a name such as MyWSRRServer to your WSRR server. In the SOAP URL field, replace hostname and port number with the values of your WSRR installation. When your are running WSRR without security, your protocol will be HTTP. When your are running WSRR with security enabled, provide an SSL proxy profile, username, and password. For information on how to configure an SSL proxy profile, see the DataPower documentation. Click Apply. The Web GUI switches back to the WSRR server list, where your WSRR server definition should appear.

2. HTTP Front Side Handler

   It is not mandatory to define the HTTP Front Side Handler in the following way. During the creation of the Web service proxy, you will be able to choose a default HTTP Front Side Handler. You would have to change this default HTTP Frontside Handler to your requirements anyway. Under the left navigation pane on the DataPower Web GUI navigate to the Objects section and locate the HTTP Front Side Handler link in the Protocol Handlers subsection. After clicking this link a list with all HTTP Front Side Handler definitions will appear. Click add to define a new HTTP Front Side Handler. You can keep all defaults.

   In the name field choose for example MyHTTPFrontsideHandler as name of the new handler. As Local IP Address choose the IP address of one of the ethernet interfaces. You can figure out this IP address at the right navigation pane under Status, IP Network, Ethernet Interfaces. If you leave the default 0.0.0.0, you will reach the HTTP Frontside Handler on all configured ethernet interfaces. Under Port Number fill in a port which isn't already used by the DataPower device, for example 9080. If you want to be able to retrieve the new WSDL of the Order Service (generated by the WS-Proxy) with ?wsdl in your browser you have to enable the HTTP GET method, too. Here are the settings:

   Figure 7a. HTTP frontside handler configuration

   Save your configuration on the upper right of the Web GUI: Click Save config.

   Figure 7b

Setting up WS-Proxy Service

In order to create a new Web Service Proxy click on the Web Service Proxy Icon on the Control Panel:

Figure 8. DataPower control panel

1. Specify the name for the Web Service Proxy Object and choose the way you want to add the WSDL files. In order to be able to subscribe to service definitions stored in WSRR choose Add WSRR Subscription:

   Figure 9. WS-Proxy configuration

2. Add a HTTP Front Side Handler and WSRR Server object to your Web Service Proxy service. Choose the previously created WSRR Server and HTTP Front Side Handler. Also the settings in Table 1 have to be applied to the configuration page.

   Attribute	Value
   WSRR Subscription Name	MyServiceSubscrition
   WSRR Server	MyWSSRServer
   Subscription Object	WSDL Document
   Namespace	http://wsrr.integration/orderservice
   Object Name	OrderService.wsdl
   Local Endpoint Handler	MyHTTPFrontsideHandler
   Local URI	/services/OrderService

   Your configuration now should look like this:

   Figure 10. WS-Proxy subscription

After verifying your settings, save your configuration on the upper right of the Web GUI by clicking Save config.

Verifying subscription status

First examine the status of the WSRR subscription. Under Status in the left navigation pane you can find WSRR Subscription Status and WSRR Subscription Service Status under the Web Service subsection. Click on both and examine the status and the retrieved information. When all worked fine the two status tables should look like Figure 10a and 10b:

Figure 10a. WS-Proxy subscription service status

Figure 10b. WS-Proxy subscription service status

Verifying the endpoint information

Now examine if DataPower has retrieved the right endpoint information from WSRR. Return to the Status section and under the Web Service subsection click on Web Service Operations. There will be a table with all currently registered Web Service operations. The very last column on the right shows the back end address which is the service endpoint DataPower has retrieved from the service definition in WSRR. Figure 10c shows the last two columns of this table.

Figure 10c. Web service operation

Testing the WS-Proxy

The Web service proxy now is ready for a first test. The new external endpoint of the Web service is the Local IP Address and the Port Number of the HTTP Frontside Handler followed by the Local URI of the WSRR subscription (http://<EthernetInterfaceIP>:9080/services/OrderService). You can use any Web service client to send a valid SOAP request to the Web service proxy's endpoint. In order to get a better understanding of what happens you can use a TCP/IP monitor to examine the SOAP request and the SOAP response. For sending SOAP requests and examine the SOAP response you can also use the build in Web Service Explorer of the WebSphere Integration Developer or the Rational Application Developer. You can send any order information as long you have at least one required order object and one required customer object in your order.

  <?xml version="1.0 encoding="UTF-8"?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:q0="http://wsrr.integration/orderservice"  xmlns:xsd="http://www.w3.org/2001/XMLSchema"  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <SOAP-ENV:Body> <q0:placeOrder> <order> <customer> <firstName>Peter</firstName> <LastName>Jones</LastName> <id>2838382128</id> </customer> <items> <articleID>837228304</articleID> <quantity>3</quantity> </items> <items> <articleID>8272666333</articleID> <quantity>4</quantity> </items> </order> </q0:placeOrder> </SOAP-ENV:Body> </SOAP-ENV:Envelope>

  <?xml version="1.0 encoding="UTF-8"?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"  xmlns:xsd="http://www.w3.org/2001/XMLSchema"  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <SOAP-ENV:Header/> <SOAP-ENV:Body> <q0:placeOrderResponse xmlns:q0="http://wsrr.integration/orderservice"> <orderID>ID_123456</orderID> </q0:placeOrder> </SOAP-ENV:Body> </SOAP-ENV:Envelope>

Now you can try sending an request without one of the following required objects. The Web service proxy will validate each SOAP request against the schema provided by the WSDL and the XSD files. When a invalid request is sent to the box the result will be an "Internal Server Error" response.

  <?xml version="1.0 encoding="UTF-8"?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:q0="http://wsrr.integration/orderservice"  xmlns:xsd="http://www.w3.org/2001/XMLSchema"  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <SOAP-ENV:Body> <q0:placeOrder> <order> <customer> <firstName>Peter</firstName> <LastName>Jones</LastName> <id>2838382128</id> </customer> </order> </q0:placeOrder> </SOAP-ENV:Body> </SOAP-ENV:Envelope>

  <?xml version="1.0 encoding="UTF-8"?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"  <SOAP-ENV:Body> <SOAP-ENV:Fault> 	<faultcode>General</faultcode> 	<faultstring>Internal Error</faultstring> 	</SOAP-ENV:Fault> </SOAP-ENV:Body> </SOAP-ENV:Envelope>

To see the System Log go to Status, System Log on the right navigation pane. When you have a look in the System Log of DataPower you will see the Schema Validation Error the invalid request has caused Figure 11:

Figure 11. System log

When all this tests worked fine, your WS-Proxy is completely configured. For the case you have encountered any problems the next sections describes how you can debug your WS-Proxy.

When your Web service proxy is not working as expected and you get runtime errors there are two options for debugging:

You can have a look at the System Log like it is shown above. You can change the log level of the system log by going to the Control Panel and choosing Troubleshooting. Under Logging change the Log Level to debug and click on the Set Log Level button. Debug will be the most fine grained level:

Figure 12. Set log level

Probe

When the System Log is not detailed enough and you can identify the component that raised the error, you can put a probe in it. To add a probe to the Web Service Proxy service, go to the Control Panel and click WSProxy => MyWebServiceProxy:

Figure 13. Enabling the probe

Click Show Probe => Enable Probe. A detailed log for this specific Web Service Proxy object is activated:

Figure 14. The Probe window

You can see this log on the current window (after you have sent the next request) when you click Refresh. For information on how to use probes in DataPower, see the documentation. Disable probes after debugging because they use significant system resources.

Advanced configuration and fault handling

This section describes how to do more fine-grained configuration of the Web service proxy and implement more sophisticated fault handling. The DataPower Web Service Proxy service leverages more features and configuration capabilities than just listening on specific HTTP endpoints and forwarding incoming SOAP calls. You can see the range of possibilities in the Configure Web Service Proxy section of your Web Service Proxy.

Configuring the validation behavior

As described above, the DataPower Web Service Proxy service validates an incoming request against a WSDL file. You can change this validation behavior for each Web service or even for each operation or port.

1. For configuring the message validation go to the Policy tab of your Web Service Proxy Configuration page:

   Figure 15. WS-Proxy configuration

2. Starting with the subscription object you can expand the tree down to the operation level. On each level you can configure the validation behavior by clicking on the green icons:

   Figure 16. Validation settings

3. You can not only enforce the validation of the SOAP request. You can enforce the validation of the other SOAP message types like the SOAP response and the SOAP fault message, either. Validation of the SOAP header and some other settings like enabling WS-Addressing are possible, too:

   Figure 17. Validation settings details

Advanced fault handling

It is not very meaningful sending SOAP faults in form of an internal server error back to the client. Especially when the root cause of the error is the request of the client it would be more beneficial when a more detailed fault message is returned. Giving back the validation error instead of an internal server error would give the client information about what is wrong with its request. The next section will cover how to set up the Web Service Proxy service in a way in which it sends the validation error information in a SOAP fault message back to the client. This can be achieved in adding a OnError action to the Web Service Proxy policy processing rules. You can add this OnError rule either in the default request rule or to a specific rule on a specific level of the Order Service. When there would be more then one service subscription you could add the OnError or any other action just to one of the services if necessary. Here we will add the OnError action to the default request rule of the Web Service Proxy. Adding an on-error action at a specific point in a processing rule means that the on-error action is triggered when any of the following processing steps (actions) contained in this rules raises a fault. In order to do so click on the link MyWebServiceProxy_default_request_rule. You will see this screen:

Figure 18. Default request rule

Stylesheet preparation

1. Before configuring the default request rule we have to create an XSLT stylesheet which will extract the fault information and put it into the SOAP fault message later on. DataPower comes with some DataPower specific XSLT extension functions. We have to use one of this extension functions to extract the fault information from one of the build in service variables. This can be done by using the dp:variable() extension function. After extracting the fault information the SOAP fault can be build. Listing 5 shows the stylesheet: Listing 5. Stylesheet code

     <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"  xmlns:dp="http://www.datapower.com/extensions" extensions-Element-prefixes="dp">  <xsl:variable name="error" select="dp:variable('var://service/error-message')"/>  <xsl:template match="/">  <soapenv:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"  xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema"  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <soapenv:Header />  <soapenv:Body> <soapenv:Fault> <faultcode xmlns:p618="http://wsrr.integration/orderservice"> p618:placeOrder_orderFailurefMsg </faultcode>  <faultstring>integration.wsrr.PlaceOrder_orderFailurefMsg</faultstring>  <detail encodingStyle=""> <p618:placeOrder_orderFailuref xmlns:p618="http://wsrr.integration/orderservice"> 	<xsl:value-of select="$error"/> </p618:placeOrder_orderFailuref>  </detail> </soapenv:Fault> </soapenv:Body> </soapenv:Envelope>  </xsl:template> </xsl:stylesheet>

2. After you have created the XSLT File either via a text editor or a development tool you have to upload the file to the device. Go to the Web GUI again and select File Management under the Administration section of the left navigation pane or on the Control Panel. Under the Actions... menu of the local storage click on Upload Files:

   Figure 19. Upload files

Now the request rule and a new error rule has to be configured for having a kind of try and catch error handling.

1. Go to the default request rule (Figure 18) and drag the Advanced icon behind (form left to right) the Match action. Double click on the new yellow squared Advanced icon:

   Figure 20. Advanced action

2. Activate the on-error radio button and click Next. Now the configuration wizard of the on-error action will appear. On this configuration page it has to be specified which data are used for input of the on-error rule and which data is used as output. In DataPower this data containers are called context. Since we want to build a new SOAP Fault message we can use any input but must use the OUTPUT context for output, because the result of the on-error action processing is written to this OUTPUT context. So specify (auto) as input of the on-error action and OUTPUT as output. Error Mode will be abort because we don't want to correct the error and continue but discard the request and send back an error to the client. Click Done:

   Figure 21. On-error action configuration

3. After that a processing rule has to be defined. This rule determines the actions which has to be taken when the on-error rule is triggered. So click on the + sign to create a new processing rule. Name the new processing rule MyErrorRule and choose error-rule as Rule Direction. Now add a action to this rule which executes the previously created XSL stylesheet. Click the + sign on the lower right and name the action MyErrorAction. Choose xform as Action Type and use your recently uploaded stylesheet in the Transform field. Input will be INPUT and Output will be OUTPUT. You can now save all opened windows by pushing either Apply or Done. Here are the settings:

   Figure 22. Transform error action

4. Don't forget to apply the changes also to the policy on the left top of the policy configuration page, which should now look like this:

   Figure 23. Rule with error handling

Now you should save your configuration and test your changes by sending the invalid SOAP request. The SOAP Fault message should not be the "Internal Server Error" anymore but detailed information about the validation error.

Conclusion

After doing the configuration work described above, you have now a dynamic Web Service Proxy, which is able to subscribe to service descriptions stored in WebSphere Service Registry and Repository. Beyond the more flexible decoupling of service consumer and provider, the WS-Proxy component in DataPower supports many additional service intermediaries. For more information, see Related topics below. For example, WS-Proxy can act as an XML firewall, service-level monitor, and security policy enforcement point:

• A XML firewall can protect the service hosting system from XML attacks with its XML Threat Protection mechanism, and can validate the schema conformance of a message before it is sent to the provider (the schema validation part was realized in the article). Doing this work on the provider side would cause unnecessary load on the hosting machine, and is more efficient on an optimized device like DataPower.
• The Web service proxy component can act as a security policy enforcement point. You can use the proxy not only for authenticating and authorizing the calling user in the provider domain, but also for mapping the different kinds of security credentials or tokens of the consumer and provider domain, using the DataPower AAA-Framework.
• Monitoring and managing a service can also be done on the proxy, such as measuring traffic going to the service and triggering appropriate actions. For this purpose, the device offers a Service Level Monitoring (SLM) component.

This article described DataPower features in conjunction with WebSphere Service Registry and Repository. You can also use more sophisticated features on WebSphere Service Registry and Repository to get more flexibility. For example, you can create ontologies to abstract from technical artifacts such as WSDL and XSD files, by grouping artifacts in a "concept." DataPower can also subscribe to this concept instead of directly subscribing to service descriptions (WSDL files). For more information on these WebSphere Service Registry and Repository features, see the product documentation.