IBM WebSphere DataPower SOA Appliances represent an important element in Service Oriented Architecture (SOA). The appliances are purpose-built, easy-to-deploy network devices that simplify, secure, and accelerate XML and Web services deployments while extending an SOA infrastructure.
One simple example for its use is a multi-service environment in which inbound requests can be sent into one of several Web service providers. In such a scenario, an incoming service request is evaluated based on the incoming parameters (URL, headers, and request parameters) and system status (date, load, and so on).
In a relatively static use case, the routing rules would not change very often. Routing rules can be configured manually using a Web GUI-based console or by uploading XML-encoded rules. While authoring the routing rules inside the appliance is not a demanding task, managing consistent configuration across a cluster of appliances increases operational effort.
IBM provides a multi-box management solution in IBM Tivoli Composite Application Manager System Edition (ITCAM SE), which enables administrators to efficiently roll out, update, and manage configurations across multiple WebSphere DataPower SOA Appliances. ITCAM SE pushes configuration to each device in turn. Further, IBM WebSphere Registry and Repository is also available for enterprise-class centralized policy storage and management. The DataPower devices can dynamically consume policies made available in WebSphere Registry and Repository for compliant Web services traffic.
This article presents a pull-based approach that can apply to both the WebSphere DataPower XML Security Gateway XS40 and the WebSphere DataPower Integration Appliance XI50. With this approach, you create a centralized management solution by configuring generic routing logic into each appliance and then have the script call external routing rule services. The rule services resolve the actual route lookup for the generic logic. This technique enables the routing rules to be managed in a single centralized repository used by several appliances. The appliances can cache route lookups so that subsequent traffic does not require an external network transaction. This method works with XML, Web services, and non-XML traffic. In this approach, you implement a simple two-step lookup to handle a class of cache staleness.
The rich, flexible, standards-based customization capabilities of WebSphere DataPower appliances enable you to easily fit the appliance in many situations. Through this demonstration, you will see the strength of the DataPower APIs and be encouraged to further investigate the rich set of configuration objects, extensions, and elements that are available. While you work with standard operating practices as they evolve and expand to fit more use cases, you should be pragmatic in utilizing customization capabilities to solve problems today
The remainder of this article presents the various steps associated with developing the routing scripts and configuring them in the DataPower appliance, starting with a brief system overview and a description of the routing rules, followed by steps to configure the appliance and develop the routing scripts.
Figure 1 shows an edited presentation of the system (omitting components such as firewalls) in which the appliance and routing scripts operate in this scenario. In general, the system is composed of:
- Several clients that access services though the network.
- Servers hosting the various services used by the clients.
- An external central routing rule repository, exposed through an HTTP URL.
- One or more WebSphere DataPower SOA Appliances that route between the clients and their servers. Inside this appliance are routing scripts that communicate with the central rule repository to make decisions.
Figure 1. System overview
Routing a certain client to a service involves these steps:
- The route script running inside the appliance fetches a routing rules list from the remote Web server repository. The rules exist as an XML document, and, for improved performance, can be cached locally by the DataPower appliance with settable time-to-live.
- The request parameters and, especially, destination URL, are matched with the rules’ patterns.
- As a backup, if a destination was not resolved (perhaps due to caching staleness), a Web services request is sent to a routing rules resolution server (the routing rules repository and the routing rules Web service can be hosted on the same or different servers). The request contains the request URL as key parameters. The URL-based key can be easily expanded to include other characteristics of the user request.
- If the destination was resolved, the original user request is forwarded to the destination; otherwise an error is returned to the user.
Routing rules and repository server
The route rules employed here are encoded using an XML format, as shown on Listing 1. Each rule contains:
- Rule name: unique name that is used for the logging and rule management.
- URL pattern: a Perl-compatible regular expression string to be matched with the URL sent by the client.
- Destination URL: the target service URL.
Listing 1. Rules.xml
<?xml version="1.0" encoding="UTF-8"?> <rules> <rule> <name>rule1</name> <pattern>http://.*/first/.*</pattern> <destination> http://rhel3-6.haifa.ibm.com:9080/TestWeb1/services/TestSoap </destination> </rule> <rule> <name>rule2</name> <pattern>.*second.*</pattern> <destination> http://rhel3-6.haifa.ibm.com:9080/TestWeb2/services/TestSoap </destination> </rule> </rules>
Following the rules encoded in Listing 1:
- The URL http://test-lnx.asdf.com/first/test will be routed to http://rhel3-6.haifa.ibm.com:9080/TestWeb1/services/TestSoap.
- The URL http://www.secondTest.com/hello/ will be routed to http://rhel3-6.haifa.ibm.com:9080/TestWeb2/services/TestSoap.
- The URL http://www.ibm.com/ will not be matched, and will be rejected.
The routing rules are stored in a rules repository with two interfaces:
- Simple GETFull HTTP-based interface: a client can send an HTTP GET request and download a list of all rules; the output of such a request will follow the format presented in Listing 1.
- SOAP-based interface with a single method named getWSDestinationForUrl: the method accepts the key (URL) as its single parameter, and returns a destination URL or null. (The Web Service WSDL and examples of the code are available in the included download file.)
Configuring an XML firewall
The first step in using the routing script is to configure a WebSphere DataPower XMLFirewall object on your appliance.
Using the WebSphere DataPower control panel, create a new XML firewall called “dw-example,” with a Firewall Type of dynamic-backend, Response Type of Pass-Thru, and Request Type of Non-XML (other request types like XML and SOAP are suitable as well, but enforce XML well-formedness, and SOAP enveloping and schema, respectively). The dynamic-backend type is important because it informs the device that the backend host and URL are subject to modification by the processing policy. This is what enables the generic routing logic to do its work.
The firewall definition UI is shown in Figure 2.
Figure 2. XML firewall
XML firewall policy definition
Next, create a new firewall policy with one request processing rule. This rule contains a fetch action and three transformation actions that implement your routing logic. All transformation actions can be combined into one, but they are split up in this example for readability purposes. The policy definition UI is shown in Figure 3. Notice that the very first icon in the policy is a disambiguation “match” action that is required by the device to decide between multiple rules. Because there is only one rule here, the match action is not used.
Figure 3. Configure XML firewall policy
The fetch action is used to get the entire set of routing rules from the repository and place them in a temporary WebSphere DataPower context. The fetch action is a predefined WebSphere DataPower action, with one source parameter that you point to the routing rules repository URL. Like its name suggests, this fetch action makes an HTTP request to fetch the configured source URL. At this point, the fetched document is generic and has no meaning to the device.
Additionally, to avoid excessive traffic to the remote repository, you can use the WebSphere DataPower provided cache, as can be managed from the XML Manager object associated in the previous XML firewall screen and described in the XML Manager definition section.
The output “tempvar1” will contain the fetched XML document. The fetch action definition UI is shown in Figure 4.
Figure 4. Configure fetch action
First route resolution action
The next action is a WebSphere DataPower transformation that executes the first route resolution stage. The XSLT script used for the transformation (Listing 2) is stored locally on the appliance, although it could also be stored remotely for even further cluster manageability. The script uses a WebSphere DataPower context variable to pass its result to the following actions. The actual transformation output generated by the script is unused because the setting of the context variable is essentially a side effect to the transform, and is the real routing work that we are interested in. The transformation definition UI is shown in Figure 5.
Figure 5. Configure transform action
The script goes over all the routing rules that were received in the prior fetch action (tempvar1 is used as the input to this action), and tries to match the inbound messages’ URL with the URL pattern in the various rule entries, using regular expressions. If a certain rule is matched, the rule name and its destination are stored in WebSphere DataPower context variables var://context/dw/route and var://context/dw/ruleName. The rule name can be logged for audit and troubleshooting reasons. (An XSLT for-each element does not have a break option, so if several rules are matched, the last one will be used.)
Listing 2. Rules processor XSL file
<?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" xmlns:dpconfig="http://www.datapower.com/param/config" xmlns:regexp="http://exslt.org/regular-expressions" extension-element-prefixes="dp" exclude-result-prefixes="dp dpconfig regexp"> <xsl:template match="rules"> <xsl:for-each select="rule"> <xsl:variable name="pattern"> <xsl:value-of select="pattern" /> </xsl:variable> <xsl:if test="regexp:test(dp:variable('var://service/URL-in'), normalize-space($pattern))"> <xsl:variable name="dest"> <xsl:value-of select="destination" /> </xsl:variable> <dp:set-variable name="'var://context/dw/route'" value="normalize-space($dest)" /> <dp:set-variable name="'var://context/dw/ruleName'" value="string(name)" /> </xsl:if> </xsl:for-each> </xsl:template> </xsl:stylesheet>
Second route resolution action
The next transformation action uses another XSLT script to query the rule server, in case of a miss in the previous stage. The input is not used (an empty XML document) and the output is ignored. The XSLT document used by the action is using a single parameter, managerURL, which is the route-resolving Web service URL.
The first activity performed by the action’s XSLT script is to check for an existing routing decision in the context variable, var://context/dw/route. If the routing was not resolved, a SOAP request is created and sent to the secondary route-resolving Web service:
- The Web service request has one input parameter, the request URL.
- The response contains the destination URL, or null in case of failure.
The discovered destination is stored in the same WebSphere DataPower context variable, var://context/dw/route. The action XSLT file is shown in Listing 3.
Figure 6. Second route-resolving action
Listing 3. Second route-resolving XSL file
<?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" xmlns:dpconfig="http://www.datapower.com/param/config" xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ejbs="http://ejbs" extension-element-prefixes="dp" exclude-result-prefixes="dp dpconfig SOAP"> <!-- URL of the Manager, e.g. http://lnx-dw.haifa.ibm.com:9081/RouterManagerWEB/ services/WebServiceDestinationSOAPRouting --> <xsl:param name="dpconfig:managerURL" /> <xsl:template match="/"> <xsl:if test="not(dp:variable('var://context/dw/route'))"> <xsl:variable name="call"> <SOAP:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <SOAP:Body> <ejbs:getWSDestinationForUrl> <pUrl> <xsl:copy-of select="dp:variable('var://service/URL-in')"/> </pUrl> </ejbs:getWSDestinationForUrl> </SOAP:Body> </SOAP:Envelope> </xsl:variable> <xsl:variable name="result" select="dp:soap-call($dpconfig:managerURL, $call)" /> <xsl:if test="nilled($result)"> <dp:set-variable name="'var://context/dw/route'" value="string($result)" /> </xsl:if> </xsl:if> </xsl:template> </xsl:stylesheet>
Route definition action
The last transformation action executes the XSLT file script shown in Listing 4. Once again, the action checks that var://context/dw/route was defined by either of the previous route actions, and sets WebSphere DataPower service variables var://service/routing-url and var://service/URI accordingly. The setting of these two special variables triggers a redirect to a URL wherein var://service/routing-url defines the protocol/host/port portion and var://service/URI sets the destination path. (For a full list of special variables, see the WebSphere DataPower documentation in Resources.) If a destination was not found, the action sets an error message and rejects the request.
Once this transformation action completes, the final action is a results action, which informs the device that the processing rule has completed and the message can be sent onto is (new) destination.
Listing 4. Route definition XSL file
<?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" xmlns:dpconfig="http://www.datapower.com/param/config" extension-element-prefixes="dp" exclude-result-prefixes="dp dpconfig"> <xsl:template match="/"> <xsl:variable name="destination" select="dp:variable('var://context/dw/route')" /> <xsl:choose> <xsl:when test="$destination"> <dp:set-variable name="'var://service/routing-url'" value="$destination" /> <dp:set-variable name="'var://service/URI'" value="$destination" /> </xsl:when> <xsl:otherwise> <dp:set-variable name="'var://context/dw/errorMessage'" value="'Undefined rule'" /> <dp:reject override="true">Undefined rule</dp:reject> </xsl:otherwise> </xsl:choose> </xsl:template> </xsl:stylesheet>
XML Manager definition
To prevent numerous fetch requests, we can configure an XML Manager to cache the fetch action results. According to the definition in Figure 8, all documents from lnx-test.haifa.ibm.com will be cached for 900 seconds.
Figure 7. Caching configuration
The approach presented in this article uses the WebSphere DataPower extension mechanisms to provide a high performance, externally-managed, and flexible routing mechanism. Although the URL is used as the sole routing parameter in the context of this article, other parameters can also be used. For example, you could instead base the routing decision on information from the inbound request payload (parameters) and headers.
While this article showcased an ad-hoc multi-box management technique, IBM Tivoli Composite Application Manager System Edition provides a far more robust management solution that can also integrate easily with the rest of your management infrastructure.
- Communicate between stylesheets using WebSphere DataPower context variables
- WebSphere DataPower SOA Appliances product information
- IBM Tivoli Composite Application Manager System Edition for WebSphere DataPower V6.1.0 product information
- Redbook: IBM WebSphere DataPower SOA Appliances Part I: Overview and Getting Started
- Redbook: IBM WebSphere DataPower SOA Appliances Part II: Authentication and Authorization
- Redbook: IBM WebSphere DataPower SOA Appliance Part III: XML Security Guide
- WebSphere DataPower blog
- IBM WebSphere DataPower Community Forum
- IBM developerWorks WebSphere SOA zone