The Python Web services developer: The power of three: Python, Web services, and XSLT

Using Python XSLT API for Web services development

Mike Olson, Principal Consultant, Fourthought, Inc.

Mike Olson is a consultant and co-founder of Fourthought Inc., a software vendor and consultancy specializing in XML solutions for enterprise knowledge management applications. Fourthought develops 4Suite, an open source platform for XML middleware. You can contact Mr. Olson at mike.olson@fourthought.com.

Uche Ogbuji (uche@ogbuji.net), Principal Consultant, Fourthought, Inc.

Uche Ogbuji is a consultant and co-founder of Fourthought Inc., a software vendor and consultancy specializing in XML solutions for enterprise knowledge management applications. Fourthought develops 4Suite, open source platforms for XML middleware. Mr. Ogbuji is a Computer Engineer and writer born in Nigeria, living and working in Boulder, Colorado, USA. You can contact Mr. Ogbuji at uche@ogbuji.net.

Summary: This column has covered the major Python APIs for Web services processing. These cover a decent amount of ground, although the enthusiasm for Web services development has been notably muted in the Python community compared to that of, say, the Java community. One way to augment the Python features for Web services processing is to take advantage of the most popular specialized language for XML processing: XSLT. Web services involve XML in description and on the wire. Inevitably there has been some work on XSLT processing of Web services formats. IBM developerWorks hosted some pioneering work in the area with Uche Ogbuji's article, "WSDL processing with XSLT" (see Resources), and there has been a lot of other development since then. Python offers several good tools for XSLT processing (see Resources), and you can tap into this mine to enhance Web services processing capabilities. This article will show you how. You should be familiar with XSLT in order to follow the examples. You can use 4XSLT and the 4XSLT API for the examples.

Tag this!

Date: 24 Jun 2003
Level: Intermediate

Activity: 5693 views
Comments:

A Kafka-esque soap end-point

"Kafka - XSL SOAP Toolkit," by Christopher Dix, is an XSLT framework for constructing SOAP end-points. It only covers SOAP 1.1, but Kafka end-points have demonstrated the ability to pass the UserLand SOAP Validator, and it looks as if an update to SOAP 1.2 wouldn't be too difficult. Listing 1 shows one of the sample Kafka end-points: a SOAP server that adds two numbers together (a typical, simple SOAP example).

Listing 1. Kafka SOAP end-point for adding two numbers

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
  xmlns:method="http://www.topxml.com/"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
>
   <!-- add.xsl : Kafka SOAP Endpoint Example, with modifications -->

   <!-- Import soap.xsl to use the framework -->
   <xsl:import href="kafka/soap.xsl"/>
   <xsl:output method="xml" encoding="utf-8" omit-xml-declaration="yes"/>

   <!-- Define the global variables for the framework -->
   <xsl:variable name="Method">Add</xsl:variable>
   <xsl:variable name="MethodNS">http://www.topxml.com/</xsl:variable>

   <!-- Add : Add two numbers and return the sum -->
   <!-- Function Add( A as Double, B as Double ) as Double -->
   <xsl:template name="ProcessPayload">
      <xsl:param name="Payload"/>
      <xsl:for-each select="$Payload">
         <!-- This is how to retrieve parameters from the input -->
         <xsl:variable name="A" select="number(A|method:A)"/>
         <xsl:variable name="B" select="number(B|method:B)"/>

         <!-- The WriteParameter template takes the qualified name
              for a response parameter as well as its value and
              a QName specifying the tpe (for the xsi:type attribute) -->
         <xsl:call-template name="WriteParameter">
            <xsl:with-param name="p" select="'Result'"/>
            <xsl:with-param name="v" select="$A + $B"/>
            <xsl:with-param name="t" select="'xsd:double'"/>
         </xsl:call-template>
      </xsl:for-each>
   </xsl:template>
   
</xsl:stylesheet>

The XSLT end-point imports the SOAP framework (in the file, kafka/soap.xsl), and sets up parameters that the framework will use and the templates it will dispatch as it processes the entire XML document that makes up the SOAP message. The global variables Method and MethodNS declare the XML element that makes up the message. After handling the SOAP envelope, the framework invokes the ProcessPayload template passing in the XML body of the payload. The xsl:for-each is a standard trick to shift the context to the desired node. The parameters A and B are read from this element using simple XPaths and the framework is invoked again to help write out the response parameters. The WriteParameter template lets you specify an element name, data type, and value for each output parameter. The response value in this case is the result of adding the two input parameters.

Deploying this end-point as a server is a matter of setting up an HTTP listener. Python's BaseHTTPServer module provides you with the machinery you need to handle the HTTP portions of the protocol easily enough. See Listing 2.

Listing 2. Python HTTP framework for the Kafka SOAP end-point implemented in listing 1

#HTTP Listener code for SOAP server

import BaseHTTPServer

#The processor class is the core of the XSLT API
from Ft.Xml.Xslt import Processor

#4XSLT uses an InputSource system for reading XML
from Ft.Xml import InputSource

SOAP_IMPL_FILE = "add.xsl"

class KafkaSoapHandler(BaseHTTPServer.BaseHTTPRequestHandler):
    def init(cls):
        from Ft.Lib import Uri
        #Set up a processor instance to use
        KafkaSoapHandler.processor = Processor.Processor()
        #Load it with add.xsl
        add_uri = Uri.OsPathToUri(SOAP_IMPL_FILE, attemptAbsolute=1)
        transform = InputSource.DefaultFactory.fromUri(add_uri)
        KafkaSoapHandler.processor.appendStylesheet(transform)
        #Now the processor is prepped with a transform and can be used
        #over and over for the same transform
        return
    #Make init() a static method of the class
    init = classmethod(init)

    def do_POST(self):
        clen = self.headers.getheader('content-length')
        if clen:
            clen = int(clen)
        else:
            print 'POST ERROR: missing content-length'
            return

        if self.path != '/add':
            self.send_error(404)
        input_body = self.rfile.read(clen)
        #input_body is the request SOAP envelope and contents
        response_body = self._run_through_kafka(input_body)
        #response_body is the response SOAP envelope and contents
        self._send_response(200, 'OK', response_body)
        return

    def _run_through_kafka(self, body):
        #In 4Suite all InputSources have base URIs in case they refer to
        #other URIs in some way and resolution is required.
        #The SOAP messages will not have any such URI references,
        #So use a dummy base URI
        source = InputSource.DefaultFactory.fromString(body, "urn:dummy")
        response = self.processor.run(source)
        return response

    def _send_response(self, code, msg, body):
        #Prepare a normal response
        self.send_response(200, 'OK')
        #Send standard HTP headers
        self.send_header('Content-type','text/html; charset=utf-8')
        self.send_header("Connection", "close")
        self.send_header("Accept-Ranges", "bytes")
        self.send_header('Content-length', len(body)-1)
        self.end_headers()
        #Send the response prepared by the SOAP end point
        self.wfile.write(body)
        return
 

listen_on_port = 8888
#Set up to run on local machine
server_address = ('127.0.0.1', listen_on_port)
KafkaSoapHandler.init()
httpd = BaseHTTPServer.HTTPServer(server_address, KafkaSoapHandler)
print "Listening on port", listen_on_port
#Go into a the main event loop
httpd.serve_forever()

We commented the listing heavily, so it should be easy to follow. Notice how simple the code is because all it has to do is take care of the HTTP part of things and defer the XML and SOAP smarts to the Kafka framework. The server is specialized to one end-point, and thus it only has to parse and set up the XSLT transform once, and then it can simply run the transform over and over again for each new request. This is why the setup of the processor is moved to a special class method which is called as soon as the handler is registered with the server. The classmethod built-in is new in Python 2.2, and indeed this is the required version for this and following examples. It provides an implicit class object (cls) on which you can attach static data, such as the prepared processor instance, which is then available normally through the self instance reference on normal methods.

We tested the end-point using the recent release of SOAPpy 0.10.1 (see Resources), with a lot of nice, new features we will cover soon in this column. Listing 3 is a SOAPpy client that exercises the end-point. Open one command shell and run python listing2.py for the server. Then open up another shell and run python listing3.py, which should report the correct response as Add result: 7.0.

Listing 3: SOAPpy client for adding two numbers

import SOAPpy
ENDPOINT = "http://localhost:8888/add"
ADD_NS = "http://www.topxml.com/"
 
remote = SOAPpy.SOAPProxy(ENDPOINT, namespace=ADD_NS)
print "Add result:", remote.Add(A=3, B=4)

Using the description

As we said earlier, a useful property of Web services is that not only are the payloads in XML, but the descriptions are as well. Listing 4 is a WSDL file for the addition service, modified from Christopher Dix's original. It is WSDL 1.1.

Listing 4. WSDL for addition service

<?xml version="1.0" encoding="UTF-8"?>
<definitions name="adder"
  targetNamespace="http://www.topxml.com/"
  xmlns:tns="http://www.topxml.com/"
  xmlns:xsd="http://www.w3.org/1999/XMLSchema"
  xmlns="http://schemas.xmlsoap.org/wsdl/">

  <message name="Add">
    <part name="A" type="xsd:double" />
    <part name="B" type="xsd:double" />
  </message>
  <message name="AddResponse">
    <part name="param" type="xsd:double" />
  </message>

  <portType name="adder-port-type">
    <operation name="Add">
      <input message="tns:Add" />
      <output message="tns:AddResponse" />
    </operation>
  </portType>

  <binding name="adder-soap-binding" type="tns:adder-port-type"
           xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/">
    <soap:binding transport="http://schemas.xmlsoap.org/soap/http"
                  style="rpc"/>
    <operation name="Add">
      <soap:operation soapAction="http://tempuri.org/"/>
      <input>
        <soap:body use="encoded" namespace="http://www.topxml.com/"
          encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/>
      </input>
      <output>
        <soap:body use="encoded" namespace="http://www.topxml.com/"
          encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/>
      </output>
    </operation>
  </binding>

  <service name="adder-service">
    <port name="adder-port" binding="tns:adder-soap-binding">
      <soap:address location="http://127.0.0.1:8888/add"
        xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"/>
    </port>
  </service>

</definitions>

Listing 5 gives an XSLT script for rendering useful information for users of the end-point. It is adapted from a transform developed in an earlier developerWorks article, "WSDL processing with XSLT" (see Resources). It takes a number of liberties and shortcuts, especially in its treatment of qualified names in the WSDL context, but it would probably work for most WSDL 1.1 files in use.

Listing 5. XSLT script

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version='1.0'
   xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
   xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
   xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
>

  <xsl:output method='html'/>

  <!-- Lookup tables for messages, portTypes, bindings and services -->
  <xsl:key name='message' match="wsdl:definitions/wsdl:message"
           use='@name'/>
  <xsl:key name='port-type' match="wsdl:definitions/wsdl:portType"
           use='@name'/>
  <xsl:key name='binding' match="wsdl:definitions/wsdl:binding"
           use='@name'/>
  <xsl:key name='service' match="wsdl:definitions/wsdl:service"
           use='@name'/>

  <xsl:template match='/'>
    <html>
    <head>
      <title>
Service summary: <xsl:value-of select='wsdl:definitions/@name'/>
      </title>
      <meta http-equiv="content-type" content="text/html"
            charset="UTF-8"/>
    </head>
    <body>
       <h1>
Service summary: <xsl:value-of select='wsdl:definitions/@name'/>
       </h1>
       <p><xsl:value-of select='wsdl:definitions/@documentation'/></p>
       <xsl:apply-templates select="wsdl:definitions/wsdl:service"/>
    </body>
     </html>
   </xsl:template>

   <xsl:template match='wsdl:service'>
     <div style="background: #ccffff">
       Service "<xsl:value-of select='@name'/>" hosted at
     <code>
       <xsl:value-of select='wsdl:port/soap:address/@location'/>
     </code>

       <xsl:variable name="binding"
         select="key('binding',
                     substring-after(wsdl:port/@binding, ':'))"
       />
       <xsl:variable name="port-type"
         select="key('port-type',
                     substring-after($binding/@type, ':'))"
       />
       <xsl:apply-templates select="$port-type/wsdl:operation"/>
     </div>
   </xsl:template>

  <xsl:template match='wsdl:operation'>
    <p>Operation "<b><xsl:value-of select='@name'/></b>" message details:</p>
    <!-- Yes, should sue CSS, but keep this example simple -->
    <table border="1" width="50%">
      <tbody>
        <xsl:if test="wsdl:input">
          <xsl:call-template name='message-role'>
            <xsl:with-param name="role-node" select="wsdl:input"/>
          </xsl:call-template>
        </xsl:if>
        <xsl:if test="wsdl:output">
          <xsl:call-template name='message-role'>
            <xsl:with-param name="role-node" select="wsdl:output"/>
          </xsl:call-template>
        </xsl:if>
        <xsl:if test="wsdl:fault">
          <xsl:call-template name='message-role'>
            <xsl:with-param name="role-node" select="wsdl:fault"/>
          </xsl:call-template>
        </xsl:if>
      </tbody>
    </table>
  </xsl:template>

  <xsl:template name='message-role'>
    <xsl:param name="role-node"/>
    <xsl:variable name="role-name"
                  select="local-name($role-node)"/>
    <xsl:variable name="message"
      select="key('message',
              substring-after($role-node/@message, ':'))"
      />

    <tr>
      <td><xsl:value-of select='$role-name'/></td>
      <td>
        <table width="100%">
          <xsl:apply-templates select="$message/wsdl:part"/>
        </table>
      </td>
    </tr>
  </xsl:template>

  <xsl:template match='wsdl:part'>
    <tr>
      <td width="50%"><b><xsl:value-of select='@name'/></b></td>
      <td><xsl:value-of select='@type'/></td>
    </tr> 
  </xsl:template>

</xsl:stylesheet>

It is often convenient to serve up a human-friendly and WSDL description of a Web service on the same host as the service itself. Listing 6 is a variation on Listing 2 that does so. It actually provides three functions:

For GET requests on port 9000: Serves up a readable description of the Web service invocation messages
For GET requests on port 8888: Serves up the raw WSDL file
For POST requests on port 8888: Executes the SOAP request.

Listing 6. Variation on Listing 2

#HTTP Listener code for SOAP server

import BaseHTTPServer

#The processor class is the core of the XSLT API
from Ft.Xml.Xslt import Processor

#4XSLT uses an InputSource system for reading XML
from Ft.Xml import InputSource

SOAP_IMPL_FILE = "add.xsl"
WSDL_FILE = "listing4.xml"
HTML_VIEW_TRANSFORM = "listing5.xslt"

class KafkaSoapHandler(BaseHTTPServer.BaseHTTPRequestHandler):
    def init(cls):
        from Ft.Lib import Uri
        #Set up a processor instance to use
        cls.processor = Processor.Processor()
        #Load it with add.xsl
        add_uri = Uri.OsPathToUri(SOAP_IMPL_FILE, attemptAbsolute=1)
        transform = InputSource.DefaultFactory.fromUri(add_uri)
        cls.processor.appendStylesheet(transform)
        #Now the processor is prepped with a transform and can be used
        #over and over for the same transform

        #Prep for WSDL requests
        cls.wsdl = open(WSDL_FILE).read()
        return
    #Make init() a static method of the class
    init = classmethod(init)

    def do_POST(self):
        clen = self.headers.getheader('content-length')
        if clen:
            clen = int(clen)
        else:
            print 'POST ERROR: missing content-length'
            return

        if self.path != '/add':
            self.send_error(404)
        input_body = self.rfile.read(clen)
        #input_body is the request SOAP envelope and contents
        response_body = self._run_through_kafka(input_body)
        #response_body is the response SOAP envelope and contents
        _send_response(self, 200, 'OK', response_body)
        return

    def do_GET(self):
        #response_body is the WSDL file
        _send_response(self, 200, 'OK', self.wsdl)
        return
    
    def _run_through_kafka(self, body):
        #In 4Suite all InputSources have base URIs in case they refer to
        #other URIs in some way and resolution is required.
        #The SOAP messages will not have any such URI references,
        #So use a dummy base URI
        source = InputSource.DefaultFactory.fromString(body, "urn:dummy")
        response = self.processor.run(source)
        return response


class HtmlHandler(BaseHTTPServer.BaseHTTPRequestHandler):
    def init(cls):
        from Ft.Lib import Uri
        #Perform the transform once and store the result
        processor = Processor.Processor()
        html_desc_uri = Uri.OsPathToUri(HTML_VIEW_TRANSFORM,
                                        attemptAbsolute=1)
        transform = InputSource.DefaultFactory.fromUri(html_desc_uri)
        processor.appendStylesheet(transform)
        wsdl_uri = Uri.OsPathToUri(WSDL_FILE, attemptAbsolute=1)
        source = InputSource.DefaultFactory.fromUri(wsdl_uri)
        cls.html_desc = processor.run(source)
        return
    #Make init() a static class method
    init = classmethod(init)

    def do_GET(self):
        #response_body is the WSDL file
        _send_response(self, 200, 'OK', self.html_desc)
        return

#Turn _send_response into a global function
#for sharing between the classes
def _send_response(handler, code, msg, body):
        #Prepare a normal response
        handler.send_response(200, 'OK')
        #Send standard HTP headers
        handler.send_header('Content-type', 'text/html; charset=utf-8')
        handler.send_header("Connection", "close")
        handler.send_header("Accept-Ranges", "bytes")
        handler.send_header('Content-length', len(body)-1)
        handler.end_headers()
        #Send the response prepared by the SOAP end point
        handler.wfile.write(body)
        return
 
def soap_listener_function():
    listen_on_port = 8888
    #Set up to run on local machine
    server_address = ('127.0.0.1', listen_on_port)
    KafkaSoapHandler.init()
    httpd = BaseHTTPServer.HTTPServer(server_address, KafkaSoapHandler)
    print "Listening for GET and POST on port", listen_on_port
    #Go into a the main event loop
    httpd.serve_forever()

def html_listener_function():
    listen_on_port = 9000
    #Set up to run on local machine
    server_address = ('127.0.0.1', listen_on_port)
    HtmlHandler.init()
    httpd = BaseHTTPServer.HTTPServer(server_address, HtmlHandler)
    print "Listening for GET on port", listen_on_port
    #Go into a the main event loop
    httpd.serve_forever()
    return

import time
from threading import Thread

soap_thread = Thread(None, soap_listener_function)
html_thread = Thread(None, html_listener_function)

soap_thread.start()
#Pause before spawning the next thread
time.sleep(1)
html_thread.start()

You can handle GET and POST requests on a single server instance by defining both do_GET and do_POST on the server, but because of the nature of the simple event loop used, in order to listen on two separate ports, you can use threading. This lets you have two server instances running at the same time. Threading is one approach, and using an asynchronous event handler is another. Python 2.2. introduced the asyncore module for easier support of the latter technique, and we introduced this approach in the last article in this column (see Resources). This time we'll illustrate the use of threading. The Python 2.2. documentation gives a good suggestion on whether to use threading or asynchronous techniques.

[Asynchronous methods are] really only practical if your program is largely I/O bound. If your program is processor bound, then pre-emptive scheduled threads are probably what you really need. Network servers are rarely processor bound, however.

Figure 1 shows a browser view of the readable Web service description.

Figure 1. Browser display of the readable rendering of Listing 4 using the transform in Listing 5
Caption for sample figure

Wrap up

Consider this all fodder for experimentation. Kafka is rather dated -- it doesn't appear to have been maintained since 2001, and it uses rather poor XSLT style (the author freely admits to having been an XSLT neophyte). But the idea is very powerful, and worthwhile. It would require little effort to update it to SOAP 1.2 and extend its capabilities. The WSDL presentation transform we offer is also just a start. It could also be updated to WSDL 1.2 and could be extended to display more information about a Web service. It should also be updated to take advantage of the namespace axis and other XSLT features for more correct processing.

XSLT is a sandbox in which developers from all sorts of languages and environments can play. Kafka was developed by a staunch .NET developer, and yet we picked it up and put it to use very quickly. This is the power of having a processing lingua franca for XML and Web services. We expect that the field of available XSLT modules for Web services will continue to grow. If so, the basic techniques we presented in this article should position Python programmers to make immediate use of the bounty.

Resources

Participate in the discussion forum.
Read Uche Ogbuji's article "WDSL processing with XSLT," to learn more about XML's involvement in Web services (developerWorks, November 2000).
Get the recent release of SOAPpy 0.10.1 from Sourceforge.net.
Find the 4XSLT processor from 4Suite. See "Information for 4Suite users" for a good overview of the use of this API. There is also some information in "A Tour of 4Suite" (O'Reilly, October 2002).
Review the standard Python documentation for information on the new asynchronous socket handler in Python 2.2. Also see the threading module page on www.Python.org.
Read the previous column of this series on SOAP over SMTP (developerWorks, March 2003), or check out all the Python Web services developer columns in this series (developerWorks, January 2001).

About the authors

Comments

Trademarks | My developerWorks terms and conditions

Close [x]

Help: Update or add to My dW interests

What's this?

This little timesaver lets you update your My developerWorks profile with just one click! The general subject of this content (AIX and UNIX, Information Management, Lotus, Rational, Tivoli, WebSphere, Java, Linux, Open source, SOA and Web services, Web development, or XML) will be added to the interests section of your profile, if it's not there already. You only need to be logged in to My developerWorks.

And what's the point of adding your interests to your profile? That's how you find other users with the same interests as yours, and see what they're reading and contributing to the community. Your interests also help us recommend relevant developerWorks content to you.

View your My developerWorks profile

Return from help

Close [x]

Help: Remove from My dW interests

What's this?

Removing this interest does not alter your profile, but rather removes this piece of content from a list of all content for which you've indicated interest. In a future enhancement to My developerWorks, you'll be able to see a record of that content.

View your My developerWorks profile

Return from help

static.content.url=http://www.ibm.com/developerworks/js/artrating/

SITE_ID=1

Zone=SOA and Web services, XML

ArticleID=11814

ArticleTitle=The Python Web services developer: The power of three: Python, Web services, and XSLT

publish-date=06242003

author1-email=mike.olson@fourthought.com

author1-email-cc=

author2-email=uche@ogbuji.net

author2-email-cc=