Two issues ago (see Resources), we took an updated look at ZSI, a Python web services library that was covered in the past. This month, we shall look at updates in another such library. SOAPpy is another component of the "Web Services for Python" project. Version 0.11.3 is the latest and, as with ZSI, the most recent set of releases have added an impressive array of improvements and fixes. We have also been using SOAPpy to access Google's and Amazon's web services in our recent articles on real-world web services (see Resources).

Installing the software

We downloaded SOAPpy 0.11.3 for installation in Python 2.3.3, but before installing it, we had to download and install the following prerequisites:

• fpconst - A Python library for special handling of IEEE 754 floating point numbers. Version 0.6.0 or more recent of fpconst is a new prerequisite for SOAPpy. We installed 0.6.0.
• pyXML 0.8.3 or more recent is also required.

The software installed fine using the now standard python setup.py install from the unpacked directories. You can also install the following software in order to enable special capabilities. However, we did not choose to install the software.

• pyGlobus to enable support for the Globus Alliance toolkit for grid computing (currently a substantial subset of the 2.2.4 and 2.4 versions of the Globus toolkit).
• M2Crypto.SSL to enable support for SSL on SOAP servers. To enable SSL support for SOAP clients, you need SSL support compiled into Python. This is the default for Python 2.3+.

Back to top

WSDL and a few more dateTime hiccups

As with ZSI, the recent feature of SOAPpy is WSDL support. The low-level APIs for SOAP access in SOAPpy are quite easy to use, but WSDL offers the promise of even less set-up code.

We tried to invoke a web service that gave us some difficulty with ZSI 1.4.1. As we reported in the last article ("Python SOAP libraries, Part 4"):

Richard Hastings' Air Fare Quote Search is implemented in Apache Axis and searches some airline web sites in real time to find the best available flight prices for a given itinerary (see resources). It aggregates and returns the results, sorted by price. The WSDL is at http://wavendon.dsdata.co.uk:8080/axis/services/SBGGetAirFareQuote?wsdl. It defines two operations: getAirFareQuote and getAirlines. The former is used to execute the price search and takes 4 parameters: two W3C XML Schema Language Data Types (WXSDT) dateTime values giving the approximate start of the departure and end of the return flights and two WXSDT string values giving the three-letter airport codes between which the travel is to take place.

The set-up code is straightforward enough:

>>> import SOAPpy
>>> wsdl = 'http://wavendon.dsdata.co.uk:8080\
... /axis/services/SBGGetAirFareQuote?wsdl'
>>> proxy = SOAPpy.WSDL.Proxy(wsdl)

      

We attempted the straightforward positional parameter method invokation that we tried with ZSI. We had already entered the two dateTime parameters following the two string parameters right into a getAirFareQuote method invokation. So, we had to figure out how to create dateTime types. The document docs/simpleTypes.txt suggested creating instances of SOAP.DateTime, but this gives an AttributeError. As a result, we had to use SOAP.dateTimeType. This takes a Python time tuple. We created this tuple from ISO-8601 strings for readibility and picked parameters to give us a fare for travel to the upcoming PyCon in Washington, D.C., USA.

>>> proxy.getAirFareQuote(SOAPpy.dateTimeType(dep), SOAPpy.dateTimeType(ret), 'den', 'phl')
>>> import time
>>> ISO_8601_DATETIME = '%Y-%m-%dT%H:%M:%S'
>>> dep = time.strptime('2004-03-24T12:30:59', ISO_8601_DATETIME)
>>> dep_dt = SOAPpy.dateTimeType(dep)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/home/uogbuji/lib/lib/python2.3/site-packages/SOAPpy/Types.py", line 79, in __init__
    self._data = self._checkValueSpace(data)
  File "/home/uogbuji/lib/lib/python2.3/site-packages/SOAPpy/Types.py", 
    line 442, in _checkValueSpace
    raise ValueError, "invalid %s value - %s" % (self._type, e)
ValueError: invalid dateTime value - invalid type
>>> 

      

After some trial and error to debug this rather cryptic error, we learned that SOAPpy does not like Python time tuples with -1 in the tm_isdst field. When this value is used, it generally indicates that the user wants the library to handle daylight savings time using sensible defaults. Since SOAPpy cannot handle the Python time tuples, we had to force everything to Greenwich Mean Time (GMT).

Back to top

We need a little structure

We were finally able to get through to the remote server after adjusting the dates to use GMT:

>>> import time
>>> dep = SOAPpy.dateTimeType((2004, 3, 24, 12, 30, 59, 4, 86, 0))
>>> ret = SOAPpy.dateTimeType((2004, 3, 26, 12, 30, 59, 4, 86, 0))
>>> proxy.getAirFareQuote(dep, ret, 'den', 'iad')


					... Long, bewildering SOAP fault snipped ...
				
			

Python's time.strptime, which has been the subject of well-deserved abuse by many developers, has no way to handle proper ISO 8601 time zone offsets. The closest format specifier, %Z, which only accepts civil time zone names such as MST, is not permitted in ISO-8601. In the end, we just used the ugly time tuples directly. A SOAP message was constructed and sent to the server. We were unable to get that far with ZSI until after the recent publication of the article (click here for an update). Unfortunately, the server was not obliging. It sent back an extremely confusing SOAP fault--a huge Java Stack trace from which the key line was: org.xml.sax.SAXException: No such operation 'getAirFareQuote. We knew something was fishy about the fault message because we could use an on-line SOAP debugger to access the getAirFareQuote method. Indeed, if we tried a little experiment:

>>> proxy.getAirFareQuote()
[]
>>>

      

We did get a result. The fault message was a confusing way of saying "I don't understand the way you are invoking this method." Googling around a bit led to the clue that we should read the WSDL more closely. The server was actually expecting a multi-ref defined in a structure called in0, comprising the four parameters we had been passing directly. So, we needed to deal with structure types. The document docs/complexTypes.txt was no help at all in finding out how to do so. We had to pore through SOAPpy code to find the probable class we needed, SOAPpy.structType(). A little experimentation led us to:

>>> in0 = SOAPpy.structType()
>>> in0._addItem('outwardDate', dep)
>>> in0._addItem('returnDate', ret)
>>> in0._addItem('originAirport', 'den')
>>> in0._addItem('destinationAirport', 'iad')

      

This time we got a different SOAP fault -- a Java traceback with initial message org.xml.sax.SAXException: No deserializer defined for array type {http://www.w3.org/1999/XMLSchema}ur-type. Time to see what WSDL was actually sending on the wire. We turned on the SOAPpy's debugging facilities. Luckily, unlike in the 0.9x versions of SOAPpy, there is a simple way of turning on and off the debugging code by setting SOAPpy.Config.debug to 0 or 1.

>>> SOAPpy.Config.debug = 1
>>> proxy.getAirFareQuote(in0)

      

This put out a huge amount of text, but we found the outbound SOAP payload buried in the output.

*** Outgoing SOAP ******************************************************
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
  SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
  xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
  xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"
  xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns:xsd3="http://www.w3.org/2001/XMLSchema"
  xmlns:xsd="http://www.w3.org/1999/XMLSchema">
<SOAP-ENV:Body>
<ns1:getAirFareQuote
  xmlns:ns1="urn:SBGAirFareQuotes.sbg.travel.ws.dsdata.co.uk"
  SOAP-ENC:root="1">
<v1 href="#i1"/>
</ns1:getAirFareQuote>
<xsd:v1 id="i1" SOAP-ENC:root="0">
<outwardDate href="#i2"/>
<returnDate href="#i3"/>
<originAirport href="#i4"/>
<destinationAirport href="#i5"/>
</xsd:v1>
<outwardDate SOAP-ENC:arrayType="xsd:ur-type[4]" xsi:type="SOAP-ENC:Array"
             SOAP-ENC:root="0" id="i2">
<item href="#i3"/>
<item href="#i3"/>
<item href="#i4"/>
<item href="#i5"/>
</outwardDate>
<returnDate xsi:type="xsd3:dateTime" id="i3"
            SOAP-ENC:root="0">2004-03-24T12:30:59Z</returnDate>
<originAirport xsi:type="xsd:string" id="i4"
            SOAP-ENC:root="0">den</originAirport>
<destinationAirport SOAP-ENC:arrayType="xsd:string[2]"
            xsi:type="SOAP-ENC:Array" SOAP-ENC:root="0" id="i5">
<item href="#i6"/>
<item href="#i7"/>
</destinationAirport>
<item xsi:type="xsd:string" id="i6" SOAP-ENC:root="0">phl</item>
<item xsi:type="xsd:string" id="i7" SOAP-ENC:root="0">iad</item>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
************************************************************************

      

We have added some new lines and indentation for formatting reasons. This is clearly a complete mess. The W3C XML Schema namespace and the basic data structuring is from long-obsolete versions of SOAP proposals. It is hard to imagine how to convert this into anything a current SOAP implementation would recognize without a huge amount of effort. Scouring through the SOAPpy code for flags and options that might help did not turn up anything very encouraging. At this point, we gave up trying to access the airfare quote service. Certainly, we could access the getAirlines method easily enough:

>>> SOAPpy.Config.debug = 0
>>> proxy.getAirlines()
['Alitalia', 'American Airlines', 'BMI', 'BMIBaby', 'British Airways',
 'Continental', 'EasyJet', 'EBookers', 'Expedia', 'Global Traveller',
 'Lufthansa', 'Northwest Airlines', 'Maersk Air', 'Opodo', 'Qantas',
 'Ryanair', 'Star Alliance', 'Travelocity', 'United']
>>> 

      

This, of course, is back to the trivial flavor of web services we had been hoping to move beyond. The WSDL module is a welcome addition to SOAPpy and ZSI because it simplifies the process of working out the low-level details of the service. However, it does not help any if you need the complex data structure marshalling that Python SOAP implementations seemingly have not caught up to yet. Again, we would like to emphasize that this is an indictment of how terribly arcane SOAP RPC complex types are. This is a problem the entire SOAP community urgently needs to address. Luckily, with the emergence of the document/literal flavor of SOAP, there is some hope for sanity.

Back to top

A ZSI update

After our last article was published, the core developer of ZSI, Rich Salz, sent an email indicating the following:

At some point, the return value of time.strptime() changed from a tuple to a <type 'time.struct_time'>. [...]The following fragment -- note the tuple() calls for dep and ret -- [gets] further.

A ZSI bug? Python advancing too quickly? Your call ...

We tried out the modified code he suggested:

>>> from ZSI import ServiceProxy
>>> wsdl = 'http://wavendon.dsdata.co.uk:8080\
... /axis/services/SBGGetAirFareQuote?wsdl'
>>> proxy = ServiceProxy(wsdl)
>>> import time
>>> ISO_8601_DATETIME = '%Y-%m-%dT%H:%M:%S'
>>> dep = tuple(time.strptime('2003-12-06T12:30:59', ISO_8601_DATETIME))
>>> ret = tuple(time.strptime('2003-12-12T12:30:59', ISO_8601_DATETIME))
>>> proxy.getAirFareQuote(dep, ret, 'den', 'phl')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/home/uogbuji/lib/lib/python2.3/site-packages/ZSI/ServiceProxy.py", 
    line 82, in __call__
    return self.parent()._call(self.__name__, *args, **kwargs)
  File "/home/uogbuji/lib/lib/python2.3/site-packages/ZSI/ServiceProxy.py", 
    line 65, in _call
    apply(getattr(binding, callinfo.methodName), args)
  File "/home/uogbuji/lib/lib/python2.3/site-packages/ZSI/client.py", line 28, in __call__
    requesttypecode=TC.Any(self.name, aslist=1))
  File "/home/uogbuji/lib/lib/python2.3/site-packages/ZSI/client.py", line 132, in RPC
    return self.Receive(replytype, **kw)
  File "/home/uogbuji/lib/lib/python2.3/site-packages/ZSI/client.py", line 261, in Receive
    raise FaultException(msg)
ZSI.FaultException: org.xml.sax.SAXException: No such operation 'getAirFareQuote'
>>> 

      

The use of the modified code got us to the point of getting the SOAP request transmitted to the server, but we were back to figuring out how to send out the precise structure the remote method is expecting.

Back to top

Wrap-up

Even though our efforts to access the air fare web service were frustrating, we can see that ZSI and SOAPpy have made important strides. Our intent in these articles are not to disparage these packages, but to lay out the traps we fell into so that others might avoid them. We do believe that a lot of the problem here is that SOAP is far too complex as soon as you venture outside the realm of trivial web services. As we mentioned, the growing acceptance of document/literal message encodings reduces the problems of data architecture to the same ones that are well understood in XML. Luckily, there is a lot that you can accomplish without venturing into the snake-pit of complex structure types. In recent articles, we have had success accessing Google's and Amazon.com's web services APIs using SOAPpy (and some testing indicates that ZSI would work as well). Even these commercial services avoid complex structure types. There is no reason the air fare quotes service could not just accept simple positional parameters. You should be able to use ZSI and SOAPpy for most SOAP tasks you come across, but it is important to be aware of their limitations.

Resources

• Participate in the discussion forum.
  
  
• Python SOAP libraries, part 4 talks about ZSI V1.4.1 with its added support for WSDL.
  
  
• In The real world, part 1 we took a look at how to access Google's web services in Python, while The real world, part 2 examined Amazon's services.
  
  
• Keep an eye on Python web Services, the SourceForge project under which ZSI and SOAPpy are currently hosted.
  
  
• See Richard Hastings' Air Fare Quote Search, which is implemented in Apache Axis and searches some airline web sites in real time to find the best available flight prices for a given itinerary.
  
  
• The weblog entry "Working with multirefs, Axis, and SOAP::Lite" suggests that Python web services implementations were not the only ones having trouble with the air fare quote service.
  
  
• Follow up on document/literal style for SOAP in the article "Reap the benefits of document style web services" by James McCarthy (developerWorks, June 2002).
  
  
• Participate in the discussion forum for this article. (You can also click Discuss at the top or bottom of the article to access the forum.)
  
  
• Check out all the Python web services developer columns in this series.
  
  

About the authors

Scott Archer is a software architect and co-founder of GlowingOrb, Inc., a software tools developer focusing on model-driven solutions and their integration into core business processes. Mr. Archer holds an M.Phil in Computational Molecular Biology from the University of Hong Kong. You can contact Mr. Archer at scott.archer@glowingorb.com.

Uche Ogbuji is a consultant and co-founder of Fourthought Inc., a software vendor and consultancy specializing in XML solutions for enterprise knowledge management. Fourthought develops 4Suite, an open source platform for XML, RDF, and knowledge-management applications. Mr. Ogbuji is also a lead deveoper of the Versa RDF query langage. He is a Computer Engineer and writer born in Nigeria, living and working in Boulder, Colorado, USA. You can contact Mr. Ogbuji at uche@ogbuji.net.

Rate this article

Comments

Back to top

Table of contents

• Installing the software
• WSDL and a few more dateTime hiccups
• We need a little structure
• A ZSI update
• Wrap-up
• Resources
• About the authors
• Comments