![Close [x]](http://dw1.s81c.com/i/c.gif){kind=small}
In the first installment of the "Python Web services developer" column, we promised we'd show you lots of code. Unfortunately, when using Python out of the box, and many of the third party applications available in Python for Web services development, there is often little code to write. This is the case in this month's column as we put together an on-line information directory. The purpose of this directory is making software packages available for download. It supports real-time updating of the data and metadata about the software packages, searching the meta-data about the packages, and, of course, downloading the packages themselves.
Before continuing with this article, you should make sure you have an understanding of some of the technologies we will be talking about in this column. The technologies we will be using include Extensible Stylesheet Language Transformations (XSLT), XML Path Language (XPath), and the Resource Description Framework (RDF). There are links to information about all of these technologies in the Resources section.
An XML repository, 4Suite Server (4SS), co-developed by the authors, will be used as the application base for this example. 4Suite Server is an XML repository with many facilities for XML data and metadata management that make it very suitable for rapid development of Web Services, whether in Python or not.
The example in this article is written using 4Suite Server 0.11 and requires Python 1.5.2 or greater and 4Suite 0.11. Links to download all of these applications are available in the Resources section.
This article is the second in "Python Web Services Developer" column, but the first in a three part series on building an on-line software repository. In this segment, we will build our infrastructure. In the subsequent columns, we will feature searching indexed content as well as agent-based addition or retrieval of content using various protocols such as Simple Object Access Protocol (SOAP), HTTP, and WWW Distributed Authoring and Versioning (WebDAV).
Our online software repository service has a schema loosely based around the RDF schema in RPMFind.net. RPMFind is a system for cataloging packages of UNIX and Linux software in the popular Red Hat Package Manager (RPM) format. It contains key metadata about the software package, including author, version, and description, in the RDF format (see Listing 1). For a brief definition of RDF, read the previous issue of this column or go to the Resources section for a link to a basic introduction of this simple format.
The actual format of the XML is irrelevant. In fact, it doesn't even need to describe software, as the techniques described will work with any type of XML content. You can use this technique to describe a book catalog, employee information, or even a wine list.
All of the code and data files used in this example are downloadable from links in the Resources section.
In 4SS's XML repository, document definitions allow you to specify a mapping between XML content and RDF metadata. To do this, you need to define sets of three XPath expressions: a subject expression, a predicate expression, and an object expression. An XPath expression allows you to define a set of node relationships in the document and return subsets of content from the document based on these relationships. These XPath expressions are evaluated against each XML document as it is added, modified, and deleted from the repository. The resulting statements, also known as triples, are automatically added and removed from the RDF database (known as the model). If the document is modified, the triples are modified to reflect the changes, and if the document is deleted, then the triples are removed from the RDF server. Document definitions can inherit information from other document definitions allowing you to define a complex mapping of XML content to RDF metadata information.
In our example application, we will extend one of the default document
definitions. The default document definition describes a mapping between
Dublin Core tags embedded in XML content to Dublin Core statements. The
Dublin Core is a metadata initiative that defines a set of standard properties
for common Web-based objects such as Creator, Title, and Date. Our derived
document definition will add one more statement for each document.
A simple declaration, as shown in the following, sets this document's creator metadata to the result of evaluating a certain XPath:
RdfStatement(subject='$uri', predicate="http://purl.org/dc/elements/1.1#Title", object="/rdf:RDF/s:Software/dc:Creator") |
(The code above is a single line statement, but had to be indented to fit this format.)
To add or update the system default data, you should run the script
populate.py
that comes with 4SS. This downloads useful data from, ftp://ftp.fourthought.com
to update your server. The downloaded data include some commonly used items
like the Dublin Core document definition and a Docbook stylesheet (Docbook
is a popular XML format for technical documentation).
The populate script is automatically installed into your demo application
when 4SS is installed. On Unix-based machines, this is typically stored
in /usr/doc/4SuiteServer-0.11 or /usr/local/doc/4SuiteServer-0.11.
On Windows machines, these directories are typically c:\Program Files\Python
or c:\Python20. Listing 2 shows the installation
procedure for populating your 4SS-based application.
Listing 2: Populating your 4SS application
[molson@penny example]$ python /usr/doc/4SuiteServer-0.11/demo/populate.py Downloading XML Documents Downloading Stylesheets Downloading DocDefs Adding XML document: 'null' Adding stylesheet: 'docbook_html1.xslt' Adding stylesheet: 'presentation_toc.xslt' Adding stylesheet: 'presentation.xslt' Adding stylesheet: 'docbook_text1.xslt' Adding document definition: 'dublin_core' Adding document definition: 'docbook1' |
4ss deserialize docdef,
[molson@penny example]$ 4ss deserialize docdef software.docdef |
We will use 4ss create document to add new content to the system
from the command line. In the example download, there are two software
listings, the XML files called software1.rdf and software2.rdf.
To add these files to the system, we execute 4ss create document,
giving it the document definition to use, the name of the file to add, and a list of aliases to assign to the resource in the system.
[molson@penny example]$ 4ss create container /softrepo [molson@penny example]$ 4ss set acl --write=uo --world-read /softrepo |
Then we add our sample download files to the repository. The
4SS repository is highly optimized for storing XML data, although it can pretty
much store data in any format. When we add our .tar files to the
repository, we specify the --imt option to set the Internet Media Type (IMT) of the file (in our case application/x-gzip). This IMT can, among other things, be used by the HTTP server when retrieving the content over the Web. Note that IMT is sometimes called "MIME type". See Listing 3 for the commands to add the content. Note that in a more complex project, you might consider placing the binaries in a separate container.
software.xslt
[molson@penny example]$ 4ss create document BASE_XSLT software.xslt softrepo/software.xslt |
BASE_XSLT is a special document definition which tells 4SS to optimize this document as an XSLT stylesheet.
With the document added, you can now connect to the 4SS HTTP server (both plain Python and Apache servers are supported)
with your Web browser and go to the page http://localhost:8080/softrepo/pong.xml.
This will fetch the pong software description document from the repository.
If you are using a browser that understands the IMT text/xml
such as Internet Explorer or Mozilla, then you can view the XML that was
added to the repository. To tell the HTTP Listener that you would like
to render the page (by running through XSLT) before it is returned, specify the xslt URI query argument
http://localhost/softrepo/pong.xml?xslt=software.xslt.
Notice that the link on the page for the download of the package points
to localhost as well. This link will also go through the HTTP Listener
and fetch the resource that we added for pong-0.0.2.tgz. When
it is returned to the browser it will specify the IMT that we defined
when we added the resource to the system.
To generate an index page, we will use some extension functions of 4SS
for XSLT to access the RDF model. There are other solutions to the problem
of generating an index page in 4SS. One such solution is to write a custom
handler in Python for HTTP GET messages. Perhaps this would query the RDF model when index.html is requested. Another solution would be to use the 4SS event
system to update a index.html document whenever a new document is
added or removed from the system.
index.doc4ss create document
[molson@penny example]$ 4ss create document index.doc BASE_XML softrepo/index.doc [molson@penny example]$ 4ss set acl --world-read softrepo/index.doc |
We will use the extension function rdf.complete in our
stylesheet to gather information about all of the software in the system.
The extension function calls the complete method on the RDF model. The complete method allows you to search the RDF model for statements that
match a specified pattern. It takes up to three parameters: a subject,
predicate, and, optionally, an object. Any of these parameters can be an
empty string. It will return a list of statements that match all of the
specified values. For example, if you pass in a subject of foo,
and an object of bar it will return a list of statements that have
a subject of foo, any predicate, and an object of bar.
4SS automatically creates RDF statements linking documents to their
document definition. The subject of these statements is the document's URI,
the predicate is http://schemas.4suite.org/4ss#metaxml.docdef
and the object is the document definiton name. Knowing this, we use
a simple complete call specifying the predicate and object
to get a list of documents in our system using the software
document definition.
The stylesheet we use to generate the index is called index.xslt.
The template that matches the root of the source document has the first
occurrence of rdf.complete. This function call does a complete
operation on the RDF model for all of the statements that have
http://schemas.4suite.org/4ss#metaxml.docdef
as the predicate and software as the object. The
result of the rdf.complete function call is a node set of
Statement
elements. Each Statement element has three children: Subject,
Predicate, and Object. We use xsl:apply-templates
on the results of the function, and display each software item in the template
that matches on Statement, as shown in Listing
4.
To view the generated index page point your browser to http://localhost:8080/softrepo/index.doc?xslt=index.xslt.
This index is not designed for looks; however, it is easy to see how this simple page could be expanded to display the software titles in any fashion. How each item is displayed can be modified in the stylesheet, and the data available for the stylesheet can be adjusted by adding more mappings to the document definition.
Well, we still haven't written any Python code, but we did get a peek at some of the features of 4Suite Server. In next month's column we will expand on this example to give our software repository the content management features and searching all of the metadata we are generating.
- Participate in the discussion forum.
-
Download the example code for this article.
-
Visit the RPMFind.net site to learn more about it and to download tools and code.
-
Review the details of the RDF specification.
-
Check out the information available on the XSLT specification.
-
Learn more about the XPath specification.
-
To learn more about the 4Suite Server, read our article on Unix insider.
-
And, of course, you can always download the Python
interpreter and programming tools.
- Get an overview of the purpose of this series by reading the first installment.
- Read the third installment of this series.
- Read An introduction to RDF to learn more about RDF.
Mike Olson (molson@fourthought.com) and Uche Ogbuji (uche@fourthought.com) are co-founders of and principal consultants at Fourthought Inc., where they develop the open-source tools 4Suite and 4Suite Server , and provide commercial consulting, training, and 4Suite customization for clients working with XML and Web services. They reside in Boulder, Colorado.
Mike Olson (molson@fourthought.com) and Uche Ogbuji (uche@ogbuji.net) are co-founders of and principal consultants at Fourthought Inc., where they develop the open-source tools 4Suite and 4Suite Server , and provide commercial consulting, training, and 4Suite customization for clients working with XML and Web services. They reside in Boulder, Colorado.
Table of contents
Next steps from IBM
Rational Tester for SOA Quality is an automated web services testing and SOA testing tool that supports Web services SOAP, HTTP(S), JMS, WS-Security, UDDI, and RESTful standards.
- Try: The no-charge Rational Service Tester for SOA Quality helps quality assurance professionals validate web service based SOA application..
- Article: See the whats new with Rational Service Tester for SOA Quality, and its Generic Service Client, which provides a single client to interact with any SOA service type.
- Tutorial: Get hands-on experience with this Transport for SOAP feature tutorial using Rational Tester for SOA Quality.
- Buy: Rational Service Tester for SOA Quality
Tags
Use the slider bar to see more or fewer tags.
For articles in technology zones (such as Java technology, Linux, Open source, XML), Popular tags shows the top tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), Popular tags shows the top tags for just that product zone.
For articles in technology zones (such as Java technology, Linux, Open source, XML), My tags shows your tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), My tags shows your tags for just that product zone.
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
