In the previous three articles in this series, I covered the development of an XML/RDF vocabulary, Description of a Project (DOAP), for describing open source projects and related resources. Through the use of DOAP, software maintainers will no longer have to register their programs at multiple Web sites. Instead, they can simply give the URL of the DOAP description. As more applications become DOAP-aware, new possibilities open up for participation in and adminstration of open source projects.

To reach these goals, it's important to do more than merely create the vocabulary. In this concluding article, I look at what's needed to get adoption for DOAP, in terms of documentation, tools, and community.

Viewing DOAP

To refresh your memory, here's a simple DOAP file -- Listing 1 shows a minimal DOAP file for the DOAP project itself.

<Project xmlns="http://usefulinc.com/ns/doap#"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/">

    <name>DOAP</name>
    <homepage rdf:resource="http://usefulinc.com/doap" />
    <created>2004-05-04</created>
    <shortdesc xml:lang="en">
        Tools and vocabulary for describing community-based
        software projects.
    </shortdesc>
    <description xml:lang="en">
        DOAP (Description of a Project) is an RDF vocabulary and
        associated set of tools for describing community-based software
        projects.  It is intended to be an interchange vocabulary for
        software directory sites, and to allow the decentralized
        expression of involvement in a project.
    </description>
    <maintainer>
        <foaf:Person>
            <foaf:name>Edd Dumbill</foaf:name>
            <foaf:homepage rdf:resource="http://usefulinc.com/edd" />
        </foaf:Person>
    </maintainer>
</Project>

The first and most basic requirement for users of DOAP is to get a user-friendly view of this data. If no other editing mechanisms exist, you can edit a file by hand and then use the viewer to check the file -- many Web pages have been created with this time-honored and admittedly problematic method. The quickest way to create a viewer would probably be to write an XSLT stylesheet to transform the incoming RDF/XML into HTML.

However, DOAP is more than just an XML vocabulary, which means using XSLT isn't a straightforward decision. The viewer will serve more than one purpose: It not only provides viewing of DOAP data, but it is also the first example of a DOAP processing application. As such, many implementors coming to DOAP for the first time will take it as an example. Therefore, it's helpful if the DOAP viewer is an RDF-aware application.

To create a viewer, I used the Redland RDF Application library to read the DOAP file into an in-memory RDF store and extract the data from the store into XML, which I then formatted with XSLT. This intermediate XML representation can either be transformed into HTML on a server-side app, or can be used to drive a graphical user interface for a client-side DOAP viewer. Figure 1 shows the transformed HTML output from the viewer.

To create the viewer, I used the Mono/.NET bindings from Redland (see Resources). The code snippet in Listing 2 shows a loop to process every project in a DOAP file (normally, there's only one) and output data inside a <project> element in an XML document. The rdf ["type"] and doap ["Project"] variables are shortcuts for resource nodes with URIs corresponding to the terms in the RDF and DOAP schemas.

foreach (Node proj in model.GetSources (rdf ["type"], doap ["Project"])) {
    w.WriteStartElement (null, "project", null);

    Node name = model.GetTarget (proj, doap ["name"]);
    w.WriteStartElement ("name");
    w.WriteString (name.Literal);
    w.WriteEndElement ();
    ...
    w.WriteEndElement ();
}

Another interesting option for writing a DOAP viewer is to use the RDF rendering support inside the Mozilla Web browser and create a XUL-based DOAP viewer. This could lead to an intriguing scenario for extensions in a browser such as Firefox.

Back to top

Validating DOAP

The ability to validate DOAP files is an essential part of processing them. Validation is useful in both the creation and consumption of DOAP. For creators, whether they're writing DOAP by hand or creating tools to output it, a validator shows compliance to the specification. For consumers of DOAP, validation is required so that software doesn't end up processing junk.

At its most basic, validation is simply the process of reporting "yes" or "no" as to whether an input file meets the specification. More helpful validators report errors and recommendations. Perhaps the most well known of these is the W3C HTML Validator (see Resources), which returns all manner of helpful information for improving your Web pages' compliance to W3C specifications.

Because DOAP is RDF/XML, it can be validated at a variety of levels:

• XML: DOAP must be well-formed XML.
• RDF: DOAP must be valid RDF.
• Semantic: The document must contain enough DOAP terms for it to make sense; for instance, a DOAP file would be pretty useless without a name, description, and homepage property.

Traditionally, pure XML validation techniques only extend to syntactic validation -- that is, the presence or absence of elements and attributes, and their ordering. While this catches a lot of errors, it doesn't help with scenarios where data processing is required in order to determine that something nonsensical, yet syntactically valid, has been written. However, certain semantic constraints can be expressed as syntactic ones. For instance, one DOAP requirement states that there must be a "homepage" property, so you can use an XML schema to catch those.

The problem with using XML schemas to validate RDF is that RDF has quite a flexible XML syntax, with more than one way of writing the same thing. Indeed, it's not possible to validate RDF with W3C XML Schema; it's easier just to run it through an RDF parser, such as Redland's raptor, to check its RDF-validity. Once you know that an incoming DOAP file is valid RDF, then you can apply a sequence of semantic tests to determine the quality of the rest of the file.

This strategy works fine as long as RDF processing tools are available to everyone, or as long as you're willing to provide a free Web service to validate DOAP. However, you can sacrifice some of RDF's flexibility of syntax to achieve greater ubiquity by providing an XML schema that achieves the first 70% of the validation process and uses syntactic methods to verify a certain amount of semantic integrity. This, together with a bit of common sense, may be enough for a lot of people.

Working along those lines, I constructed a RELAX NG schema for a restricted RDF/XML syntax version of DOAP. Note that the schema does not validate all DOAP files, but what it does validate is guaranteed to be processable by an application as DOAP. Listing 3 shows the schema in RELAX NG Compact notation.

default namespace = "http://usefulinc.com/ns/doap#"
namespace foaf = "http://xmlns.com/foaf/0.1/"
namespace rdf = "http://www.w3.org/1999/02/22-rdf-syntax-ns#"
namespace rdfs = "http://www.w3.org/2000/01/rdf-schema#"

grammar {

rdf-resource = attribute rdf:resource { text }
xml-lang = attribute xml:lang { text }
literal = xml-lang?, text

Person-element = element foaf:Person {
            element foaf:name { literal }
            & (
                element foaf:homepage { rdf-resource } |
                element foaf:mbox { rdf-resource } |
                element foaf:mbox_sha1sum { text }
            )+&
            element rdfs:seeAlso { rdf-resource}*
        }

cvs-repository = element CVSRepository {
            element anon-root { text },
            element module { text },
            element browse { rdf-resource}?
        }

svn-repository = element SVNRepository {
            element location { rdf-resource },
            element browse { rdf-resource}?
        }

bk-repository = element BKRepository {
            element location { rdf-resource },
            element module { text },
            element browse { rdf-resource}?
            }

arch-repository = element ArchRepository {
            element location { rdf-resource },
            element module { text },
            element browse { rdf-resource}?
            }

start = element Project
{
    element name { literal }&
    element homepage { rdf-resource }&
    element old-homepage { rdf-resource }*&
    element created { text }&
    element shortdesc { literal }+&
    element description { literal }+&
    element mailing-list { rdf-resource }*&

    element maintainer { Person-element }+&
    element developer { Person-element }*&
    element documenter { Person-element }*&
    element translator { Person-element }*&
    element tester { Person-element }*&
    element helper { Person-element }*&

    element category { rdf-resource }*&
    element release {
        element Version {
            element name { text },
            element created { text },
            element revision { text }
        }
    }*&
    element license { rdf-resource }*&
    element download-page { rdf-resource }?&
    element download-mirror { rdf-resource }*&
    element repository { ( cvs-repository | svn-repository
        | bk-repository | arch-repository ) }*&
    element bug-database { rdf-resource }?&
    element screenshots { rdf-resource }*&
    element wiki { rdf-resource }*&

    element programming-language { text }*&
    element os { text }*
}
}

One very handy spin-off of constructing the schema is that XML editing tools can now be used to write valid DOAP files. Figure 2 shows James Clark's nxml mode for the Emacs text editor being used to edit the DOAP description for DOAP itself. The nxml mode uses RELAX NG Compact schemas. Note the underlining of the validity error. At the hand-authoring level, the tradeoff between losing some expressivity of syntax and getting editing tool support is worth it.

Using verification with the schema, both XML well-formedness and a certain amount of semantic integrity can be assured. That's not enough, however, and you must resort to RDF processing to perform the remaining steps of the validation. Such steps might include checking the license URIs given to see if DOAP tools recognize them, spell-checking the text where appropriate, and so on.

I have begun work on a validator that works along these lines, giving XML output in a way that's similar to the viewer application. This way, the validator code can be used both for Web and client-side GUI applications. The validator performs four levels of checks:

• XML well-formedness
• Validity against the RELAX NG Schema (which can be optionally disabled)
• RDF validity
• A series of DOAP-specific semantic checks

Listing 4 shows two example runs of the validator that catch XML well-formedness and syntactic validation errors. The intention is that the test name from the <Test> element can be used as an index into further, more detailed, explanation and recommendation.

<Problems>
<Problem>
<Test>ParseXML</Test>
<Title>Not well-formed XML</Title>
<Description>DOAP files must follow the rules of XML syntax.</Description>
<Detail>unmatched closing element: expected name but found Project Line
27, position 10.</Detail>
</Problem>
</Problems>

<Problems>
<Problem>
<Test>Doap.Tests.Xml.RelaxValidate</Test>
<Title>XML syntax validation</Title>
<Description>This test checks to see that the DOAP file validates against
a restricted XML syntax, which guarantees its validity.</Description>
<Detail>Invalid start tag found. LocalName = Person, NS = 
http://xmlns.com/foaf/0.1/.  line 26, column 3</Detail>
</Problem>
</Problems>

You can obtain the code for the validator at the DOAP home page (see Resources). A public-access instance of the validator will be set up on the Web. It's important for potential adopters of DOAP to be able to check their output at an early stage. Additionally, I will recommend that all DOAP processing applications validate their input first -- at the very least with the RELAX NG schema -- in order to set minimum expectations for the quality of the DOAP content on the Web. A brief look at the mess caused by lax parsing of RSS feeds on the Web should be enough to convince you that some level of strictness is helpful. To help this, the code for the validator and viewer will be given a permissive open source license so that they can be integrated into other programs without worry.

Back to top

Creating DOAP

How will DOAP files be created in the first place? The very first adopters will write it by hand in a text editor. The combination of the validator and the XML schema file will help ensure that such files do not contain errors. As mentioned above, DOAP files that conform to the schema will lose some RDF expressivity, but for most people this loss will not matter.

Most people who create DOAP files will have no interest in the DOAP vocabulary itself. They will merely want the resulting functionality, allowing them to register their projects with software registries. They are likely to take an example file and tweak it to include their data. Because of this, it's critical to ensure that enough instructional matter, good examples, and validation tools are available to make sure that bad examples do not proliferate. Once a bad cut-and-paste gets out in the wild, there's little stopping it.

The cut-and-paste phase of DOAP creation should be kept to a minimum. You can do this by quickly introducing some level of tool support for DOAP file creation. Most software developers already use some kind of packaging and configuration system that contains at least some of the metadata required to generate a DOAP file for their software. Such systems include GNU Autoconf for C programming, Python distutils, Perl's MakeMaker files, and so on. If an easy DOAP generation solution is available for a developer's system of choice, it maximizes the chances of getting the DOAP file right.

In addition, you can use various guided methods of DOAP file creation:

• DOAP-a-matic: Leigh Dodds' FOAF-a-matic is a JavaScript-enabled Web page that makes it easy to create FOAF (Friend-of-a-friend) files. A DOAP version of this would be useful, so all a developer has to do is answer a few questions.
• Freshmeat-to-DOAP conversion: The Freshmeat software registry is probably the largest around, and it provides an XML export of its content. It would be a relatively easy matter to write a program to generate DOAP descriptions from Freshmeat content.
• GUI application: A graphical application could be integrated into an IDE such as Eclipse or MonoDevelop, with the metadata necessary for DOAP stored with the build information for a project. The DOAP file would then be part of the resulting build.

However DOAP is created, the two important goals are that it should be as easy as possible for the developer to use, and the output it creates should be as rich and as high quality as possible. The ubiquity and the quality of the available data are both key to DOAP's prospects.

Back to top

Community

Even with tools in place, if there's no community gathered around the DOAP project, then it is unlikely to last very long. When introducing a new technology, communication is paramount. It is important that the aims of the project are clearly expressed, as are the rules of engagement. The most basic step in communication is to construct a Web site that will hold all the relevant documentation and point to resources that those interested in the project can use.

Figure 3 shows a screenshot of the DOAP home page. Note two important features: clear navigation and news. A static Web site may often give the impression of a stagnant technology, and having regularly updated news meets the dual need of informing people and making the project seem alive. Using an RSS feed for the project news is also a must these days.

Although it's not necessary to have all conceivable questions answered from the get-go, the project Web site needs to be responsive in adding documentation and tutorial material as quickly as possible. How-tos and FAQs will often be the first place new adopters look.

However, communication isn't all one-way. There needs to be a forum in which those interested in using and adopting DOAP can meet each other and ask questions. The mailing list is historically the most successful medium for doing this with open source projects, and indeed DOAP has one of those too (see Resources). As the community grows, third parties that use DOAP can announce their own products and achievements there, spurring adoption and further growth.

Finally, the project must be seeded by promoting it to those who are likely to be interested. In addition to presenting on DOAP at XML conferences, I will be promoting it in various mailing lists and to key people in the open source world.

Back to top

Conclusion

Although this article is the end of a four-part series on the creation of DOAP, it also marks the beginning of the project's lifetime in the public eye. I believe that DOAP hits the right compromise between accessing the power of semantic Web technologies and remaining "Webby" enough to gain large-scale support. I will revisit the project in this column in some months' time to see just how these decisions have worked in the medium-to-long term.

Resources

• Read the previous developerWorks articles in this series, which cover the earlier stages of the development of the DOAP vocabulary:
  • Part 1 (February 2004)
  • Part 2 (March 2004)
  • Part 3 (June 2004)
  
  
  
• Learn more about the Redland RDF Application Framework, which was used to write the viewer and validator for DOAP.
  
  
• Validate your markup with the W3C's HTML Validator, the gold standard for Web validation services.
  
  
• Take a closer look at RELAX NG. David Mertz wrote a three-part series on this XML schema language for developerWorks:
  • Part 1 (February 2003)
  • Part 2 (March 2003)
  • Part 3 (May 2003)
  
  
  
• Read a report about the capabilities of James Clark's nXML mode for Emacs and download it from his Thai Open Source site.
  
  
• Check out Leigh Dodds' FOAF-a-Matic, a JavaScript-guided creator for FOAF (Friend-of-a-Friend) files.
  
  
• The Eclipse IDE would be an ideal place to add DOAP support.
  
  
• Visit the DOAP home page, where you'll find news about the project and links to helpful resources, including the newly-created DOAP mailing list.
  
  
• Find hundreds more XML resources on the developerWorks XML technology zone. Read previous installments in the XML Watch column series.
  
  
• Browse for books on these and other technical topics.
  
  
• Learn how you can become an IBM Certified Developer in XML and related technologies.
  
  

About the author

Rate this article

Comments

Back to top

Table of contents

• Viewing DOAP
• Validating DOAP
• Creating DOAP
• Community
• Conclusion
• Resources
• About the author
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

Dig deeper into XML on developerWorks

Discover knowledge paths

Skill-building guides for XML, Linux, open source, cloud, Java, business analytics, and more.

Special offers