Simon works in the CTO organization of Rational Software and is responsible for the business-level tooling strategy. Simon has undertaken a number of standards-related activities for both Rational Software and now IBM in the area of XML (W3C Schema working group), Web Services (RosettaNet architecture team) and Modeling (OMG UML and OCL teams). Simon has also written articles on the subjects of business modeling, software modeling and SOA and is interested to see where and when these threads will combine.

The "cosmic significance" of triples

Another interesting question from Bill Higgins on the Jazz web UI team, was why in so many conversations with those of us deep in the Foundation the term "triple" seemed to be embedded in the answer to almost any question. The answer is of course that in the Foundation we use RDF considerably as our index format, as the resource format for certain system resources and also of course as our query language (SPARQL). So, it's inevitable that triples, as the core abstraction in the RDF data model, appear a lot in our conversation.

But the more important point, the point Bill was trying to make was, why triples at all, why RDF, why not the SQL data model we used in previous tools? The answer to that question is longer and trickier, but boils down to the fact that we can't define a single data model for all the tools we have today, given the fact that many change on independent schedules, we have acquisitions that bring in new tools all the time and in some domains the resources themselves are not fixed but are constructed/configured by customers. So, how do we develop a single platform for all of these tools? Quite simply we look for a model that is inherently flexible, that allows us to represent information from these tool-specific resources in a common and general way. The answer to that is RDF and therefore there's a lot of us worrying about triples.

So. while maybe not "cosmic", the significance of triples can't be downplayed in the Foundation and you'll find all manner of people writing N-Triples and Turtle in email, documents and the Jazz Foundation wiki.

RDF background reading

I recently gave a presentation to the broader Jazz Foundation team on our use of RDF in the Foundation server, why we chose RDF, how we use RDF and where clients can expect to deal with RDF when talking to the server. One question asked was on any resources that might be helpful, specifically books, to read up on RDF. I had three suggestions from my own reading of books I found particularly helpful or at least worth the cover price.

Practical RDF by Shelley Powers (O'Reilly).

While it would be nice to see a new edition with the inclusion of SPARQL and some of the newer vocabularies (FOAF perhaps instead of RSS) this is a good introduction and like all O'Reilly books very readable and well put together

Semantic Web for the Working Ontologist: Effective Modeling in RDFS and OWL by Dean Allemang and James Hendler (Morgan Kaufmann)

Focusing much more on the development of ontologies this book gives a much more complete view of RDF Schema and OWL, again nicely put together.

The Description Logic Handbook: Theory, Implementation, and Applications (Cambridge University Press)

A very deep book, not an easy read (very few good characters and not much plot) but useful to understand why RDF is the way it is and some of the advantages and limitations of RDF based on it's theoretical foundations

Jazz Foundation moving forward

We've published the goals and scope for the M3D1 milestone which will include the migration of old JRS applications to the new Foundation core services. I also put together a brief topic on the Foundation Architecture which does help somewhat in understanding the terminology used in the specifications. Finally in review yesterday we approved the following service specifications:

• Storage API - The API for storing resources in a RESTful manner, this is the core of the Foundation.
• Query API - The API for executing queries against the index data extracted from stored resources.
• Index Contribution API - The API for storing index data extracted from stored resources ready for query.
• Application Query Library - A helper library to allow applications to provide simple, URL-encoded, queries to their clients.

JRS at the IBM Information on Demand Conference (again)

Edison and I have posted the charts we used at the IOD conference to the Jazz Foundation wiki. For those interested in our use of XQuery you can get the charts here (it is behind the jazz.net registration process unfortunately).

OSLC Podcast available

John Wiegand (IBM Distinguished Engineer and IBM Rational Chief Software Architect) and Steve Abrams (Open Services for Lifecycle Collaboration Chief Architect) have a nice podcast interview about OSLC on IBM developerWorks, published this week. You can access the podcast through the following links:

1. The developerWorks podcast index.
2. The developerWorks new media blog posting.
3. The developerWorks home page this week and in our iTunes channel.

Among other things, they talk about the open-services.net wiki which is now also available as well as the relationship between OSLC and Jazz.

Jazz Foundation Wiki

The new Jazz Foundation wiki is now up and running at jazz.net. The wiki will be used as our primary location for discussing design issues and publishing specifications for the foundation. We're also linking the wiki and the jazz.net work items together to provide a lightweight approval process. There is already some content posted - comments always welcome.

JRS Charts from RSDC

I'v been asked a few times for the JRS introduction slides we used in the Rational Labs at RSDC this year. I've converted them to PDF and made the available here. Enjoy.

JRS at the IBM Information on Demand Conference

I'll be co-presenting a session at the IOD conference with Edison Ting (one of the DB2 pureXML architects) about how we used DB2 9.x XQuery support in JRS. The original JRS incubator relied on XQuery to allow clients to write complex queries over the RDF/XML that describes our indexed properties (see the Open Service for Lifecycle Collaboration specs on Properties, Indexing and Query).

While this has definitely worked, the fact that the Open Services for Lifecycle Collaboration sample server uses SPARQL as it's query language means we get a lot of questions about our choice of XQuery. One answer is that it would require us to develop a lot of code to implement a SPARQL based solution for the Jazz Foundation Services going forward, the major advantage to using XQuery in DB2 was that all the smart folks in Almaden have done the hard work already.

So for all the fun details, see you in Vegas.

JRS has graduated!

While JRS may still be listed on jazz.net as an incubator project internally we've reorganized and refocused. The team has been reorganized into the Jazz platform team and has become the core of the "Jazz Foundation Services" development. JRS as an implementation has also moved on, the new Jazz Foundation Services will be based on the JRS but not before we complete some serious refactoring. So one thing we're trying hard to do is differentiate between JRS as the incubator project (and the server which will be shipped with the Rational Requirements Composer) and the Jazz Foundation Services as the common platform moving forward.

In terms of focus the move has also required us to look at how the incubator was structured and those areas where we replicated function that had been developed for the Jazz Team Server. In a few cases we'd had to either branch and enhance a Jazz Team Server function or we had developed our own in parallel (as in the case of our Lucene full-text indexing and search). We're in the process now of merging the changes we made in the Jazz Foundation Services platform and in the case of the Lucene services we will replace the Jazz Team Server code with that from the incubator.

Beyond the tactical measures and refactoring we are planning on opening up the server infrastructure itself, for example to allow storage, indexing and query components to be separate, co-operating, servers rather than all in a single server process. This, we believe, will allow more interesting topologies, scalability and distrubution characteristics, but impacts a lot of the current design both of the JRS incubator but also of the current Jazz Team Server. So, just looking at the to-do and to-think-about list for the Jazz Foundation Services we have:

• A common local/remote caching infrastructure (see work item).
• The ability to store and execute scripts in the Foundation Storage Services (see work item).
• Distribution of Lucene indexes (a scalability feature of the Rational Asset Manager).
• Separation of indexing and query services into a stand-alone and shared server.

So, it seems like we should have plenty to keep us busy for some time to come.

Open Services for Lifecycle Collaboration

Today Danny Sabbah announced the Open Services for Lifecycle Collaboration initiative in his RSDC keynote. There is, as noted in the FAQ, a direct relationship between the Open Services work and JRS. JRS has in some regards been the proving ground and many of the documents released today were JRS specifications that have been released to the public - though the JRS specs needed quite a bit of editing to make them look quite so nice. Also there are some code samples, the client-side examples are taken from the JRS project whereas the server has quite a history on this blog. The code that is being distributed is the experimental REST server we developed 18months or more ago in Python, the pre-cursor to JRS and now it comes back to life as the Open Source sample implementation of the Open Services specifications.

Note also that this server is the experiment I mentioned on Sunday where we have implemented the query service using SPARQL.

SPARQL and RDFLib

Well, I've posted here about JRS and XQuery which has proven to be very interesting as a query language over our RDF indexes. Today I'd like to discuss our prototype using SPARQL. Basically we built a Python server and used the very cool RDFLib implementation to allow us to store the same indexes and experiment with SPARQL queries over them. Here's a few of our initial (and non-scientific) results

• The queries seemed easier to write, though the jury is out on which language is easier to read.
• Mapping the JRS URL-encoded query to SPARQL was far more straightforward than the equivalent translation to XQuery.
• Obviously the fact that the query language and data share a common data model makes the query more logical.

Obviously we didn't compare performance, but one thing is clear while the major database vendors are scrambling over themselves to support XML there's no one talking openly about SPARQL. This is definitely a shame, because it's clear from just the experiments we have done that there are a class of queries that actually are easier to express in SPARQL than XQuery.

Look for a posting on Monday concerning the experiment :-).

JRS at RSDC 2008

After some discussion we've decided that JRS will be represented at RSDC this year, specifically we'll have our own pedestal in the solution center. So if you want to find out more about the fun we've been having, about the REST services we've been putting together and our RDF and XQuery plans stop by.

Updated: also look for us at the Jazz live event on Tuesday evening.

RDF - the good, the bad or the ugly?

I've mentioned before here that one great feature of the JRS server is it's indexing ability, in some ways it's akin to what Lucene does for text search in that there are a set of format-specific components that are able to extract properties from resources and a store into which these are put and made available for query. The difference between JRS and Lucene[*] is that we are trying to extract structured properties that can be made available via a more traditional query language - think XQuery and you'll have anticipated a future post. Some of these components have a fixed set of things to extract, for example we have an EXIF indexer that pulls a pre-defined set of properties from image files. Some however are configurable, so our XML indexer has a declarative specification that tells the server which parts of your resources the server should index. So what has this got to do with RDF?

Having indexed properties of resources we need not only to make them available for query, but we also want a query to be able to return some of these properties as well. We also support the ability to ask for all the properties that have been indexed for a resource by appending "?properties" to the URL of a resource. We wanted a format that would be able to encode these indexed properties in a regular way, and if possible pick up a standard one. We chose RDF as a standard, but also because our internal indexing components and storage have been inspired by RDF in a number of ways already. So, when you do a GET on {resource-uri}?properties you get a nice RDF document back, something like this:

<rdf:Description 
  xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
  xmlns:dc="http://dublincore.org/documents/dcmi-terms/"
  xmlns:ns="http://example.com/xmlns/music#"
  rdf:about="/jazz/resources/musicdb/albums/album-1">

  <dc:contributor rdf:resource="/jazz/users/zoe"/>
  <dc:modified rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">
      2008-03-02T00:00:00
  </dc:modified>
  <dc:format>application/xml</dc:format>
  <rdf:type rdf:resource="http://example.com/xmlns/music#album"/>

  <ns:name>A Matter of Life and Death</ns:name>
  <ns:releasedYear rdf:datatype="http://www.w3.org/2001/XMLSchema#integer">
    2006
  </ns:releasedYear>
  <ns:artist rdf:resource="/jazz/resources/musicdb/artists/artist-1"/>
  <ns:genre>Rock</ns:genre>
  <ns:genre>Heavy Metal</ns:genre>
</rdf:Description>

Where the properties in italics are system properties - JRS maintained properties for all resource types, and the rest have been extracted from an XML resource using the declarative indexer, so you can imagine that we extracted only a subset of element values from the source resource.

The point of the post is that RDF is a great format for this, the body of work and literature on RDF allows us to think about what these property documents mean, especially in the presence of many linked resources. So for us the use of RDF in JRS has been really good for the server team, and we had thought that the client teams would think so too. Well the first thing that came up was the <rdf:RDF> tag that we had originally put around all the description documents, this seemed like unnecessary overhead (the ugly) and so having re-read the RDF spec and found that it was optional anyway we took it out. The next reaction was not one we had expected, one development manager here asked whether this meant all his developers now have to become RDF experts. Well, we had not anticipated that, we had used RDF as first and foremost an XML dialect and as such we internally and in our samples manipulated it only with XQuery and XPath. So no, we expect most developers do not need to know RDF, they just see a particular XML form with some particular idioms (we could even use another prefix than "rdf" and I wonder how many people would even recognize that it is RDF?).

The interesting opportunity is when you DO treat this body of index data as RDF, for example XQuery is great, but like SQL it really falls down trying to navigate a graph of resources unless the query has fixed the relationships ahead of time (think nested queries). SPARQL, as an RDF query really excels at this kind of query but falls down in other ways. We expect that many of the first JRS applications will use only XQuery and XPath, but over time we expect that some application requirements will be much better expressed in SPARQL which we hope to provide in JRS at some future point.

I am interested that there seems to be a bad impression of RDF in the general developer community, personally before working on JRS I had had no practical experience of RDF but it does fit the need we had. I wonder if there is an assumption that to consume RDF you must also consume RDFS, OWL, think in Ontologies and so forth? Perhaps the hype around the semantic web has put some people off? Whatever the reason, the comment we heard on RDF in our meeting seems to be common, and we've learned to smile sweetly and spend time explaining to many people why this is good for them (there are some people we just tell to suck it up - but that's a different matter).

Good? Yes, for us, in this case it is. Bad? No, but it does take some selling. Ugly? Not really, actually when carefully used someone commented that "you really couldn't have made it any simpler or clearer if you tried".

* - JRS does use Lucene as well, so we support both structured query as well as full-text search.

New entry on the Jazz blog

James Branigan, one of my fellow JRS developers has a nice article on the Jazz blog about the journey taken by the Jazz team and which brings us up to date with JRS. A brief history of the Jazz Team Server interface: Our journey from a J2EE server towards a RESTful server

Oh, and for those in the know the Python server mentioned in James posting is the project I have mentioned here a few times, it used Python and Django to build a fairly complete RESTful content repository and in fact the documentation from that project became the initial specs for JRS. We never intended for that server to see the light of day outside the lab, it became affectionately (I like to think) known as the "Rinky-Dink" server.

JRS and JCR (JSR-170) again

David Nuescheler Said (here):

Hi Simon,

Thanks a lot for this post. I am very interested in the future development of JRS and as the Spec-Lead of JCR (aka. JSR-170 & JSR-283) I would like to ask for some more detailed clarifications around some of the above statements.

(1) Flat, Hierarchical & non-contiguous JCR in my mind is not limited to exposing hierarchical only structures. As a matter of fact I think of flat as a very simple hierarchy to begin with. In a content repository is certainly acceptable that there is no accessible (readable) direct parent node of an item (be that for access control reason reasons). In my mind the URL space itself is hierarchical though its path segments as defined in http://gbiv.com/protocols/uri/rfc/rfc3986.html#path

(2) Personally, I believe that the tie into "Java" is more of tie into the JCP as a standardizing body and not so much about the tie into the "Java language". Maybe the discussions around APP & JCR http://dev.day.com/microsling/content/blogs/main/jcr-loves-atom.html and a couple of comments around some of the non java languages that use JCR may be interesting: http://dev.day.com/microsling/content/blogs/main/fudbusting2.html

(3) I really think that JCR allows both for typed and untyped nodes (we call them structured vs. unstructured) as a matter of fact I am big proponent of a "Data First" architecture and find this feature in JCR one of the most fascinating features.

Anyhow, thanks a lot for your excellent post and I would be happy to engage in a more detailed discussion, feel free to contact me at any point.

David, I would be interested in the discussion, I think the points you make are valid and interesting. In terms of hierarchy it's important to JRS to allow clients to be able to PUT a resource to /a/b/c/mydoc.txt without having to have created a node at /a, /a/b, /a/b/c first which is how I meant to use the term non-contiguous in this context. Of course we also allow /a to be an Atom feed, so you can POST a child feed to /a/b and then another to /a/b/c and finally POST mydoc.txt, providing a completely hierarchical space - or you can mix both styles.

I understand the point about JCP, however we wanted to ensure that we didn't "pollute" our design with any assumptions and so we worked only from the HTTP and APP specs and developed test clients in Java, Python, BASH scripts, etc (see "J" is for Jazz, not Java). As you note in the fudbusting link the JCP API still needs to be implemented in different languages and that there are difficulties with type conversions and so forth. Your third integration approach - that of accessing the content repository through a RESTful interface is exactly the approach taken by JRS and allows maximum freedom in client choice.

I must be showing my ignorance of JCR (my apologies), again I mean that we do not have the notion of resource type in the JRS repository apart from one specific case - Atom feed. On a brand new server I can PUT MS Word documents, code, audio, video, any resource that exists on my machine in fact, into the server without having to define any types ahead of time. All resources are stored simply as they come in, with their content-type and other associated meta-data without intervention or assumption by JRS. I mentioned Atom feeds, the way you create a collection in JRS is simply to PUT (or POST to an existing collection) a valid Atom feed document with the content-type "application/atom+xml" and the server does assume that you want to create a collection at that point - it therefore creates additional structures server-side to allow for clients to now POST to the feed. There is the ability for clients to teach the server about interesting things in XML resources for our indexer, but I'll hold that thought for a posting of it's own.

I hope that makes things a little clearer, and I hope I didn't completely dis the JCR spec :-)