Smarter Everyone, Smarter Everything, Smarter Everywhere
John M. Boyer 060000VMNY 1,376 Views
The IBM Workplace Forms product suite will make your life easier. It simplifies by an order of magnitude the work of creating and maintaining forms applications.
Sometimes I hear people say "Forms. Hey, name, address, pepperoni, extra cheese. I know about forms. Nothing new there." Well, guess again because we're not talking about those kind of forms. After all, hey, you do already know all about those kind of forms.
In this blog, I'd like to be able to talk to you, the developers in our community, about what makes IBM Workplace Forms different and better for forms applications. We'll talk about bigger picture trends, architectures and models, but in at least equal measure we'll talk about the nitty gritty, the pointy brackets, the XForms, the XML and other technologies behind it all. And best of all through our dialogue we'll develop a community of the most efficient and effective forms application developers on the planet.
For now, let's start with some links you can use to get your hands on some software to test drive.
Here are the reference manuals and user docs for the currently released version:
Now, for me, here's the really exciting link. Our upcoming new version will be available in 26 languages and combines rich XML processing capabilities of XForms with the precision presentation, rich user experience and superior authoring experience of Workplace Forms.
Here's a link to the current public beta:
(This is the Lotus main beta page. The direct link is http://www.lotus.com/ldd/beta/workplaceformsbeta.nsf/)
John M. Boyer 060000VMNY 1,471 Views
Got a request today to provide more specific information on exactly where to find the XFDL reference manual.
The documentation site link in my prior post includes a link to the currently released XFDL Reference Manual. For ease of use, the following appears to be the specific link on that documentation site:
Note that you will ultimately receive the XFDL reference manual in PDF; we always publish our language manual in PDF because PDF is good at static readonly documents. XFDL is good at dynamic interactive precision forms applications, but that doesn't mean the language manual should be expressed in it.
On a separate note, you can find a fairly comprehensive technical introduction to the integration between XFDL and XForms in my XML 2005 conference paper. The upcoming version of IBM(R) Workplace Forms(tm) will be accompanied by an updated reference manual that describes the union of XFDL and XForms exhaustively, but at 36 pages, the conference paper actually gives quite a thorough start to those interested in the pointy brackets. Here's the link:
I did a customer site visit out in Washingto DC earlier this week (I live on the west coast of Canada). On the plane next to me there was a guy who had a book to read with a title that was something like "Three men, a raft and the most dangerous river". It appeared to be about three guys who navigated the Amazon in a raft.
Perhaps it was just my mood at the time, but it struck me as a little ironic to read a book about it. It's just so much the kind of thing you have to do.
Well, while we're in the land of giant leaps... forms applications are like that too. You can read, but you have to *do* too.
For my own case, the Workplace Forms beta download is accompanied by a reasonably substantial sample form that I wrote to help illustrate for you a number of the important features that are available, such as a wizard-like or guided-interview modality attached to the front end of a "traditional" form.
The internal markup is highly commented and laid out so that you can really get an understanding at the pointy bracket level even if you only have a text editor for exploration (though the beta also includes our eclipse-based designer product).
So, speaking of doing (and also to fulfill my promise to talk about the nitty gritty), it seemed a good idea to share with you one of the interesting things I learned from this form.
The typical form is designed to be used in a particular business context. In general, you will not find that a form (of the size and scope that Workplace Forms targets) is easily purposed to serve multiple locales or business contexts. The "mortgage preapproval" sample form was created to give people in many locales some idea of what the product set can do, but it's not really the intent to address all locale-specific peculiarities. For example, a more realistic form in Russian would have the formulae and inputs adjusted to collect monthly salary information rather than yearly information. And in some locales, it isn't even appropriate to collect interest on a loan, nor is it appropriate to accept a service like a loan for free, so interest is something that is informally negotiated rather than being driven by an exact compound interest formula.
Anyway, as I said we just wanted to provide a form that would give a good idea, so we ended up with the task of translating the form to many languages.
Something I didn't expect happened because I put some localizable strings into certain XML attributes, like an XForms calculate or the value attribute of an xforms:setvalue. The word "Electricity" becomes "L'Electricitie" (pardon my omission of accented characters).
The addition of the apostrophe is problematic when in an attribute because the attribute contains an XPath expression. So you have double quotes around the attribute value, so that single quotes are used around the literal value, and the apostrophe conflicts with that. Moreover, the apos character entity is of no help because that is an encoding consumed by the XML parser, not the XPath parser, so it is converted from a character entity to the literal character by the XML parser so that the XPath parser still receives the offending character.
So, it seems forms have to be designed so that strings intended for humans don't appear in XML attributes that contain XPath expressions. My apologies to my European friends, who no doubt know this already. As I said, pardon my French.
John M. Boyer 060000VMNY 1,600 Views
Very soon, I'll be heading off to the W3C Technical Plenary.
Sure, it's in Cannes, but it's also in February, so it's all business.
This is a time when many W3C working groups congregate in order to have time to have intergroup meeting times. So, the XForms working group will have its face-to-face meeting later in the week, but so will a dozen or more other groups. Similarly, the compound document format group chaired by Kevin Kelly will meet earlier in the week. I plan to sit in on some of that as an observer and to talk as needed about the intersection of XForms and compound document formats.
The mid-day of the plenary is a time for common presentations, one of which is a panel presentation called "Rich Web Application Backplane". Kevin is the chair of the panel, which has a number of panelists including yours truly.
The panel is basically about the notion that we have lots of rich XML processing capabilities available now, expressed as XML. This allows for compound documents that are capable of aggregating the data, views, rich user experience, and web services needed to complete a sophisticated web application transaction.
While some further work can be done to do a better job of creating more seamless integration of various components in the W3C stack, a lot of the foundational technologies are already there and in pretty good shape, so herewith is the rallying call to pull them together into an XML backplane capable of supporting the design of rich web applications.
One of the things I really like is the potential for XML events to be the glue that binds together nicely modularized XML technologies into an effective, intelligent compound document. I think an update to XML events would be useful to add certain key features, but even as-is the capture and bubble phases combined with event context have allowed me to express an integration of XML signatures into a compound document format that includes XForms and a first class presentation language (XFDL, of course).
Of course, whenever we talk about advancing the conceptual model of the web in some way, we get a certain amount of feedback about whether we are talking about evolutionary change or revolutionary change. The latter causes IT folks ot go weak in the knees over deployment issues, so branding something revolutionary is really about trying to stop it from happening.
This means that you can have a staged migration to a rich native client that directly runs the high level language because the conceptual model expressed by the language can be delivered, with few limitations, to today's client platforms-- zero footprint.
In the IBM Workplace Forms space, we call the native client "Viewer" and we call the zero-footprint version "WebForm Server". The idea is that you design an XFDL form containing precision presentation, rich user experience and business logic/dynamic interactive capabilities using XForms within the XFDL. Then, you deliver it using one of these two interactive rendition engines.
A demo of this capability is how my panel segment will end. After all, nothing trumps being able to show some of the kinds of capabilities that would be enabled by the rich web application architecture.
John M. Boyer 060000VMNY 1,580 Views
Well, it's only two days into the W3C Tech Plenary and already I have enough things to write about that I cannot possibly inflict it all upon you in one blog entry. So, I'm just going to give an overview so that I'll remember to talk about some of this stuff in more detail later.
But first, an excellent bit of humour from another long-time W3C member who also happens to live in Canada (a little knowledge of Canadian geography and politics helps with this one):
Liam: Where do you live?
John: I live in Victoria, which is near Vancouver.
Liam: Oh, I live in Ontario, which is near Toronto.
Well, it was funny when Liam said it :-)
Anyway, for the first two days here I have been able to sit as observer on other working groups as well as have some individual discussions of some importance. Here is a summary:
The chat with Liam also included his hopes for XForms eventually adopting XPath 2.0. More about this in another blog entry...
Sat in for a while on the Compound Document Formats working group. Major topic of discussion was event flow between DOMs. How do a parent and child document come to agreement on whether events flow between them? Where does the capture phase start? More to come...
Sat in on the XHTML working group. Major topic: XML events. Specifically, how to upgrade XML events to offfer authors the ability to dynamically add and delete listeners after the document is first loaded. There's not a lot of complexity semantically, but the discussion of the vocabulary involved led to a very exciting discussion about the meaning of namespaces, especially as applied to attributes. This will be a beaut of a blog entry...
Talked with Doug Schepers, who was very interested in having a declarative way of doing AJAX-like things so that the functionality could be securely enabled even with scripting off. I told him I would demo it tomorrow in my part of the Rich Web Application Backplane panel. It's called XForms. More to come...
Finally, had a very good chat with Jose Kahan and Thomas Roessler about making progress on correcting the impedance mismatch between canonicalization and xml:id, with implications for a revision of XML Signatures. Lots more to come...
Stay tuned for more on these points!
John M. Boyer 060000VMNY 1,266 Views
A lightning talk at the tech plenary is about 3 minutes long and introduces something very specific to the W3C.
I gave a lightning talk on the effecs of adding or changing the stuff in a namespace. You can see the diagram and notes here: http://www.w3.org/2006/03/01-Boyer-Lightning/SignaturesAndNamespaces_Boyer.html
Basically, got a lot of nods all the way up to TimBL himself when I said you either have to use a new namespace, or you have to internally version the language so that old processors for a vocabulary don't try to render new documents with graceful degradation of unrecognized content when the documents have been signed.
Too bad this is exactly what happened with xml:id. It got added to the XML namespace rather than some other namespace, and the version of XML didn't change. Lo' and behold, it broke something. When doing a C14N of a document subset containing orphaned nodes, C14N copies XML namespaced attributes into orphaned nodes when they don't contain their own settings for the nodes.
This is good for xml:space, xml:lang and xml:base, but it isn't good for xml:id.
Truth be told, it's kind of an edge case. In XFDL, we don't even allow you to orphan nodes when signature filtering because the structure of the language is such that an orphaned node is useless without its ancestral chain.
Still, while the problem doesn't affect Workplace Forms, we (in W3C capacity) will still endeavor to fix the problem.
Since the ship has already saled on what namespace xml:id lives in, we're going to be doing a new C14N algorithm that doesn't do the inheritance behavior on xml:id OR one that doesn't do the inheritance behavior except for lang, space and base.
Actually, it's a little more complicated than that, since either of the above choices means that C14N will again be broken in the future when either a non-heritable or a heritable attribute, respectively, is added to the XML namespace.
I think we may have to add a parameterization to the new C14N that allows the author to specify the heritable attributes. This will allow document authors to keep up with adjustments to the XML namespace.
The core WG feels that further additions to the XML namespace are highly unlikely, but I'm not convinced. Just at this tech plenary alone, I heard calls for xml:role (like HTML's role) and xml:profile (like DOM's hasFeature, it would declare that a document has a feature so the processor needs to have the feature or the document won't work). In the past, I've heard a need for xml:src (like HTML src, except HTML's default is wrong-- content should override the attribute rather than the reverse). And my personal fave would be xml:compute to express that the content of an element is computationally derived from other content. The list really does go on once you start to think about XML as an intelligent object...
John M. Boyer 060000VMNY 963 Views
XForms 1.0 Second Edition has been published today at http://www.w3.org/TR/xforms/
To get an idea of the quality and quantity of improvements made to XForms, please see http://www.w3.org/2003/10/REC-xforms-10-20031014-errata.html
Based on this improved foundation, the XForms working group will now be focusing its energies on the completion of XForms 1.1. To get an idea of what will be available, check here: http://www.w3.org/TR/xforms11/
The one comment I would make about the above working draft is that the we will almost certainly revert to using the same namespace currently used for XForms 1.0, and instead use some mechanism within the language to do versioning.
In keeping with my prior post about signatures and namespaces, it is important to version a language either internally or by updating the namespace URI. Previously, we chose the namespace route because XForms is designed to be hosted within another language, so it has no root element of its own to which a version could be attached.
However, XForms 1.1 is using some special schema wizardry that allows it to have a "chameleon" namespace, which will make it easier to import XForms into a host language like XHTML without namespace qualification. I'm not a big fan of doing this, especially for host languages other than XHTML because it becomes harder to find the XForms within another document and host the XForms functionality separately from the original host language.
Nonetheless, the feature is there and it occurred to me during the W3C tech plenary that the chameleon namespace could be used to put XForms 1.1 back into the XForms 1.0 namespace. That means that XForms processors trying to determine what semantics to attach to the vocabulary need some other way to make their decisions. So we simply have to solve the versioning problem without using a change of namespace URI.
The camp that wanted us not to change namespaces will be happy. My own XSLTs will be happy too.[Read More]
Well, no blogs from me next week, so seems a good idea to knock off another one this week...
XML namespaces rec (pardon the pun) states that an attribute which is namespace unqualified is 'local' to an element and is uniquely identified by a combination of the attribute local name and the type and namespace URI of the containing element.
The word identify really should be used more sparingly, as here is a case where its misuse has caused years of confusion and acrimony in the XML community. An identifier is something that established the identity of something else. You cannot have two things associated with the same identifier unless they are identical things.
I am often frustrated by seasoned W3C folks who say that "this depends on your definition of identify" and honestly believe this is a defense of the confusion in the namespaces rec. This is like saying ot me, "Well you're right unless you have a definition for identify that doesn't identify things, which is what we did at the W3C."
To see the problem, you have to look earlier in the spec where a namespace n1 is associated with a URI and then the following code appears:
<good a="1" n1:a="2" />
The problem is that the element 'good' is in the same namespace as n1. So now you have local attribute a that is essentially given meaning by an element that is in the same namespaces as the 'global' attribute n1:a. Yes the two attributes have different values.
From this we have to infer that, although a local attribute is given meaning based on the containing element and its namespace, a global attribute with the same local name and namespace qualified into the same namespace can actually mean something totally different..
In other words, we have two attributes with the same local name, contained by the same element, and given meaning by the same namespace URI, but they are not identical. This is the local attribute not being 'identified' (in my sense) by local name and containing element type and namespace.
The technically subtle W3Cer will tell you that there was no reason to spell out using words in the normative part of the spec the fact that the two attributes are different things because the spec says that local attributes are in a different partition than global ones. Problem is, this partitioning info is in a non-normative part of the spec, and particularly in the same part that has the language about how local attributes are 'identified' by local name and containing element type and namespace.
Anyway, the upshot is that an XML vocabulary does not need to but is allowed to say that local and global attributes with the same local name can mean different things. If they're supposed to mean the same thing, then the XML language has to define a precedence rule for what happens if the two attributes differ in value. Here's an example:
Question: Is the price in USD or Euros?
Answer: Depends on who designed the language.
Second answer: Don't do that.
Interestingly, the XHTML working group came up with a fascinating example where it is legitimate and sensible to have a global and local attribute with the same local name but completely different meanings. It has to do with the next version of XML events. After hours of discussion, the decision was that they weren't going to do that (second answer above). Not because it's illegal, but because it's too subtle for most XML people.
Well, the XHTML group may change their minds, but even if they don't, the example is really worth understanding because it actually makes sense why you'd want to have the two attributes mean something different. Stay tuned, I'll tell you all about it when I get back...[Read More]
John M. Boyer 060000VMNY 1,978 Views
Continuing from the last post, here is a reasonable use case for having a global (namespace qualified) attribute with the same local name as a local (unqualified) attribute, where the attributes do not have the same meaning and the global attribute is in the same namespace as the containing element.
The current version of XML Events allows you to declare a listener for an event on a node. There are multiple ways to do this, but the easiest is to use a global attribute in the events namespace (signified by the prefix ev below):
An xforms:trigger can be represented, for example, by a button widget, and pressing the button results in a DOMActivate event on the trigger.
The ev:event declares the name of the event to listen for, and the default value for ev:target (when it's not specified) is the parent element of the one containing the ev:event. So, the above declaration causes a listener to be created for the occurrence of the DOMActivate on the trigger. The handler for the event is given by the content of the xforms:action that bears the ev:event. An example of an action to be performed would be inserting a node of data (which would likely correspond to adding a row to a repeat table).
All of this is just plain vanilla XForms so far. But at the W3C tech plenary, the XHTML working group began discussing upgrades to XML events. The reason is that the current version of XML events declares listeners that are created at the moment when the DOM node being listened to is created. There's no way to create a listener later in the life time of a document, possibly contingent on some condition or the occurrence of some event.
Syntax for this could look something like this:
In this example, ev:listener is an action handler for the DOMActivate event that happens on the trigger. So when you activate the trigger (press the button), a listener for event Y is created on the node with id="X".
The local, namespace unqualified attribute 'event' tells the event that the listener will listen for. The global, namespace qualified event 'ev:event' tells the event whose occurrence causes the ev:listener action to be performed.
This is a very neat and compelling example where a local attribute (event) which is given meaning by a namespace (XML Events) might legitimately have a completely different meaning than a global attribute with the same local name and same namespace as the containing element (ev:event).
Although the XHTML working group quite liked this solution, they had a lengthy discussiona and concluded that this aspect of XML namespaces is not understood well enough to rely on the feature without confusing document authors. Of course, they are free to change their minds later, I would say that the above syntax is not likely to see the light of day in a W3C recommendation.
That's why I'm blogging it. Because even if it goes away, we still need to try to increase understanding of this aspect of namespaces and attributes, and nothing does that better than a good example, and this is the first one I've seen.
And we can thank Steven Pemberton for it![Read More]
John M. Boyer 060000VMNY 1,481 Views
IBM is interested in all manner of open standards, including XForms, because they allow organizations to express their information assets in a format that can more easily be preserved as one incorporates more technologies into an application over time to meet evolving business requirements.
As Kevin Kelley (STSM, emerging standards) explained in a telecon this morning, the standards are like an on-ramp that allows assets to be developed over time. If you care about reuse of those assets, then you care about standards.
To be honest, this is the same argument that drove XML to the ubiquity it currently enjoys. With XML, we sought to standardize how data models are expressed. With XForms, we seek to standardize the expression of processing models over that data.
Some upcoming posts:
1) What does XForms actually offer?
2) How does XFDL fit into the above open standards story?
3) What makes XFDL+XForms an ideal format for archival, transaction auditabilty, and transaction security purposes?
John M. Boyer 060000VMNY 1,310 Views
The powers that be asked the dW bloggers to help spread the word about the download availability today of DB2 Viper Release Candidate 1.
Please see www.ibm.com/db2/viper
You can also get more info from
Anyway, why does this make sense on a blog about Workplace Forms?
As I said before in this blog, the purpose of a form is to collect data. If you want to collect data about a pizza order, you don't want us, but obviously there are more sophisticated information needs than that, right!
Question is, once you've got the data, what do you do with it?
There's always a server side to any web application. Products like DB2, Content Manager, and Portal Document Manager are about providing high strength persistence tiers for the consumption of data collected across an enterprise or across enterprises. Products like WebSphere Application Server and WebSphere Portal Server are about creating that middle layer that creates the logical bridge between from the point where you have the data to the point where you know what to do with it (something has to decide where to put the data, what data to retrieve, etc.).
And IBM Workplace Forms is there to put a beautiful face on it all by intelligently collecting data of any level of complexity while providing a richly textured user experience. And DB2 Viper is an especially good fit with IBM Workplace Forms because our forms contain XForms, which is a pure XML play, and the most significant addition to DB2 in Viper is the native-XML data model.
The purpose of XForms is to express the core XML data processing asset used in sophisticated data collection scenarios.
In fact, it would be better if XForms were called the XML data processing language (XDP or XDPL) because XML is about standardizing data and about 80% of business transactions are based on filling out some kind of form to collect the transactional data.
An XForm contains one or more XML data instances. An instance is an arbitrarily structured XML data document that is typically an instance of some XML schema that expresses the static validation rules for a target namespace.
One can write an XForm without an XML schema by just expressing the XML data in an instance. This is because XForms provides other channels of data validity checking that can be easier to work with when only simple data type validation is needed. For example, you can use an XForms
But XForms validity checking is also dynamic, in recognition of the fact that validity of some values can be based on other values or the aggregation of other values. For example, in an interlibrary article request, the upper bound page number in the journal must not be less than the lower bound page number. Or, the user is only authorized to make a purchase order with less than $10,000 total value.
The latter example is important because it leads to the conclusion that we not only need a way of testing data values relative to other data values, but also that we need a way of calculating data values that are then used in validity tests.
From there, it is not a big leap to conclude that generalized XML data processing requires some way to indicate dynamically whether further changes should be allowed to certain pieces of data or whether certain parts of the data are still applicable to the transaction based on other data values. A good example is a mortgage preapproval form that can handle both single and joint applications. The co-applicant data is only relevant if the user selects the joint application mode.
XForms allows the form author to express formulae for these aspects of data, which are called model item properties, or just MIPs. Not too surprisingly, the names of these MIPs are
Of course, there is no point in representing data, calculating values over data, and validating data if you have no way to change the data. XForms allows simple content data values to be changed, but it also allows insertion and deletion of larger blocks of data that contain internal structure because this is essentially what's needed to add or delete a row from a table.
Most importantly, XForms offers form controls that expose data to the surrounding application context. If the data changes, the form controls change. This includes not only exposing a changed simple content data value, but if a set of form controls are associated with a repeated sequence of structured data, and the number of data nodes in the sequence changes, then form controls are created or destroyed as needed to respond to the change of data.
XForms is all about thinking of the data first and driving outward to how that data gets exposed to applications. Perhaps the most prevalent of such applications are for presenting the data to a human user, though even human users have highly varied capabilities. For example, the desktop user and the PDA user have very different visual capabilities. Of course, this argument extends easily to meeting the far greater accessibility needs of the sight-impaired.
For this reason, the XForms form controls represent what I've often called an intent-based user interface. It's kind of neat to see the term popping up more frequently now. It gets to the heart of the matter: XForms does not provide a presentation layer. XForms relies for presentation on a host language like XFDL (in Workplace Forms) or XHTML (in web browsers). I am certainly hoping that VoiceXML will come to the conclusion that they should soak up the benefits of XForms rather than reinventing all of this stuff over again (partly because almost everybody underestimates how much work goes into it until it's too late; but maybe they will prove to be wiser than the rest).
I sometimes get asked whether XForms will next extend itself to standardizing the actual presentation layer. Clearly, from above the answer is no. XForms standardizes the core XML data processing asset, and more work will go into doing a better job of that. The key issue we want is to address interoperability and reusability across applications and user contexts of the data processing behaviors that are fundamental to completing a transaction.[Read More]
An interesting comment showed up on my prior post that seemed worth discussing as part of getting around to talking about what XFDL does for (and with) XForms.
In the last post, I talked about what XForms does and what it doesn't do from the big picture perspective. XForms does seek to standardize the language for the core information processing asset expressed within a form. XForms does not seek to standardize the exact presentation of a user interface. XForms delegates this task to different XML host languages like XHTML or XFDL (or others in the future like, hopefully, VoiceXML). These host languages exist to satisfy different requirements that exist beyond the core information processing requirements, but XForms allow us to design the underlying application once and handle presentment and other orthogonal requirements as the separate issues that they are.
The comment came in saying that one thing XForms doesn't do is allow the URL of a submission to be dynamically calculated. This is a technical limitation that does not affect the big picture of what we're trying to achieve with XForms. The XForms language does contain some technical limitations like this. Many of them are being addressed in XForms 1.1. This issue in particular should be addressed in the very next working draft of XForms 1.1.
However, because we have to deliver products that can be used to build applications now, it sometimes happens that implementations have to lead the standard with custom extensions until the standard comes along and specifies the common way that everyone on the working group agrees is the way the language will express a feature.
In XFDL, we make relatively few changes to XForms because we want our documents to be as conformant as possible. However, this particular issue of a dynamic URL comes up in almost all of the forms applications we have every deployed, so we did not feel we could make our next release of Workplace Forms without some ability to do a dynamic URL on an XForms submission. At the same time, we do like it to be clear to the form author when an extension is being used. As a result, the addition was made using an attribute in the XFDL namespace. This makes it easier to find those bits of our XForms-based documents that will need special attention when trying to get them to interoperate with other XForms implementations.
In the upcoming release of Workplace Forms, you can create XForms-based applications that include a dynamic URL component by using an
In XForms 1.1, I am expecting a more general mechanism that will allow the instance data to be used to set not just the URL, but eventually many of the other parameters too.[Read More]
I am the happy owner of an excellent new book entitled "Document Engineering" by Robert Glushko and Tim McGrath. The reader goes on journey through many of the XML technologies that are brought to bear to solve business informatics problems. The journey does not jostle and stab with so many pointy brackets, but rather focuses intently on why we need all of these technologies and how to put them together sensibly.
There are so many nuggets of wisdom in the book that of course I cannot tell you all about it in this blog. Just like the Matrix, you have to see it for yourself! But I would be remiss in not giving you at least a taste, especially since the message is so closely aligned to the value proposition of XForms and, indeed, XFDL+XForms.
In other words, XML documents are the lifeblood of a service oriented architecture, and XML technologies are valuable because they help us overcome the limitations of rigid, monolithic systems.
With Workplace Forms, we combine XFDL and XForms to achieve a somewhat elaborated version of this view in which the forms themselves are the documents that make their way through a service oriented architecture, interacting according to their own rules of engagement to achieve validity of the contained XML data document and more efficiently achieve the intent of a business process.
In other words, the SOA is the infrastructure, the XFDL form is the medium, and the XML data is the message. With this analogy, it is easy to see that the powerful words of Marshall McLuhan are applicable: The medium is the message. The more powerful the medium, the more powerful the message. An XForms layer around XML data trumps a system in which only the XML data is standardized. An XFDL layer around the XForm... transaction auditability, digital signature security, comprehensive accessibility, rich text, globalization, and on and on. All the things we get to talk about in future installments of this blog.[Read More]
John M. Boyer 060000VMNY 1,495 Views
What is the value proposition of an open standard?
XML is fairly pervasive, so we rarely ask this question of XML anymore, but once upon a time the question came up a lot as business managers tried to figure out why the technical people were insisting on spending money to move to XML. And the truth is that the impact is difficult to measure precisely, so open standards are sometimes a bit of an uphill battle. Nevertheless, the software engineering benefits are tangible and increase in magnitude over time.
One benefit is, of course, the human resource factor. Given a schema or DTD for a pile of pointy brackets, human beings can learn a lot about your document format quickly, which means they can become proficient more quickly and be more efficient overall at moving information into and out of the document.
This has an impact on the development of software systems. The software engineering benefits of increased interoperability/looser coupling of system modules have a significant positive effect on the time and cost efficiency of software development. Really, it's the same benefits as a service oriented architecture, which is why SOA and XML documents are such a good match.
But XML standardization has a deeper impact as it also places a value on the document format. In other words, the document format becomes a product in and of itself. A software system based on an XML document format is more valuable than one that is not because it is easier for enterprises to migrate to or from the document format. The benefit to a vendor of enterprises being able to migrate to the vendor's format is immediately obvious, but the ability of the enterprise to migrate from the vendor's format is also surprisingly valuable to the vendor. This is true not just for the obvious reason that being trapped in a document format is inherently costly to an enterprise. So, the enterprise can more readily adopt a vendor's solution when it does not imply vendor lock-in, but frankly it is the capability to more easily migrate away from the vendor's solution that becomes a selling point. A vendor can say, "We know you have a choice, so we're going to be responsive to your needs and deliver quality software so you keep choosing us."
It is with all these benefits in mind that we moved the predecessor of Workplace Forms to an XML syntax called XFDL. The XFDL language is an XML vocabulary that simplifies the design, development and deployment of high precision, secure forms applications that provide a rich user experience.
Of course, the first thing we did with our new XML syntax was to report it to the W3C in a document which became a W3C Note. The purpose of a W3C Note is to bring to the attention of the W3C something that contains aspects worth of consideration for standardization. The W3C does not and never will standardize a vendor's submission. But it does take note of its own notes! A positively reviewed note is likely to result in some movement in the standardization world. In the case of XFDL, that movement has occurred all over the place, including the likes of XPath, XML Schema, XML Signatures and Canonicalization, and XForms.
Of course, XFDL now incorporates XForms to express all aspects of XFDL that it can. And like a good standard ought to do, XForms itself incorporates other W3C technologies where appropriate, like XPath and XML Schema. But XForms depends on a host language to deliver the actual user experience, and there are aspects of a precision presentation and rich user experience that properly belong at the host language level. And XFDL even encodes these bits with the most pervasive standard of all -- XML.[Read More]
John M. Boyer 060000VMNY 1,413 Views
We usually think of a document as a file. A file is, at the operating system level, a unit of information. The OS can associate properties with it (like who can read it or write it), and the file can easily be moved from directory to directory or computer to computer with standard tools like FTP, email, or flash drive.
The XML recommendation defines a document as that which conforms to BNF rule #1 of the XML syntax. Of course it is useful if an XML document is the sole content of the unit of information being transferred, i.e. if the XML is the only thing in a file. This is because XML tools tend to be built with programming languages that expose services of the operating system.
However, the most important part of an XML document is capable of being contained within another document. In fact, XML canonicalization strips away the part of an XML document that would prevent the result from appearing as element content.
This is the idea that XForms uses when it expresses XML instance data within a larger document, and specifically within the
This paradigm works well enough if you think of a form as an ephemeral view of a back-end database or content repository. However, people find it easier to work with a document in the file sense of the word. we like to save our document, email it, reload it the next day or on a new computer, and especially on a computer that has become disconnected from the net or from the original web application context that would be associated with the ephemeral view kind of form.
Enter the document-centric approach of Workplace Forms, the technological basis of which is an XML document whose element names belong to the XFDL and XForms vocabularies. Moreover, the underlying XML data document is still incorporated by the
A key difference is that being a document implies that the form with eventually have to be re-serialized, e.g. to save or sign the form. XForms doesn't really talk about how to reintegrate the live, modified XML data back into the document, but in truth it's really not that hard. The main technical hiccup on the XForms side is that the
The conclusion, though, is that a Workplace Form, or XFDL document, is a cross between an office document and an mobile intelligent agent. It can be saved, reloaded, printed, emailed, archived, signed, submitted to web application and portal servers, and participate in document management, records management and content repository processes. This is how the notion of document makes a form be more than just an ephemeral view of data.[Read More]
Today is an exciting day for the IBM Victoria Software Lab because today we have been pushing the new version of the IBM Workplace Forms product line into the IBM release queue. General availability is on June 13th, but it's on its way folks!
The IBM Workplace Forms product line is designed to simplify the process of creating and maintaining forms-based web applications. The flagship knowledge product of the product line is an XML vocabulary called XFDL, which is an XForms-enabled language that provides a rich and precise presentation layer for XML data. The IBM Workplace Forms software supports the full life cycle of XFDL documents, from creation to run-time to processing of completed form documents.
The IBM Workplace Forms Designer is an eclipse-based visual development environment for creating XFDL documents. It has a main canvas for drag-and-drop design of the user interface. The drag-and-drop palette has all the usual suspects for atomic form controls as well as containment controls like repeat tables and group/switch panes. The user can also add custom groups of controls to the palette as single new controls. This makes it easy to standardize constructs like toolbars or address blocks across all forms in a workspace.
It is no surprise that a form filled with the Viewer and with WebForm Server are indistinguishable to the web application developer. Our intent is to provide a single document paradigm for the form design and the web application development. So, both of our form run-time products are based on a common API that understands and implements much of XFDL. Although the nature of XFDL as an XML vocabulary means that standard XML tools could be used to process XFDL documents, the XFDL API is also made available to the web application developer. The idea is that your servlet/portlet will require less coding if you use an API that understands not just XML, but the XML vocabulary of the document. For this reason, the IBM Workplace Forms Server contains not just the WebForm server, but also the API.
Rounding out the Server product is the IBM Workplace Forms Server - Deployment Server. This server facilitates mass deployment of the Viewer and custom viewer extension files by providing an on-demand install experience that can be customized by the server adminstrator.[Read More]
John M. Boyer 060000VMNY 1,642 Views
Since I couldn't put it better myself, let me quote from an email today from John Schumacher, our Development Director for Content Management Applications for Websphere Portal, Workplace and Domino:
"Please join me in congratulating the WorkPlace Forms team. This morning, on schedule, WorkPlace Forms 2.6 was made available for eGA on passport and is also available for download internally on Extreme Leverage!!!! I have already downloaded my copy. Keith MacKenzie and his team have done a tremendous job staying focused on the delivery of this major release ... The Forms products are now Internationalized to help support world wide sales, provides market leading support for the XForms standard which differentiates us from our proprietary competitors... and has aligned our development tool story with the rest of the IBM."
I am very pleased personally that we have been able to create the union of XForms, product globalization, and precision forms technology offered via the XFDL language. It is particularly satisfying this week since the IBM Victoria Software Lab is hosting the XForms face-to-face meeting starting tomorrow. It's always good news to see XForms implementers deriving power, efficiency and overall benefit from XForms, but being able to announce general availability on the first day of the face to face meeting you are hosting... cat to the cream!
And finally, this release and the completion of the XForms face-to-face will open up my ability to talk about more pointy bracketed things on this blog in the coming weeks :-)[Read More]
We've been quite busy today in the XForms working group. Our work has been focused on modifications of the functionality of XForms submission for XForms 1.1. Tomorrow we will address any details that come up as the pieces of the spec are put together for the next working draft, but here is an overview:
Tomorrow we will be discussing aspects of the XForms type model item property. I'm anxiously awaiting the outcome because, frankly, I've been waiting a few weeks now to tell you about these aspects, but I need the group resolutions to occur before posting my comments. I think it will turn out well, though![Read More]
I got a note from Eric Bruchez reminding me to comment that XForms 1.1 will also make available submission response header information in the xforms-submit-done event context, which helps complete the story about ATOM.
Yesterday was also a really good day in which we worked out some challenging issues related to the XForms
One reason this was important was for ensuring that we had a proper answer for the last technical question about XForms basic. We confirmed it was not an issue for basic, so you can expect to see basic progress through the rest of the W3C process RSN (real soon now).
We also came to agreement on the meaning of the
And the answers are... the
This is a reasaonably nice synchronization of what a user can modify via user interface bindings, what a calculate can modify, and what the type MIP actually validates.[Read More]
John M. Boyer 060000VMNY 1,579 Views
In the final day of the XForms face-to-face meeting today, we made great progress for XForms 1.1 defining synchronous or asynchronous submission.
A number of current implementations of XForms 1.0 perform instance data relevance pruning and validation followed by initiation of the data submission. The upload and download of octets occurs asynchronously, and when completed, the xforms-submit-done or xforms-submit-error is dispatched. However, the current recommendations for (DOM2) XML events and XForms actually describe a synchronous procession that starts with dispatching an xforms-submit through fives steps of default processing from relevance pruning and data validation all the way through to completion and dispatch of xforms-submit-done or xforms-submit-error.
A difference of behavior occurs when an action sequence contains a
In reality, form authors should put the follow-on actions after a send into a handler for the xforms-submit-done event, since actions over data should only be run if the submission succeeds. Still, what's a working group to do?
We'll be adding a new attribute, probably called mode and probably having the value synchronous or asynchronous, so that the form author can choose which behavior is best. The synchronous behavior will have other advantages like suspending user interaction with form controls, but the default will undoubtedly be asynchronous.
Meanwhile, XForms 1.0 authors, get into the habit now of putting actions that operate over submission response data into the xforms-submit-done handler, not after a send action.[Read More]
John M. Boyer 060000VMNY 1,429 Views
Now you can get free access to the trial versions of IBM Workplace Forms software. The trial software includes the Eclipse-based visual design environment as well as the rich client viewer (a desktop application and web browser plugin). The suite includes a server component that delivers the form viewing and rich user experience with a zero footprint on the client (using the existing web browser only with no client-side install). However, the server product is not part of the trial software.
Workplace Forms supports development and deployment of XML forms and XForms-based XML forms in the following languages: Chinese Simplified, Chinese Traditional, Croatian, Czech, Danish, Dutch, English International, Finnish, French, German, Greek, Hungarian, Italian, Japanese, Korean, Norwegian, Polish, Portuguese, Portuguese Brazilian, Romanian, Russian, Slovakian, Slovenian, Spanish, Swedish, and Turkish.
One final note is that in order to get the trial software, you have to get an IBM developerworks ID. But that's free and easy to do, and the benefits immediately begin with access to the most powerful, advanced, pure XML electronic forms system on the market today.[Read More]
John M. Boyer 060000VMNY 1,680 Views
Someone wrote me recently asking how they could aggregate a number of form controls together to achieve a common purpose.
Sidebar: It is legal to use the word they as a singular pronoun. And not just when the answer to the question he/she or s/he asked is applicable to a plurality of people, it can be used at any time to avoid those horrible slashing language kludges. Still, I have to admit it's easier on the ear when the answer applies to many.
Of course, the answer to the above question applies to many! I'll use the syntax of pure XForms as a pseudo-code for your visual experience in the Workplace Forms designer, where you can add whatever presentational beautification seems appropriate. So, let's say you want to aggregate the input controls to collect the parts of a phone number. Since forms are often written per locale, I'll focus on the North American format, since the same idea applies to other formats. The format consists of a 3 digit area code, a 3 digit exchange and a 4 digit local. So your instance data would look something like this:
<xforms:instance xmlns="" id="X"> <phone> <areaCode/> <exchange/> <local/> </phone></xforms:instance>
Now you'd hook up input controls to allow the user to enter data for each of these parts. We'll talk about the elided part of each control in a moment:
<xforms:input ref="areaCode" incremental="true" id="areaCode"> <xforms:label>Area Code</xforms:label> ...</xforms:input>
The automatic tabbing feature begins with the
Every time you change the data associated with an XForms form control, the control receives an
<xforms:input ref="areaCode" incremental="true" id="areaCode"> <xforms:label>Area Code</xforms:label> <xforms:setfocus ev:event="xforms-value-changed" control="exchange" if="string-length(.)=3 and . >= 0"/></xforms:input>
OK, all of this is simple enough, but why did I use the future tense. The reason is that XForms 1.1 is a working draft. XForms 1.0 is the recommendation, so what can you do now. Well, for all of its elements, XForms 1.0 defines the "Common Attributes" bundle as including any foreign namespaced attributes. This was done to allow host languages (like XFDL for Workplace Forms) to be able to add their own enhancements in advance of the recommendations to meet pressing business needs and also to experiment with the best ways of adding new features to the XForms language. In this case, the need for conditional execution of an action is clearly needed, and it's pretty much a done deal that the
<xforms:input ref="areaCode" incremental="true" id="areaCode"> <xforms:label>Area Code</xforms:label> <xforms:setfocus ev:event="xforms-value-changed" control="exchange" xfdl:if="string-length(.)=3 and . >= 0"/></xforms:input>
That's all folks![Read More]
John M. Boyer 060000VMNY 3,509 Views
Continuing with the last blog, you saw autotabbing at work, and clearly three XForms controls were working together to collect some common data. But here we're going to refine the aggregation of the controls in both the user interface and the model layers.
For starters, suppose a larger example in which the phone number being collected is not the only piece of data, but rather is nested deep within a larger XML data structure:
<xforms:instance xmlns="" id="X"> <data> ... <customer> ... <phone> <areaCode/> <exchange/> <local/> </phone> ... </customer> ... </data></xforms:instance>
In the prior user interface, the three input controls relied on the fact that the default evaluation context for user interface bindings is the root element node of the instance. Even though the
<xforms:group ref="customer/phone"> <xforms:input ref="areaCode" incremental="true" id="areaCode"> <xforms:label>Area Code</xforms:label> ... </xforms:input>
The second thing you may want to do is to combine the three components of the phone number together into a single result. There are quite a few ways to do this, so here we'll just pick a simple way and run with it. Say, you were to add an attribute called
<xforms:instance xmlns="" id="X"> <data> ... <customer> ... <phone number="2505551212"> <areaCode>250</areaCode> <exchange>555</exchange> <local>1212</local> </phone> ... </customer> ... </data></xforms:instance>
It's then an easy matter to combine the three phone number components together to form the full result using an XForms calculate formula:
<xforms:bind nodeset="customer/phone/@number" calculate="concat(../areaCode, ../exchange, ../local)"/>
The advantage of XForms calculation is immediately clear when you consider that any user data entry
I have a three items of interest for you today.
First, I've updated the wikipedia entry for XForms. It now mentions the IBM Workplace Forms implementation in appropriate places. I also updated it to make the intro more understandable and to draw attention to the Ajax-like behavior of XForms instance replacement submissions. I also added a few more external links, including links to XForms-related blogs (including this one, of course).
Second, in its ongoing effort to refine the technical precision of XForms, the working group today published a collection of errata for XForms 1.0 Second Edition. In my view, the three most important are
The working group currently plans to publish further errata for XForms 1.0 Second Edition in August.
And thirdly, the working group today initiated the publication of an update XForms 1.1 working draft, which should appear by the end of the week. This will be the second to last "thin" spec (a spec describing only new features). We expect to publish one more thin spec near the end of August, then we will migrate to a full spec by merging XForms 1.0, the errata, and the new features in the thin spec.[Read More]
John M. Boyer 060000VMNY 1,356 Views
Well, I haven't blogged for the last couple of weeks because, as it turns out, I was swimming with the sharks!
Well, actually, I tried my best not to be in the water when a pair of hammerheads and a trio of grays came callin', but one morning my wife and I had to rush into the water to help an older man who was merrily floating along, ears below the water line and a pair of hammerhead fins vectoring his way.
Otherwise, a peaceful and restful vacation and back in the XForms saddle. And while I was away, the W3C publication process did indeed unfold as expected (with a little help from Steven Pemberton) to allow delivery of the latest working draft of XForms 1.1
Notice that the above link takes you to the diff-marked version of the spec. These show the differences between the current working draft and its predecessor, which was published in December of 2005. If you have familiarity with the prior spec, this allows you to see the new things that have made it into XForms 1.1, and the changes to existing features of XForms 1.1
There are quite a few smaller tweaks here and there, but I'll focus a bit here on the big ticket items. One is the effort the XForms team has been putting into the access that XForms authors will have to event context information in XForms actions. This is going to be a very powerful addition to the language. The other major area of change involves the addition of features to the XForms submission module. The goal of these changes is to open up XForms to a wider variety of sympathetic web technologies. Frankly, there is an interplay between the changes we made to submission and to event context. Expect some further changes and additions in the next working draft.
A small change that has, perhaps, the most important and far-reaching implications is the namespace URI that will be used for XForms 1.1. This is a very challenging issue, but the net is that XForms 1.1 will use the same namespace URI as XForms 1.0. Like those sharks, this really bites in some ways, but also like the sharks, it turns out to be unavoidable. Stay tuned, and I'll tell you why in an upcoming blog entry.[Read More]
There's a new movie due out soon with a really cool name (Snakes on a Plane) and starring an actor I like, Samuel Jackson. Josh Friedman was supposed to help edit the screenplay, but he walked when studio execs threatened to change the name to something banal. However, Sam apparently signed up for the movie because of the name, so it had to stay. Anyway, Josh blogged about months ago, and since then the movie itself has taken on this kind of Web 2.0 existence where the people's conjectures about the movie's content have actually caused the studios to adapt the movie!
Aside from changing the actual film content, the fan base (surreal as that sounds) have actually been responsible for creating marketing hype, movie paraphernalia and even an internet meme for the title. Apparently, Snakes on a Plane is now synonymous "That's Life!" or my personal favorite way of saying the same ("Shh! It happens.") An example from everyday life might be
Programmer on Friday afternoon: "I was going to be done by code module today, but such-and-such didn't properly implement the standard, so now I have to work around their bug."
Hip Manager: "Snakes on a plane. What time did you say you'd be in tomorrow?"
Neat but what's this got to do with Workplace Forms, you say. Well, it's really good to know background cultural information no matter what. Background information is always good to have around. Take for example, an exciting new extension I've recently heard that someone has built for the Workplace Forms Designer. It allows a form author to select a group of form controls, it then figures out the associated markup, and allows the form author to indicate how many times to iterate that markup. Makes creating tables a snap!
Snakes on a plane! Here we are talking about all the cool new features that will be coming in the future of XForms, and the background cultural information that clearly needs further socialization is that
That last point is really important. Suppose you have some popup selection control that you need duplicated 15 times. Then you decide it needs to have another selection. If you use an XForms repeat, you only have to change the one selection control in the repeat template. If you duplicated controls at design time, then you would have 15 changes to make.
But, hey, at least when the customer asks how to write more maintainable tables, we already have an answer.[Read More]
John M. Boyer 060000VMNY 1,432 Views
I'd like to make this extra posting this week to draw your attention to a series of excellent articles recently completed by Kurt Cagle. The series is entitled Understanding XForms and consists of the following entries:
These articles give a lot of information about the value proposition of XForms to projects, and they're worth your time.[Read More]
A few posts ago I promised to get back around to the discussion of the namespace and versioning of XForms 1.1.
In the early days of XForms 1.1 (late 2004), we expected to upgrade the namespace URI as the means of indicating which processor should be used to interpret the XForms vocabulary within a document. There were, after all, not just new elements but also new attributes for existing elements and in some cases slightly adjusted behaviors for the same elements.
Well, a lot can happen in 18 months! For one thing, we originally intended to publish XForms 1.1 as
The second thing that change is that we decided that an important aspect of increasing XForms adoption was to make it easier for web content authors to write. For this reason, the HTML working group communicated the need to import the XForms module directly into the XHTML2 namespace rather than leaving it in the native XForms namespace. To accommodate this requirement, we created the notion of a chameleon namespace in the schema for XForms 1.1. This concept allows us to define XForms in the XForms namespace, but it allows a host language like XHTML to more easily import the XForms vocabulary by substituting its own namespace instead of the default XForms namespace.
Of course, any host language that attempts such an import has to deal with possible name conflicts, and we did encounter some, like the
So, to be clear, XForms content should remain in the XForms namespace whenever possible, such as when used with host languages other than XHTML, because there are significant technical benefits to doing so.
Now, how does this relate to the versionining of XForms or the fact that XForms 1.1 will use the same namespaces as XForms 1.0? Well, along separate lines, the XForms working group had received a number of communiques indicating that the change of namespace made it harder to write software that processed XForms content in ways that were not necessarily affected the vocabulary changes. Basically, the change of namespace was viewed as costly to the community. While mulling over this feedback several months ago, I realized that people who were dissatisfied with our change of namespace URI between XForms 1.0 to XForms 1.1 could simply use the chameleon namespace feature to put the XForms 1.1 markup into the XForms 1.0 namespace. They didn't have to use the feature to put the markup in the namespace of a host language.
This thought was very good news for solving the problem of those who wanted us not to change the namespace of XForms in 1.1. Since the chameleon namespace feature could defeat the intent to use the namespace URI to control the language version, clearly we had to change to versioning 'within' the XForms vocabulary. Hence, you can see in the latest working draft of XForms 1.1 that we now have reverted to the one and only XForms namespace URI originally allocated in 2002, and we have added an internal version attribute on the XForms model, defaulting to 1.0, to indicate the version of the language.[Read More]
John M. Boyer 060000VMNY 1,523 Views
On a recent ad for a movie or TV program, I heard an irreverent but nonetheless funny line about there being no team in I. Good for a laugh, but it did remind me of that more deeply satisfying idiomatic saying: There is no 'I' in team. As well, it reminded me that I wanted to say a thing or two more about a recent interview I did with Jonathan Erickson for the Dr. Dobb's Journal Developer's Diary.
The purpose of the diary is to provide snapshots in the lives of real developers at various stages of their careers to find out what they do, what makes life harder for them, and what makes their job easier. I was honored to be selected, and Dr. Dobb's did a good job of relating what I said. If you read the print version, you'll see that they said 'Turing incomplete' instead of 'Turing complete' but that was immediately corrected in the online version. But the main thing I wanted to talk to you about was the question of what makes my job easier because I talked about two things, but they only had space for one.
In response to the question of what makes my life easier, I would say that I see my job as a senior technical leader being comprised of both challenging problems to work on as well as interesting people to work with. On the problem-solving side, I made mention of the important role that algorithm analysis has played in not only effective problem-solving, but in seeing where the problems really are and how to either solve or sidestep them as needed. However, I'd like to at least make public my thoughts on the people side of my role, too.
Specifically, I would like to call your attention to the Workplace Forms team and the XForms community. Whether I am directly participant in a collaboration or simply observing the members of either group, I have to say that the level of dedication you demonstrate inspires me. You, the people, certainly give meaning to my own work, and I am proud to be working with you to build Workplace Forms and XForms.[Read More]
John M. Boyer 060000VMNY 1,841 Views
The W3C recommended a migration to XML of web content in January 2000. It's called the XHTML recommendation. The language was further updated to XHTML 1.1 in 2001, and again it was approved by the W3C membership as a full recommendation. The point is that these recommendations represent more than 6 years of the W3C recommending an XML basis for the future of the web.
The XForms working group defines an important web technology: the next generation of web forms. The technology became a recommendation in 2003 and had a significant refinement earlier this year. Again, XForms has been approved by the W3C membership.
One reason behind the strength of the XForms community is its broad applicability as a web technology. XForms defines the core XML data processing asset, whether the data processing needs be simple or complex. It is designed to be connected to numerous host languages like XHTML, SVG, Voice XML, XSL-FO and even XML vocabularies like XFDL (the one used on Workplace Forms to provide a secure, high-precision presentation for XForms).
The point is that the web is more than just what web browser makers implement. Part of the reason for this is that web browser makers don't tend to make much money on web browsers, so their desire to pour development dollars into their proper maintenance is stunted. As a result, web technologies have flourished from numerous third parties who are helping to define the future nature of web content. Frankly, the browser makers so this coming and created the notion of a plugin precisely to allow web technologies to flourish without their having to do all the work. Really, it's analogous in that respect to open source.
I think the W3C has somewhat lost sight of these facts recently. There appears to be some fear that browser maker lag in implementing some of the recommendations equates with the W3C not "leading the web to its full potential." I respectfully disagree. The situation is not nearly so black and white. The web is supposed to work this way. The web cannot afford to be constrained to the limitations of a browser maker hegemony, especially when they don't even really want the job and have given us the tools to catch our own fish, so to speak.
This brings us to the topic of the recent escalation of Web Forms 2.0 from a member submission by a web browser maker to a first public working draft. The original W3C Team acknowledgement strongly encouraged that the work of accounting for the member submission be placed squarely in the new charters of the XForms and HTML working groups. I can only conjecture that its publication as a working draft of the WAF WG was designed to make sure that the issues raised by Web Forms 2.0 do get addressed, and right soon.
Don't get me wrong. Web Forms 2.0 has some good ease-of-authoring features. By ease-of-authoring I do also mean to imply a smoother migration path for existing HTML-based web content. Members of the XForms working group have been tracking the development of this document, and it turns out to be not too difficult to accommodate the most critical features. However, there are features of Web Forms 2.0, like the repeat construct, which diverge from the conceptual model of XForms for no apparent reason. I would expect that such divergences would be replaced by the XForms basis in the future. One aspect of Web Forms 2.0 that does need the most work is that it really is asking the W3C to take a giant 6 or 8 year leap backward away from an XML basis for web content. This seems a less viable demand, esp. since tag soup HTML UAs are perfectly capable of recognizing markup that isn't ill-formed!
In general, it is true that ease of browser implementation of new features is one important concern, but IBM believes that there are many important factors that contribute not only to the total cost of ownership of web applications but also to the ability to create and maintain a significantly broader spectrum of current and future web technologies. IBM believes that these factors are important enough that it is essential to use new features of HTML as an enticement toward greater conformance of web content to XML syntax. Moreover, IBM strongly advocates for the renewed charter of the XForms and HTML working groups to include unification of the Web Forms 2.0 work with emphases on the ease-of-use benefits from WF2 and the XML basis from XForms. There will be compromises required of all parties, but also significant synergies that become possible by accommodating the full range of forms expertise available in the W3C.
The above are excerpts from the full IBM position statement, which has been posted to the W3C chairs list, the W3C AC Forum, and the public lists of the XForms, HTML and Web Application Formats working groups. See the XForms posting and the WAF WG posting.
In conclusion, I do not by any means want to constrain XForms to only one host language (XHTML). However, I also do not want to downplay the importance of the XHTML host language in helping to bring the great technology that XForms is to the people. The XForms working group is committed to a unification of the best that the Web Forms 2.0 has to offer. We already have a decent implicit data generation feature, and we look forward to expanding that to implicit data model generation so that the easier
Throughout the history of mathematics, new ideas have tended to be labeled in a demeaning way. The first few that come to mind are negative numbers, irrational numbers, and imaginary numbers. It's always as if the person who invents a new concept to solve a problem is somehow losing his mind because all the sane people use numbers that are positive, rational and real!
The same thing seems to be happening in the web world. I've recently heard the assertion that it is illegal to use XML constructs to upgrade the HTML language because it is illegal to send content over the wire as anything but tag soup if the content-type is text/html. On the one hand, I could be losing my mind, but on the other, maybe the assertion doesn't hold up to scrutiny. The argument seems to be that the user agent would have to commit the sin of content-sniffing in order to determine whether or not the text/html content can be fed to an XML parser. See how the label 'content-sniffing' just oozes negative connotations?!
Well, all of those clever mathematicians of old won out in the end because, quite frankly, their ideas worked. And so it will be with content-sniffing.
To be honest, I don't see the logical difference between testing a document for XML well-formedness versus checking a
In fact, all features that are placed into IBM Workplace Forms are activated by version. So, if you write a form using, say, version 6.0 of the XFDL language, the form continues to work in the same way in the latest version of the product. Changes to the way features work, and even most bug fixes, are implemented in a version-specific manner so that forms generally don't change behavior as the user agent is updated.
In essence, the Workplace Forms viewer products do a form of content-sniffing by activating certain subprocessors based on some of the content in the document. The point is that the document is still of content type application/vnd.xfdl, but we don't assume that a document processor must be dumb except for the information it derives from HTTP content negotiation. Perhaps this comes from our more document-centric focus-- after all, you don't get HTTP content negotiation when loading a form previously saved to the local disk drive!
The same kind of thing needs to happen with HTML processors. The html tag could bear a boolean attribute that indicates whether or not the content is well-formed XML. Or, there could be added a version tag whose value would imply whether XML was used. This way, new features of HTML could be added to the XML-based variant, and user agents could be
But either way, serving out well-formed XHTML with the content type of text/html is not only a valid thing to do, the simple truth of the matter is that people do it all the time. Why? Because it works.[Read More]
The XML Schema language is a good language for capturing the syntactic constraints of an XML vocabulary. But let's face it, it is really designed more for describing data. It's just not powerful enough to be used for making normative contributions to W3C Recommendations like, say, XForms.
This is not to say that a disagreement between the recommendation and the schema won't sometimes be resolved in favor of what the schema says, but there is a reasonably popular view that the schema is also normative. Actually, the recommendation is normative. In fact, the recommendation is typically provided in two or three formats (sliced up version, single HTML file, diff marked version), and only one of those is considered normative. Even more to the point, English is, for better or worse, the only language in which the normative recommendation is expressed. Any other versions are also considered to be informative.
Quite apart from what anyone tells you about the normativeness of a schema, you can determine that the schema associated with a recommendation is informative, not normative, if it does not appear in TR space. This is a subdirectory of the W3 website named 'tr' and used to publish all technical recommendations of the W3C. In the case of XForms, the schema lives in the working group space, not TR space, so it is informative.
However, the location of the schema isn't really the deciding factor for me. Even if a schema did appear in TR space, it still would only be informative in my opinion because there are lots of language constraints that you just cannot express in schema but which are expressed in the recommendation. A number of great examples of this can be found in XForms, two of which are explained below.
Perhaps the easiest is the use of XPath expressions in XForms. In the XForms schema, a number of the attributes like
<xsd:simpleType name="XPathExpression"> <xsd:restriction base="xsd:string"/> </xsd:simpleType>
Shock of shocks, it's a no-restriction restriction from the base type of
The point is that schema just can't touch this. But, the rules of XPath are clearly and normatively defined in the XPath recommendation, and the XForms recommendation dutifully cites XPath in its list of normative references. That's why the recommendation text is normative and the schema isn't.
Not to put too fine a point on this, but you might claim that the XPath example is a perverse one. Well, it's not the only one. For example, a number of XForms elements like
John M. Boyer 060000VMNY 1,316 Views
The Workplace Forms Designer allows the form author to use an XML schema to automatically generate the instance data for an XForms model. The data nodes are needed to allow drag-and-drop associations to be created between the data and user interface components. You can drag from an existing UI control onto the data to make the association, or you can drag the data onto the form design canvas to cause the creation of a UI control that will then be bound to the data node. Either way, in XForms the association is made between instance data nodes and UI controls.
This is true even when the XML schema contains an element whose content is a choice. When this happens, the Workplace Forms Designer generates data according to the content of all possible choices. This allows form design to proceed for each of the possible choices. The expectation from XForms is that the form author will make all of the possible choices be non-relevant except the one actually chosen by the end-user. For example, if you have a choice of address block based on country selection, once you select "UK" then you get the UK address block, and once you select "US" then you get the US address block, and so forth. All address blocks other than the one for the selected country become non-relevant.
The use of non-relevance in XForms is significant. It affects two things, but the most important is that all non-relevant nodes are removed from XML data at the start of submission processing. This is important because an XML element with a schema choice content model is not valid until all but one of the choices is removed. This is called "pruning non-relevant nodes" in XForms, and it means that XForms does not expect instance data to be schema valid
The public Forms working group mail list has been getting some traffic recently about a correction we made to XForms 1.0 in the second edition. Under the original recommendation, XForms processors had to take a snapshot of initial instance data before any data is filled in so that the empty version of the data can be used to insert new rows of data into a repeat. This is an idea that looks elegant at first, but it breaks down really quickly.
In language design, it is important to have a balance between the appearance of elegance and elegance in fact. Keep things as simple as possible, but no simpler! The initial data snapshot idea is too simple because it works in some cases, in others it only works with a highly sophisticated implementation, and in some reasonable use cases, the insert simply could not act as specified if the initial data snapshot method is used. For example, if the initial instance data did not contain the prototype because the repeating data is obtained by a submission after the form starts, then the insert would fail.
Due to these difficulties, many processors did not even implement the idea. Instead, they were implementing the insert action by making a copy of the last node in the currently running instance data. To correct the design and reflect the will of the XForms community, the recommendation was amended to say that insert creates a new node by copying the last node of its nodeset.
At the same time, XForms was also corrected to say that insert and delete perform no operation when their nodesets resolve to empty nodeset. It seems obvious to say this because if there are no nodes in the delete nodeset, then there is nothing to delete, and if there are no nodes in the insert nodeset, then you wouldn't know where to put a newly duplicated node even if you had one. However, we decided to explicitly say this so that all implementors would know to check for this condition rather than, say, assume they should fail with a binding exception. The reason is that the no-op behavior actually comes in quite handy in solving the full spectrum of use cases for repeats.
Here is an example. It shows how to manipulate a repeat that starts empty and is allowed to become empty and non-empty during run-time. Like a shopping cart. It starts empty, you add stuff to it, then you change your mind and delete the shopping cart items, and then you choose more stuff to buy.
<!-- Initial instance data contains the prototypical node as the last element -->
The above example showed that having the delete action resolve to no-operation when its nodeset is empty is very useful. Now, let's see an example in which it is useful for insert to have no effect on empty nodeset. Suppose that instead of a zero row repeat, you want a repeat that stays at one row. If the user hits delete for that row, then the row stays, but the data is cleared from it.
<xf:trigger> <xf:label>Delete <xf:action ev:event="DOMActivate"> <xf:delete nodeset="item[position()!=last()]" at="index('repeat-cart')"/> <xf:insert nodeset="item[last()=1]" at="1" position="before"/> </xf:action></xf:trigger>
Here the delete trigger could be activated a number of times, but it will only deleted down to the last row of data that the repeat is actually showing. When the number of elements drops to just the original prototypical data, further pressing of the Delete trigger produces an empty nodeset, so the prototypical data is not removed. The insert follows that up with a nodeset that resolves to empty unless the only data left is the prototypical data. If there is anything more, the insert has no effect. But when only the prototype is left over, we make a new copy of it, which presents a single empty row of data to the user.
The neat thing here is that the above is a general method in which a constant number of XForms actions achieve the desired result regardless of the schema for the data. The data may have inner sequences that correspond to inner repeats in the UI, and you can still use this method on the nodes bound to the outer repeat because the last row of the outer repeat is deleted no matter how many rows are on its inner repeats, and then the insert puts in place a copy of the prototypical data in which node sequence for the inner repeat is in the initial state.
So, you can create repeats that can start and return to the empty state, create repeats that enforce a one row minimum, you can create repeat/insert/delete constructs that operate correctly regardless of when the data is obtained in the lifecycle of the form, and especially, you can create repeat/insert/delete constructs that operate correctly in the save/reload case (e.g. in the presence of prepopulation data). And you can do it without using special attributes being added to XForms 1.1. You can do it today, right now, in XForms 1.0.[Read More]
We're just finishing up the XForms face-to-face meeting in Amsterdam. Given my prior post and the approach of XForms 1.1 toward last call status, it seemed a good idea to talk about the improvements to XForms 1.1 that make the repeat construct easier to work with.
It's about being able to say more exactly what you mean, really. For example, on my first night in Amsterdam, my hotel room became quite cool, but it wasn't clear how to turn up the heat. Apparently I'm just not old school enough to relate to a radiator, so I called down to the front desk. I was informed that in order to turn the heat up in my room I had to "squeeze the knob". On the radiator was implied. Despite being pretty good with a pun, it seems I still needed a friend to help me fully appreciate how important my complaint about the advice truly was, especially in Amsterdam. In fine propeller head form, I complained that the proper advice was to "turn" the knob. On the radiator was again implied. Oblivious to any possible alternate interpretations, I proceeded through the explanation that while squeezing the knob was a necessary component of turning it, it was also an implicit part of the process which created the friction necessary to ultimately turn up the heat. From the radiator was implied.
Anyway, the point is that a lot of trouble can be avoided when one is able to say exactly what one means to say. In the case of XForms 1.1 authors, there are two new attributes on the insert action that make it a lot easier to manage repeat constructs. In XForms 1.0, we add to the container element of a sequence an extra subelement to act as the prototypical data. We must add the prototype as a child of the container element because the insert action in XForms 1.0 can only get the prototype from the last node of the sequence of items over which it operates. This limitation forces you to adjust the repeat to omit the last node, and it forces you to add application logic to remove the last node when the data is submitted or alternatively before it is processed on the server side.
In XForms 1.1, there is a new origin attribute on the insert action. This attribute allows you to give an XPath that says where the prototype for insertion is located. This allows it to be placed in another instance rather than being stored in the container element of the sequence, which eliminates the necessity of adjusting the repeat and of removing the prototype later (it's already in a separate instance).
The second issue that we solved in XForms 1.0 by putting the prototype data at the end of the container element is that we avoided the problem of the container element becoming empty. It is easy to understand the need for an empty shopping cart, but if the
In XForms 1.1, we solved this by adding a context attribute to insert. The context attribute set the context for evaluating the nodeset attribute. The expected use of this attribute is to choose the parent or container element of the nodeset. So, when the nodeset resolves to empty nodeset, the context node is used as the container item into which the prototype node is inserted. So here's the shopping cart example in XForms 1.1.
<!-- Initial instance data contains the prototypical node as the last element -->
To complete the example from XForms 1.0 in XForms 1.1, we should now look at an example in which you want a repeat that stays at one row. If the user hits delete for that row, then the row stays, but the data is cleared from it.
<xf:trigger> <xf:label>Delete <xf:action ev:event="DOMActivate"> <xf:delete context="/cart" nodeset="item" at="index('repeat-cart')"/> <xf:insert context="/cart" origin="instance('protoCart')/item" if="not(item)"/> </xf:action></xf:trigger>
In the final insert, we see the appearance of the new if attribute to more precisely communicate that the action is conditionally run. In XForms 1.0, you have to use an XPath predicate to produce an empty nodeset, which is a bit like saying "squeeze the knob". It gets the job done, but it ain't pretty.
The final insert also shows off one of the new things decided at this face to face meeting of the XForms team. In the latest working draft, the nodeset binding is listed as required, but we are changing that to optional in order to make writing inserts like the final one above look more natural. It just identifies the container element as the destination of the insert, the origin as the source of the insert, and a condition that says when to do the insert. And now my room is too hot![Read More]
John M. Boyer 060000VMNY 1,390 Views
An interested reader asked "How do you scroll a repeat?" so that it's content doesn't visually spill onto other controls. The question was specific to the Workplace Forms product, but the answer is reasonably generic.
One answer is that when you put things below a table (the XFDL element that contains and presents an XForms repeat), don't use absolute positioning. Use relative positioning. Use an itemlocation that puts the content "below" the table so that as the table grows, the content below it automatically shifts down.
The above suggestion is useful if you are allowed to the let the content grow arbitrarily. However, if you are trying to match the appearance of a paper form, then the table/repeat must fit into a prescribed space and location. So, you ask, how do I limit the size of the table and allow movement to the extra data?
Well, the first response is to ask if you are sure that's what you want. There are no scrollbars on paper, so if you try to print your form, you will not be able to see all of the table content. You may actually want to limit the table size to some maximum number of rows. It is very easy to make the insert trigger (button) become non-relevant when a maximum number of rows is reached. The easiest way is to add a
<xf:trigger ref="item[3 >= last()]"> <xf:label>Insert <xf:insert ev:event="DOMActivate" nodeset="../item" at="index('repeat-cart')+1" position="before"/></xf:trigger>
Relative to the post on Nov. 3, two things happened. The
However, when the total number of item elements is three or lower, then the trigger
Finally, let's assume you don't care about the printing problem and so you really do want to have a 10 row table fit into the space of three rows with scrolling. Well, an XForms repeat just repeats things, it doesn't decide whether or not to grow a scrollbar. But, you can use an XPath predicate on your XForms repeat to control how many nodes it is showing. In fact, we highly recommend that you do use such predicates to keep repeat tables down to a reasonable size for efficiency. There's no point in creating 200 rows when the user can only really see 10 or 20 at a time!
To do repeat scrolling, you should create an instance data variable that can hold an index of your current location within a larger dataset. You could have a second variable to control the size of the table, or you could just pick a constant. I will pick 10 here. You then adjust your repeat nodeset as follows:
<xf:repeat nodeset="item[position() >= instance('temps')/curpos and position() < instance('temps')/curpos + 10]"> ...</xf:repeat>
Now, you just need scroll up and scroll down triggers (buttons) that add or subtract 10 from the curpos variable using a setvalue action. Of course, you should also use the above trick to make those triggers become non-relevant when doing so would cause the curpos variable to go out of bounds (less than 1 for the scroll up trigger, and greater than count(item) for the scroll down trigger).[Read More]
John M. Boyer 060000VMNY 1,781 Views
There are lots of scenarios in which additional work must be performed on an electronic form document after a digital signature is affixed. For example, suppose you are filling out a form that has an "office use only" section in it. When you're done, you want to sign the document to authenticate yourself and authorize the signed content. But those "office" people still has work to do to fill out that office-use-only section after you sign and submit the form to them. Another easy example is the multiple signer scenario. For a simple signer/co-signer loan, the loan applicant has to sign the document, and then the co-signer signs it.
In cases like those above, it is clear that the office-use-only section of the form was blank whent he signer signed or that the co-signer had not yet signed when the loan applicant signed. But if you modify the document after the first signature is affixed to fill out the office-use-only section or to add another signature, this seems to be indistinguishable from tampering with the document. The document hash in the digital signature would detect the change, and the user would be told not to trust the document any more.
Fortunately, the XML digital signature world has a solution for this problem. It is called a digital signature filter. A filter allows the document hash in the signature to cover part of the document instead of the whole document. This makes it easy to create a digital signature which logically covers the whole document except for the office-use-only section or the co-signer signature.
Unfortunately, there is a bit of a tendency in the wild to misuse digital signature filters, and the result is forms that do not provide nearly as much security. A proper digital signature filter should almost always use exclusive logic. The digital signature filter should say "sign the whole document except ...". The elided part that forms the exception should, as precisely as possible, indicate the part of the document over which additional work must be performed. For example, "sign the whole document except for the office-use-only section" or "sign the whole document except the co-signer signature".
Almost always, a digital signature filter that uses inclusive logic is evil. People have a natural tendency of wanting to say "sign this and this and this" rather than saying "sign everything except for this and this and this", but the former inclusive logic statement has far less power than the latter exclusive logic statement. The reason has to do with the fundamental principle of digital signatures: What you see is what you sign. For the visually impaired, the word 'see' is being used metaphorically, of course, but the point is that users do not care about octet stream security. They care about whether or not the technology has secured the on-the-glass appearance of the document.
So, when an inclusive signature filter identifies markup for what must be signed, it implicitly omits anything else that might be in the document. This means an attacker can add, change or delete any other content before or after the signature creation occurs. The problem is that such arbitrary content can include commands like 'position me over top of the real terms and conditions of this contract'. So, even though the digital signature continues to validate, the end-user would no longer be looking at the same terms and conditions that the signer saw when he signed the document.
An exclusive signature filter does not allow arbitrary content to be added, changed or deleted. The only additions, changes or deletions allowed are those that are explicitly spelled out in the exclusive signature filter. It is easy to only allow the data and not the layout of the office-use-only section to be changed in the document after a signature is affixed. And it is easy to have the loan applicant signature sign the whole document except for allowing the co-signer signature to be added, again with no changes of the overall form layout. Everybody knows that these things were empty when the first signature was affixed, so the exclusive filter allows only the changes that are reasonably expected of the document.
There is a viable use case for an inclusive signature filter. It is useful as an optimization when a signature is co-signing over a second signature. For example, in the signer/co-signer scenario, the first signature covers the whole document less the second signature. The second signature could cover the whole document, but it really only needs to cover itself plus the first signature because the first signature already covers everything else with an exclusive signature filter. However, the whole signature validation step is fast enough that it is rare for this optimization to make a meaningful improvement, and it will certainly never be more than twice as fast as just signing the whole document a second time. This is why I recommend that people should simply never use inclusive signature filters.
To be honest, the greatest disservice of inclusive signature filters is not even to signers who may be duped by the unscrupulous into authorizing transactions that they don't really approve of. The greater disservice is to the organization that deploys a security system with the understanding that it will reduce the risk of transaction repudiation. A signer can simply show that it is possible to attack a signature containing an inclusive filter, so no one has to even change a single byte before or after the signing event and the signature can still be repudiated! Again, this is why I recommend that people simply should never use inclusive signature filters. By which I mean, just don't ever use inclusive signature filters. And Happy Holidays.[Read More]
John M. Boyer 060000VMNY 1,595 Views
It's an exciting time to be in the Lotus division of IBM! The Workplace Forms team is busily preparing for numerous presentations and labs to be held at the annual Lotusphere conference next week. This is the time to see the best of what Lotus has to offer, so of course the Workplace Forms team has lots of content to contribute.
Look for Mike Rhodin, the chief of chiefs in Lotus, to include Workplace Forms in his demonstration of social networking software.
Look for strong presentations, round tables and BOFs throughout the week.
All week you can Meet the Developers at two labs. You'll meet Mike Mansell and Chris Phillips to talk about the Workplaces Forms Services Platform, a new offering that streamlines the development of the server-side processing within Forms Applications. And you'll meet Veronika Irvine and me, who are there to show you anything you want to see on the Workplace Forms Designer, Viewer and Web Form Server products, including what's coming in the upcoming 2.7 release.
And finally, take some time to visit the User Experience lab, where you can meet Jeff Sokolov and test drive the Workplace Forms products and provide your own feedback on what you like and what you'd like to see in the future of these products![Read More]
I have to say it was no small perk for Lotusphere participants that the guest speaker for the opening general session was Neil Armstrong. Putting a man on the moon is probably the single greatest technical achievement of humankind, and it is a matter of great pride to be working for a company like IBM that had a hand in that achievement. Mr. Armstrong's description of the IBM computer with the 4K of RAM, no mouse or keyboard, and a 7-digit readout for a monitor was also a pertinent reminder of just how far we've come in the last 40 years.
Yet, no matter how things change, some things remain the same, which Mr. Armstrong proved with a wonderful story that included a priceless punchline. He described a scientific achievement that was being sought at the time: to determine the distance to the moon with an accuracy of less than one foot. To do this, scientists would measure the time it took for a laser pulse to travel from Mt. Lick Observatory to the moon and back. Mr. Armstrong said he did not serve as a scientist for this mission but rather as a field technician: "My job," he said, "was to install the mirror." Nice. Everyone in the computer industry can relate to a site visit, but going to the moon and back for the customer is usually more a figure of speech in our profession. Anyway, he then proceeded to describe the difficulties that were had with the experiment because the pulses were not able to be detected by the observatory on their return. It turned out that the latitude and longitude for the Mt. Lick observatory, established in 1889, were incorrect! Seems that if you want to achieve something great, you have to know exactly where you stand.
It was with that message in mind that the Forms team set out to meet customers, business partners and net new interested parties. For a number of customers interested in what will be new in our upcoming 2.7 release, we had a great time dazzling them with the full range of things they didn't know were in the current release. This was understandable in many cases since the current release came out after Lotusphere 2006 and after they had adopted our products. But it really made the show a lot more exciting for us since we were able to show interesting things they could get their hands on today as well as how we're making those features even better in 2.7.
In terms of feature requests and responses, we got lots of results. People really seem to like the attention we have been paying to simplifying the forms design experience in the Forms Designer, including the reorganized eclipse perspective, the snap-to geometry guidelines, the click and drag of data subtrees to create grouped controls, the point-and-click compute wizard and the three wizards for creating repeat tables and row and column calculations over those tables. We got lots of requests to make multipage forms faster in designer, viewer and webform serveer, and we got lots of requests for digital signature functionality. Most notable there was the desire to have an interface for the viewer to allow business partners to redefine the signing ceremony, e.g. to comply with German signature statutes. We definitely got a lot of good feedback to take to the lab.
In terms of net new offerings, perhaps the biggest additions to our 2.7 forms product line are the Streaming Forms API and the Services Platform. The Streaming Forms API is a pure Java module that understands our forms language, including XForms, and allows highly streamlined functions for extracting the data from completed forms to drive back-end business processes as well as writing data into the form to accomplish functions such as pre-population or mid-population of forms at the start of or during a business process. The services platform is designed to radically simplify the server-side development of forms processing modules. Each logical step needed to process a form is called a pipe, and the processing of a form can be viewed as a pipeline. This allows you to organize tasks such as form template prepopulation, consumption of web services during execution of a form, and the complex set of tasks that often occur once a form is completed to archive the form, and shred data from it to kick off various business processes. The services platform will include the ability to leverage IBM Websphere Datastage TX as a pipe, so you can get access to any data source that TX can touch!
Over the next month or two, I'll be blogging about these additions and all the important pieces of the upcoming 2.7 release. Stay tuned...
John M. Boyer 060000VMNY 1,615 Views
XForms 1.1 is now a Last Call Working Draft!
The transition to "last call" is a declaration that the document is ready for general review by W3C working groups and the web community at large. It is also the final step in the working draft phase of the W3C Recommendation track. The general review period ends April 5, 2007. Afterward, the last call review comments are answered and the document typically proceeds to "Candidate Recommendation".
The spec now weighs in at a healthy 195 pages (well, the browser reports that it would take that many letter sized pages). You can get a more visual idea of the amount of work that has gone into XForms 1.1 by viewing the diff-marked version, which shows changes relative to XForms 1.0 Second Edition.
So, you ask, why would one want to use XForms 1.1? The answer is that you may already be using some aspect of 1.1 even in your XForms 1.0 forms today. The reality of standards work is that implementations lead the ratification of the full spec as implementers try the various designs to make sure they solve real customer problems. So the different implementations of XForms are each shipping with no small fraction of XForms 1.1 functionality pertinent to their respective customer bases. Often, the features are expressed as some kind of extension, but when the customer needs the feature now, then building it a way that the upcoming version of the language describes is just using good sense.
Some specific examples of XForms 1.1 functionality you may need right away are
There are lots of further additions and refinements to XForms 1.1, from more XPath functions to refinements of user interface controls, and I encourage you to see those diffs and find out what's coming.[Read More]
John M. Boyer 060000VMNY 1,476 Views
The upcoming release of Workplace Forms 2.7 is very exciting for our team due to performance improvements across the board as well as a wealth of improvements to the design time experience. However, the Workplace Forms product line has also benefited from some refinements to the XFDL and XForms layers, and that's what this blog entry is about.
The focal point for the XFDL presentation layer changes has been higher precision layout of tables. The form author can now control the amount of padding between table "rows" to the point where one can essentially create a grid appearance in which there are absolutely no unused pixels. Of course the use case is to be able not only to achieve zero unused space, but to have fine grain control that allows the form author to get down to zero but also to allow a little bit of breathing room for effects like row highlighting. Finally, XFDL now has the ability to create collapsible content such as "details" panes as well as collapsible table columns.
At the XForms level, more of the features from the XForms 1.1 last call working draft are now supported. The focus was on simplifying the handling of repeated data, which again is aligned with the focus on tables at the presentation level. The origin attribute is now supported on the insert action, which means that the prototype for repeated data can be copied from any instance of the XForms model. So, rather than copying an existing row of data then running a pile of setvalue actions to clear it out, the insert action can simply grab a new empty prototypical row of data, which means forms can be faster and take up less memory. The context attribute is also supported for insert and delete actions, which makes it easier to implement a table that is allowed to be empty. Finally, the while attribute is now supported for all XForms actions and sets of actions (the if attribute was already supported). This allows lots of data processing abilities for repeated data, including more complex results obtained from web services.
And last but not least we did make some small but meaningful additions to the language that were implemented across the product line but primarily motivated by the desire to make XFDL forms fit better into a corporate web site when the form is delivered via conversion by Web Form Server. This includes additional styling for XFDL button items to help them look and feel more like anchor links and offering the ability to hide the usual form toolbelt (which has open, save, and other form operation buttons on it). Hiding the toolbelt is useful when the form is intended to be in a portlet or as a component of a larger web application user interface.
I am proud of every release we do, and yet as each release nears, I just can't wait to get it out there into your hands because of how much better it will make your life developing forms applications. Once again, Workplace Forms 2.7 is going to be that kind of release.[Read More]
Well, I have always looked at Workplace Forms as an eco-friendly product line since, after all, it does dramatically cut down on the use of actual paper, which cuts down on the cutting down of actual trees. But, this post is actually quite a bit more technical than that. In this case, we're going to talk a little about the the *Workplace Forms Designer 2.7* (part of the 2.7 GA release this week) and the three wizards it includes to help form authors with creating dynamic tables and the components and formulae that make them dynamic. The focus, of course, will be the new wizard in 2.7.
The first *Table Wizard* helps you to create a repeating set of user interface controls over some data. You get a chance to set up nice presentational features like column headers, column dividers and row highlighting, and you also get Add and Delete buttons for the table. The second *Row Operation Wizard* allows the form author to express a formula over two columns of a table, with the result being placed into a third column. The formula is defined on the underlying data, of course, via an XForms calculate. The wizard is ideal for setting up that "line total = quantity * unit price" formula that shows up in every purchase order. It could also be used as the starting point for setting up the XForms constructs for a more complex calculation.
Although we've made improvements to these wizards in 2.7 (and will do so again in the future), the new kid on the block is the *Column Summation Wizard*, which is ideal for setting up a summation over a column of data, e.g. for taking the subtotal of a purchase order. The XPath expression written by this wizard is very interesting and relates to the title of this post. All you do as the form author is gesture at the user interface object in the column to be summed and at the user interface object where the result will go. Here is a sample of the underlying XPath calculate that results:
This formula requests the summation of all the line "total" values on each row of a purchase order form, and if the result is not a number (NaN) for some reason such as bogus user input, the formula produces an empty string result rather than the summation value.
This is helpful in and of itself because XPath, XML Schema and other W3C technologies thend to regard an empty string as not a number. So if you try to add or multiply a data node that has no value yet (like the quantity of items desired), then NaN starts to percolate through your form unless you take special measures.
But the really special sauce is the XPath predicate shown here:
Suppose for a moment that we didn't add this extra tidbit to ignore the empties. Suppose a user has a partially filled out purchase order form with three or four rows of data leading to a non-empty column total. If the user then adds one or more empty rows to add more data, then at that moment the column total formula would update itself to calculate the total over rows that included empties, which would result in a NaN, which the 'if' part of the expression converts to an empty string result. So the user would get treated to this odd experience of having the column total disappear each time he tries to add another row to the purchase order. The extra predicate created by the *Column Summation Wizard* says to ignore the empties so that the user will have a chance to fill in a row of data before it contributes to the column summation.[Read More]
The title of this blog has changed to begin "IBM Lotus Forms..." yet many of the prior entries on this blog refer to "IBM Workplace Forms...". This is because, as of the next major release, Workplace Forms is transitioning to "Lotus Forms".
To me, this transition positions the Forms product line squarely among the top-of-the-heap offerings by the Lotus division. Based on the overall technical direction announced at Lotusphere with Quickr, Connections, Expeditor, and Notes, as well as the clear place that Forms has in that technical direction, you can see that great things are coming for IBM Lotus Forms.[Read More]
John M. Boyer 060000VMNY 1,387 Views
As with any computing problem, the issue of performance arises. Here are a few tips that help increase the performance of forms regardless of the version of product being used.
The overarching theme is to know about and use the features of XForms when they are appropriate rather than using XFDL host language features to do the same things. The purpose of XFDL features is to add value to XForms as a host language, but sometimes those features are being used to do the same things as features in XForms. This is less efficient because XForms operates with XPath on the underlying raw data whereas XFDL operates on the formatted presentation layer. Specifically,
Basically, operating on the XFDL presentation layer is slower for two reasons. First, the presentation layer data has to pass through internationalization/localization filters. Second, the presentation layer often has more than one user interface control bound to the same data, so the logic would have to be repeated if expressed at the wrong layer. Overall, I'd have to say that the biggest performance enhancement from the above has been observed by people who used an XForms bind with a constraint rather than an XFDL format constraint. Generally, XFDL formats are expensive, and the XForms repeat has made it easier to create lots of them, so use XFDL formats sparingly within XForms repeats (i.e. within XFDL tables).[Read More]
The annual World Wide Web Conference is being held next week in Banff, Canada. It's about an hour's flight from Victoria, where I live, so of course I'm going!
I served on the program committee of the XML Web Data track this year. Competition to get into the conference is quite stiff, so recommending a paper for acceptance is a low probability event. Of the nine papers I reviewed, most were really good, and a couple that I had hoped would make it to the program will indeed appear. I am pleased to be chairing a session on Thursday May 10 that includes one such paper (I did not receive the other two for review).
I will also be presenting in the W3C Track on Friday at 1:30 as part of an Architectural Integration session organized and chaired by Steven Pemberton. I will be presenting on The W3C Rich Web Application Backplane, a forward-looking view of the possibilities for integration and composition of web applications leveraging W3C formats and APIs.
Finally, I'll be presenting in the Dev Track on Saturday at 1:30. My talk will demonstrate a schema-initiated drag-and-drop design experience for XForms using the Lotus Forms Designer.
Hope to see you there![Read More]
John M. Boyer 060000VMNY 1,802 Views
Well, finished off the WWW 2007 conference in Banff in fine form. There were quite a number of good papers in all. Due to my interests, I did of course enjoy those in the XML and Web Data track the most.
There was one interesting presentation on an IBM Research project led by Noah Mendelsohn called iScreamer, which was noteworthy because it essentially provided a method for doing schema validated XML parsing much faster than expat (which is considered quite fast and only checks for well-formedness). Of course, the performance comparisons were done based on a SAX model. The upside is that this filters out application-specific work, but the downside is that you don't get a clear idea of how much a DOM-amenable application might benefit from this work. Who knows, maybe there will be future work done on using iScreamer parsing to generate a DOM compatible with, say, Xerces. It would be nice, too, to understand why iScreamer parsers are faster than expat. Since the focus of the work seemed to be about speedy schema validated parsing, it is easy to believe that they might have nailed it and hence would be able to beat other schema validating parsers. But beating expat makes me wonder how to distinguish what was the actual performance improvement due to the novel schema validation method being reported versus what was due to simply having more efficient low-level parse routines (which could also be applied to other parsers like expat and xerces without adopting the novel schema method). Put another way, is there something at the lowest level of iScreamer that could be done to expat which would then make expat faster than iScreamer? Well, maybe some of the answers are in the paper or its predecessor (I was not able to read the paper as a referee because its authors are from IBM, so only now can I access the paper). Anyway, fast is fast, and that always gets my attention!
Speaking of which, there was a good paper on an efficient XML storage method: "Querying and Maintaining a Compact XML Storage". The paper is focused on being able to quickly perform various kinds of XPath twig queries on huge XML documents based on the creation of some kind of 'index' for the XML. The method used is so compact, that there is even room to leave some empty spaces, enough so that efficient time bounds can be achieved on certain operations. I liked the use of amortized analysis to argue that the space added was enough so that only occasionally did more space need to be added, lending to overall highly efficient performance characteristics. To me, the most important contribution was not the ability to do the queries efficiently, but to more efficiently update the index (more efficient in the amortized order of magnitude sense).
Honorable mentions (from me, anyway) would also go to "XML Design for Relational Storage" and "Visibly Pushdown Automata for Streaming XML". The first provided a normalization routine for XML that eliminated redundancy in the XML, as measured in the classic relational database sense. The second provided an interesting language theoretic model for describing various classes of streaming XML algorithms, which allows you to make certain types of statements and guarantees about the streaming algorithms. It's probably the most theoretical work in the bunch, but it gets you thinking in new ways, which is quite valuable.
Our own W3C Track presentation on the "Rich Web Application Backplane" went very well. I felt really good about how the whole architecture story played out, and it fit really well as the front end of the talks by Rafah Hosn (IBM) and Mark Birbeck (x-port.net) as well as connecting to material on SCXML that Rafah presented in the morning session. After the talk, I learned that the W3C will be publishing the track presentation slides on its website, so I'll let you know a link to that when I get it. For my own part, I took the time to annotate all of my slides so you will get much more of the talk that I actually gave than if you just got the slides alone.
Well, that's it for WWW 2007, and now I'm off to India to talk up XForms. More on that next time... stay tuned![Read More]
As promised, here are the links to my W3C Track talk at WWW2007 on the Rich Web Application Backplane (html, click here for the ppt file). Also, note that I heavily annotated my slides so you can read more about what I said than just the points on the slides. Let me know your thoughts on the content...
The W3C Forms working group held its quarterly face to face meeting last week in New York at IBM's T. J. Watson research center in Hawthorne. The bulk of the agenda was devoted to addressing XForms 1.1 last call issues. We had an extremely productive session, but there are over 150 comments to address, so we have only made it a little more than half way through.
One issue that arose at the face to face (thanks to a last call comment from Erik Bruchez) was a disconnect between was XForms does with single node user interface bindings and the setvalue action versus what schema validation does and what the type model item property does. The schema definitions and the type MIP validate the entire content of an element node, but the text for UI bindings or setvalue actions say that when they reference an element, the binding is to the first text node child of the element.
There were a few reasons why this was done in the past. One concern was about the confusion of binding to the whole referenced element, which you couldn't really do because the element may have attributes. However, at the time, no one really thought about it in terms of binding to the element content. Instead, the larger concern was that people wanted to be able to edit mixed content, which meant resolving down to individual text nodes so that interspersed comments and processing instructions would not be lost. However, one can still bind to individual text nodes by referring to them directly rather than referring to their parent element. This would preserve any sibling comments or PIs. So there are solutions to the key issues of interpreting an element reference as a reference to its full content, and here are some additional cases that caused problems:
All of the above are preposterous. Implementations aren't doing the first and third, and there is incompatibility on the second. I was therefore very pleased with today's Forms WG telecon, where the group resolved to change the wording of UI bindings and of the setvalue action so that when they reference an element with the single node binding (@ref or @bind), then they manipulate the entire content of the referenced element, not just its first text node.[Read More]
A question came past the other day about how to preserve the previous value of a form control before the one currently entered by the user. Immediately, the most famous story from Italian playwright Luigi Pirandello came to mind. Like six characters in search of an author, here we have a few XForms techniques in search of a blogger, and those techniques are getting their 5 minutes of fame now because the topic I intended to present has fallen victim to a limitation of the blogging system.
I'm sure there are other ways to do this, and comments are of course welcome, but here is what came to mind for me.
First of all, the originator of the question was attempting to use a special XFDL form control that can contain its own data, rather than binding the XFDL form control to the XForms model. This was being done because they did not believe the previous values could be captured in a way that uses XForms. However, having another control that can contain its own data is really no different than having a control that binds to a separate piece of data in the data model, so if it can be done with the special XFDL control, then it can be done with XForms too. Moreover, I would assert that it is better to solve this problem using XForms all the way because then you will have a better chance of creating something that even works when the controls in question are inside a
To be honest, once you buy into the fact that storing the previous value implies storing extra data, then you don't really need extra form controls to show the extra data to the user. You may need to show the previous values to the user, but it's trivial because you can just bind a control to the data node that stores the previous data.
So, now you get to learn about my Einsteinian philosophy for tackling programming problems. Remember the formula E=mc2? Well, mass is that which takes space, which is like memory, so mass is like variables. Energy is that which is expended over time, just like commands. So any time you're having difficulty figuring out what code to write to solve a problem, it's probably because you don't have enough variables. Put another way, often you can add just one variable and save yourself lots of code. This is what seems to be happening with the problem of preserving the previous value of any data node. So, suppose you store not just the previous value of a data node but also a copy of the current value. This problem gets a lot easier to solve. So let's take the following instance:
<xf:instance> <data> <node current="" previous="">VALUE</node> </data> </xf:instance>
So, the node has an initial value of "VALUE", and we want to be able to change the node content to, say, "NEW VALUE" while also preserving "VALUE" in the attribute named previous. So, easy stuff first is that we need to give the end-user the ability to change the node from "VALUE" to "NEW VALUE", and heck we might as well show the previous value to the user:
<xf:input ref="node"> <xf:label>Type a NEW VALUE in here</xf:label> </xf:input> <xf:output ref="node/@previous"> <xf:label>The old VALUE is here</xf:label> </xf:output>
Now the fun stuff. On form initialization, you would want to store the current value of the node because it will feed the previous value once the user actually makes some changes. You do this by capturing one of the XForms model initialization events, such as
<xf:model> <xf:instance> <data> <node current="" previous="">VALUE</node> </data> </xf:instance> <xf:setvalue ev:event="xforms-model-construct-done" ref="node/@current" value=".."/> </xf:model>
Last but not least, we now need to respond to changes the user makes by updating current and previous. This can be done by modifying the
<xf:input ref="node"> <xf:label>Type a NEW VALUE in here</xf:label> <xf:action ev:event="xforms-value-changed"> <xf:setvalue ref="@previous" value="../@current"/> <xf:setvalue ref="@current" value=".."/> </xf:action> </xf:input>
Every time the end-user enters new input, the change is sent to the data model and placed in the bound node. The XForms model then ensures that all calculations are run and then dispatches the event
That's really the end of the story, so I'm going for dinner. Hmm. Italian sounds good![Read More]