Information system architects, and even vertical industry standardization or government IT standardization bodies, derive immense value from XML schema definitions that describe the data structures and data types expected in valid transactions of the information system. However, these assets are focused on defining the completed transaction. In many a presentation, I've talked to potential customers about how IBM Forms documents express far more value because they are about the human interaction that takes place during the fill experience needed to produce those completed transactions.
As a bit of an aside, it's important for the technically minded reader to be familiar with the "sell" side of this equation. It is important to be able to easily justify technology adoption decisions with business owners who need to be able to understand how you will be able to better server your customers, reduce development and maintenance costs, increase competitiveness, eliminate vendor lock-in, etc.
In this blog, I'd like enumerate various benefits that you get from the standards basis of IBM Forms documents and their implementations, but I'd also like to separate the enumerations into two lists: 1) benefits above using XML Schema that you get just from using the XForms markup within an IBM Form, and 2) the benefits you get from the XFDL processor (XFDL is the XML vocabulary that IBM Forms provides as a presentation layer for XForms).
Firstly, an XForm is a clearly a superset of XML Schema since an XForm can incorporate the XML Schema if it is available and provide its validation information set to the fill experience and the submission experience. But an XForm provides many additional benefits, including the ability to:
Express data validation constraints that are based on other XML data values entered during the fill experience.
Automatically compute data values based on other input entered during the fill experience, rather than requiring users to perform the error-prone task of calculating and inputting summative results manually.
Describe the inputs, outputs and triggering mechanisms of web services to invoke, such as to use entered XML data to obtain database query results that fill other parts of the form or to invoke server-side validation or calculation logic from a business rules engine.
Control whether a data value is readonly or whether the user can enter a value, for example based on conditions related to a business process step or an access rule.
Express user interface controls that indicate which XML data values will be available to the presentation layer for input or output.
Conditionally show or hide the user interface controls (and hence their presentation) in response to conditions, such as those that may relate to a business process step or access rule.
Define a step-wise fill experience to guide a user systematically through incremental successes in entering valid data.
Provide customized help and validation error messages to help users fix data input errors when they occur.
Provide prompting label text to be associated with the presentation of each input or output control, both visually and aurally (for accessibility)
Associate a selection list with any XML data node in a way that constrains user input capability to the provided list
Define labels, help messages, validation error messages and selection lists in more than one human language within the same form, thereby ensuring citizens receive the same form logic and interaction behaviors regardless of which official language they select to request government services
Associate repeated data with a logically tabular set of user interface controls and encode the means by logical rows of the table are added or removed in response to insertions or deletions of data
While the above benefits indicate what additional behaviors and features of XML forms can be expressed above those that can be expressed by an XML Schema alone, XForms is also more interesting for standardization of a forms repository due to what it does not express. An XForm does not rigorously bind its many behavioral benefit to a specific presentation layer of the form. The intention of this language architecture was to address multimodal requirements of forms applications, e.g. rendition on a desktop, tablet, smartphone, telephone call-in voice service or instant messaging interaction. Different presentation layer implementations can address these requirements, and such implementations can even be provided by different vendors. The XForms working group also anticipated that there would be a wide array of varied technical requirements for presentation layers, and this language architecture allows XForms to be used with fundamentally different XML presentation languages that address these disparate requirements. Examples range from ODF for flowing text with fill-in-the-blanks fields to XFDL with its high-precision contract-style layout capability.
Due to the above mentioned language architecture, XForms markup does not comprise a well-formed XML document until it is incorporated into a presentation layer XML document. XFDL in the XML vocabulary used in IBM Forms to provide a presentation layer for XForms.
Of course, in a "baseline standard" version of a form, the default presentation layer associated with the XForms markup can be minimal in nature so that the only benefit is to provide a well-formed XML document to host the XForms markup. Interestingly, once you have this from XFDL, then the result is in fact an XML document, and so it can be processed by readily available XML processing tools like XSLT. These XML tools can be usd to automate creation of different versions of the baseline form that may have a richer presentation description, alternative natural language usage, or even different presentation layer markup. In addition, various consumers could use simple XML tooling to rebrand the forms.
The Extensible Forms Description Language (XFDL) in an XML vocabulary describing the presentation layer and richly interactive behaviors of modern web-based electronic forms. This XML vocabulary was first introduced to the W3C in 1998 (http://www.w3.org/TR/NOTE-XFDL
) and over the years, the versions of the language have consumed XML data-processing components of the W3C XML technology stack as they have been standardized by the W3C, especially including XML Schema, XForms, and XML Signatures. XFDL is a royalty-free open format whose current version specification, which can be obtained from -
XFDL is a host XML language for the XForms standard, and so the many benefits of XForms described above are inherited by XFDL. The implementations of XFDL as a presentation layer language add the following benefits over the core XForms processing:
very high precision control over the layout and rendition of the user interface
integration of XML Signatures with both XForms and a high precision user interface
comprehensive treatment for accessibility, localization and language support
dynamically computable presentation layer effects based on data value changes
multiple page support and ability to easily integrate web analytics within the form
ability to print or generate a PDF snapshot of the form or of just the agreement style pages
built-in ability to save the form locally and reload it later to continue the fill experience
a built-in file attachments folder system, including the ability of digital signatures to protect files attached into the form
a “public data” map for mapping XForms instance data to business process/workflow variables and case management properties
integrations to standard application server processing of form results and to run-time processing by JSR 168 and JSR 286 compliant portlets
the choice of zero-install operation within a web browser, using a server translator module
the choice of a client install to support both offline and online processing
In a larger sense, though, this only part of the benefit derived from XFDL. Still more benefit is derived from the availability of the XFDL forms visual design environment, which gives form authors integrated access to GUI features for XForms, for XML Signatures, for schema-driven design, for web sevice connections, and for the XFDL language benefits included in the list. The design environment even includes a converter that helps preserve the layout of PDF Forms that are brought over into XFDL. Finally, the XFDL forms visual design environment also provides features for maintaining a collection of forms, such as SVN and other team repository plugins and management of form parts.
Industry solutions is an important area of endeavour for IBM. An industry solution is an IT asset that helps solve an industry-specific problem and is easily reconfigurable to meet specific needs of each client. A solution often helps a client to reach out to and interact with their own customers or users.
An important segment within industry solutions is called case management, which takes the view that a customer/user interaction pattern can be orchestrated by a case. The definition of a case includes data structure and data type definitions, metadata definitions, business process and user access rules, and other possible resources. Based on an initial request by a customer or user, the case management system instantiates the case definition, and the resultant case orchestrates the interaction to achieve the goal or goals implicit in the defined pattern. For example, a case management system could be used to orchestrate the means by which a customer makes and successfully completes a warranty claim for a defective product. The process would begin with collecting initial information about the defective product, about the defect, and about the purchase. The process would include determining the legitimacy of the warranty claim, providing basic support to qualify the defect and determine a course of action, and ultimately to effect a repair or replacement of the product.
A case management user interface is a collection of interactive components that collect data from a user and store it in the data structure of a case (an instance of a case definition). The presentation of the user interface is also affected by the metadata of the case. A common piece of metadata is an enumeration of the valid values that a data item may take. If such a list is available, then it would be presented in a dropdown menu or list box, and the input would be collected via list selection rather than by free-form typing in a text entry field. Other common metadata are boolean flags such as for indicating whether a data node is readonly or required to fill in a step of the case processing. The user interface components would be affected by enforcing the readonly property or providing a sensory indication of the required property. Still other metadata can define validity constraints, such as a numeric datatype or a minimum or maximum value or length. The user interface of a component associated with a data node would be affected by indicating whether the current data value is or is not valid according to the constraints. A case management asset would also typically forbid progress to the next step of the orchestrated pattern when the data associated with the current step contains invalid values.
The term “case management solution” has been used to describe a software solution that supports the design, deployment, execution and reconfiguration of case management assets. As the field of case management matures, the term “advanced case management” has emerged as a way to characterize case management assets that have advanced feature requirements, such as the requirement to collect many dozens, scores or hundreds of fields of data. Some examples of advanced case management include: home or car insurance claims, credit card charge dispute resolution, citizen-facing ombudsperson cases, contagious disease outbreak tracking, and management of complex medical or psychological treatment cases. IBM Case Manager (ICM) is IBM's advanced case management solutions.
A problem arises in advanced case management solutions with respect to the expected maturity of the user interface. The typical case management solution generates a user interface presentation layer for the data of a step using a simple linear columnar approach or a column of expandable stacks of related data values. The advantage of this approach is that it most easily adapts to a reconfiguration of the case management asset in which the data structure is amended to add or remove data nodes. However, there are a number of disadvantages to this approach. For one, it provides a one-size-fits-all approach to the user interface layout in which usability substantially degrades in quality as the size of the data set grows.
As well, larger data sets tend to correlate to more advanced requirements in an overall solution that a case management asset simply cannot begin to address using only a simple user interface approach. There are many such features, including creating multipage guided interview style wizards, creating mutiple print-style pages to reflect a “document of record” for the case, and of course adding the ability to digitally sign the “document of record” as a way to create a legally binding agreement or a record that can stand up to rigorous auditability requirements.
The new release of IBM Forms is the strategic IBM forms technology that now solves this problem for IBM Case Manager (ICM). An IBM Form combines an XML data structure with a template describing interaction behavior rules and a comprehensive user interface definition. The how-to for connecting an IBM Form to an ICM solution is as simple as going into the IBM Forms Designer, right-clicking on any number of XML data nodes, making them “public” and giving them public names equal to the ICM case property names they must map to, and then ataching the IBM Form into the ICM solution.
The IBM Form can then be used in the Case eForm widget anywhere in the ICM solution where the Case Data Widget would have been used. During execution of a case under the ICM solution, the case property values are automatically injected into the XML data of the IBM Form as it is rendered to the user, based on the public data mapping mentioned above. When the user completes or saves the form, the updated data values from the IBM Form are injected back to case properties so that the Form and the Case are in synch.
Oh, but the story is so much cooler than just adding high precision, multipage user interface control for the data. And it's cooler than having a “document of record” for the data that can be digitally signed. IBM Forms contains this fabulous technology for defining live interaction with XML. It's called... you guessed it... XForms. XForms manages not just XML data, but also metadata pertaining to each data node, such as the node's datatype, or whether the node is required, or whether it must be valid according to some constraint expression. XForms also allows a list of values to be associated with the input mechanism for a data node. Remember above where I said that case management assets define metadata just like this for case properties? Well, when you design a case solution that includes this metadata, and then map a XML node to a case property by assigning the public data name, the IBM Forms integration automatically injects not just the case property values, but also any lists as well as XForms binds for the metadata. During the form run-time, the XForms processor then automatically combines these XForms bind results with any metadata settings that might be defined within the form itself. In effect, there is a seamless bridge from ICM case processing to the user's interaction with the IBM Form.
Finally, this seamless bridge works in the other direction too, from the IBM Form back to the ICM case solution. In addition to synchronizing data updates from the form back to the case, the key lifecycle operations of saving or completing a form interaction are gated by a validation operation. IBM Case Manager delegates the validation operation to the IBM Form technology, which executes a validation operation based on the behavior of an XForms submission. This means that non-relevant nodes are automatically pruned from the validation, and the validation result is the sensible combination validation rules injected from the case solution and validation rules expressed directly in the form.
Netting it out, IBM Forms is a first-class citizen of IBM Case Manager solutions, and the case data and metadata of an IBM Case Manager solution are handled as first-class citizens of the user experience provided by an IBM Form.
Recently, I was experimenting with one of the features planned for the next version of XForms. The feature is the iterate attribute for XForms actions, which will perform a for-each loop operation based on a nodeset obtained from the xpath expression in the iterate attribute value. XForms 1.1 already has a while loop, but iterate makes many data processing loops easier to write and also more performant (subject of a future blog). There were lots of iteration use cases to choose from, but I decided to experiment with sorting because it is a well-known benchmark algorithm.
Before we go any further, let me say up front that XForms action scripting is intended for very lightweight data manipulation, like adding or deleting a data node corresponding to a table row or copying data results to or from the SOAP envelopes of a web service. By the time you get to nested iterations like those needed sorting, you should be considering alternatives expressed in full-blown imperative languages available in the information system within which the form is being used. For example, in the case of sorting, it is a better idea to request sorting in the database query whose results are returned from a web service into your form so that your form logic does not even have to do the sorting.
So, with the disclaimer out of the way, let's abuse the technology a bit to get a better sense of what is feasible in those customer-needs-it-yesterday circumstances. It turns out that XForms 1.1 does allow full nodeset processing in the insert action's origin attribute and the delete action's nodeset attribute. Without even needing the new iterate attribute, this is just enough iteration capability to perform efficient sorting -- so there are some kinds of iterations that can be done now without the iterate attribute.
We're going to do a divide-and-conquer "partition" sort that I personally created as a university freshman after my 1st semester instructor told our class that linked lists could only be sorted slowly. At the time, the usual computer languages only allowed static allocation for arrays, and even though I didn't know what a "quick sort" was, I had seen the light of dynamic allocation, and I was never going back! I later learned how great a merge sort is on a linked list, but the effort of turning an array quicksort into a linked list partition sort comes in handy now because a merge sort cannot be efficiently expressed in XForms until the iterate attribute is added.
The way a quicksort works on an array (or subarray) is that you pick a random element to be the 'pivot' value. Then you run two index variables at the same time, one from the start of the array upward and the other going from the end of the array downward. The 'up' index is advanced until it finds a value greater than the pivot, the 'down' index is decremented until a value less than the pivot is found, and then the values at the 'up' and 'down' locations are swapped. This keeps happening until 'up' and 'down' meet somewhere near the middle of the array. At this point you've partitioned the array into a subarray of values less than the pivot value and a subarray of values greater than the pivot value. The quicksort is then invoked recursively to sort both subarrays.
The main challenge with this approach is the 'down' index, which is a reverse iteration. In a singly linked list, you can only go forward. XForms insert and delete actions have a similar limitation: they can only identify a nodeset of nodes to insert or delete, but not really a direction of iteration. But the important bit is what the quicksort is doing, not how it is done. Think of the list content as being completely messy, and each partitioning stage must make it somewhat less messy by dividing the content into a partition of lesser elements and a partition of greater elements. Then, the next partitioning stage is invoked recursively to do a better job of cleaning up the mess within each partition.
Let's explore this concept by sorting a list of elements, such as sorting a list of <person> elements by a <lastname> child element. We begin by copying the list into an initial partition element of a temporary instance called 'sortdata', like this:
1) <xforms:insert context="instance('sortdata')" origin="instance('partition')"/>
2) <xforms:insert context="instance('sortdata')/partition" origin="instance('data')/list/person"/>
3) <xforms:delete nodeset="instance('data')/list/person"/>
Next, we initialize the random number generator so we can randomly select pivot values for all the partitioning stages:
4) <xforms:setvalue ref="instance('sortpivot')" value="random(true)"/>
Next, we start up a simple while loop that continues to process partitions until none are left.
5) <xforms:action while="instance('sortdata')/partition">
Within the loop, we grab the last partition from the sort data and determine whether it is non-trivial or trivial (only 1 or 2 elements). A non-trivial partition is subjected to further divide-and-conquer processing.
5.1) <xforms:action if="count(instance('sortdata')/partition[last()]/*) > 2">
The first part of dividing and conquering is to create a new empty <partition> element, which we obtain from an instance that expresses a template empty partition element.
5.1.1) <xforms:insert nodeset="instance('sortdata')/partition[last()]"
Step 5 and step 5.1.1 are more interesting than they seem at first. The list of partition elements in the sortdata actually implements the recursion stack, and we just pushed a new element into that stack at the second-to-last position. Because we have an explicit stack, we only need a loop in step 5 to implement recursion.
The next thing we do here is grab a random last name to serve as a pivot value for the partitioning. The first setvalue just picks a random location, and then the second step uses the location to get the value. Notice also that I use * rather than partition before the [last()] predicate because the sort data only contains partition elements, so there is no point in doing a name test for partition.
5.1.2) <xforms:setvalue ref="instance('sortpivot')"
value="1+floor(random(false) * count(instance('sortdata')/*[last()]/*))"/>
Now the magical part happens. All elements in the last partition whose key element (lastname) is less than or equal to the pivot value are moved to the newly created second-to-last partition. By combining the nodeset processing capability of XForms insert and delete actions with the predicate-based node selection capability of XPath, the matching nodes can be selected and moved using two single XForms actions, i.e. without using an XForms while loop.
5.1.3) <xforms:insert context="instance('sortdata')/*[last()-1]"
origin="instance('sortdata')/*[last()]/*[compare(lastname, instance('sortpivot')) <= 0]"/>
<xforms:delete nodeset="instance('sortdata')/*[last()]/*[compare(lastname, instance('sortpivot')) <= 0]"/>
If the last partition is now empty due to the move operation in step 5.1.3, then the new second-to-last partition received all its elements. If all the moved elements are equal to the pivot value, then we can output them back into the original data list and then remove the last two partitions. Note that the insert is configured to prepend the elements into the data list, and we're copying them from the last non-empty partition, which has the elements with the greatest key value.
5.1.4) <xforms:action if="count(instance('sortdata')/*[last()]/*) = 0">
<xforms:action if="not(instance('sortdata')/*[last()-1]/*[compare(lastname, instance('sortpivot')) != 0])">
<xforms:insert context="instance('data')/list" origin="instance('sortdata')/*[last()-1]/*"/>
<xforms:delete nodeset="instance('sortdata')/*[position() >= last()-1]"/>
</xforms:action> <!-- End of 5.1 non-trivial partition handler -->
Now, we've finished with the non-trivial partition handler, and we turn our attention to processing a trivial partition containing at most 2 elements. The content of the partition is moved to the original data list and the partition is removed. Again, note that we're processing the last partition, which has the greatest key values, and the insert prepends to the data list, so the sorted data list starts with the greatest values and grows as lesser and lesser values are prepended over time as all the partitions are processed.
5.2) <xforms:action if="count(instance('sortdata')/*[last()]/*) <= 2">
If the partition contains two elements and the greater one is first, then it is removed from the partition and put in the original list:
5.2.1) <xforms:action if="count(instance('sortdata')/*[last()]/*) = 2 and
instance('sortdata')/*[last()]/*/lastname) > 0">
<xforms:insert context="instance('data')/list" origin="instance('sortdata')/*[last()]/*"/>
Now the partition contains either no elements, one element, or two elements in sorted order, so we move the content to the data list and then delete the partition.
5.2.2) <xforms:insert if="count(instance('sortdata')/*[last()]/*) > 0"
</xforms:action> <!-- End of step 5.2 trivial partition handler -->
</xforms:action> <!-- End of step 5 recursion while loop -->
As a final note on all this algorithmic fun, the question arises whether this sort achieves optimal O(N log N) performance. The answer is no, not quite, due to hidden costs of data instance management and data node selection. However, the sort will be much faster than a "simple" sort because it does perform only O(N) XForms actions.
Gaining customer insight has very high business value. In the last blog, I showed you how you can gain customer insight without introducing bias by randomizing the answers to each question you ask. In a prior blog, I showed you how you can make the presentation layer be dynamically responsive to the type of question you are asking. In this blog, we'll cover how to randomly select the subset of questions to be asked of a particular user. This is important in avoiding survey fatigue and low response rates that can result from surveys that take too long to complete.
As in the prior blog entry that presented survey items of different types, suppose you have an XForms instance that includes any number of items of interest, where each <item> includes a prompting question, a type of answer, storage space for an answer, and so forth. The goal is to have the form template initialize itself with a small subset of these <item> elements, and once again to do so randomly, i.e. without bias, so that over a large customer population, we will get a close-to-equal response rate on all items of interest.
Let the following two XForms instances exist:
<xforms:instance id="AllItems" xmlns="">
<!-- All survey item elements -->
<xforms:instance id="ChosenItems" xmlns="">
<items random="" i="" number="3">
<!-- Initially empty, will contain "number" item elements chosen from AllItems -->
Data initialization in XForms can be done in the xforms-ready event but is more typically done in the xforms-model-construct-done event because it is more efficient to manipulate the data before the user interface is created. So, you'd randomly choose some survey items from all the items like this:
<xforms:action if="count(instance('ChosenItems')/item) = 0"
<xforms:setvalue ref="instance('ChosenItems')/@random" value="random(true())"/>
<xforms:action while="instance('ChosenItems')/@i <= instance('ChosenItems')/@number">
<xforms:setvalue ref="instance('ChosenItems')/@random" value="random(false())"/>
The if statement on the main action ensures that the random selection only happens once, when a copy of the form template is used for the first time, thereby becoming a form. The first setvalue action seeds the random number generator, and the second action initializes the loop variable. The main loop randomly chooses "number" items from AllItems and places them in the ChosenItems. Each selected item is then deleted from AllItems to avoid duplicate items.
The "number" of questionnaire items to select is shown to be 3 in the example, but it is really up to the designer(s) of the survey to set that number into the form template based on how many questions they feel their customers will answer without impeding the return rate due to survey fatigue.
Finally, rather than binding a user interface to the 'AllItems' data instance, you simply bind it to the 'ChosenItems instance instead. An XForms repeat iterates the user inteface controls within it for each of a set of nodes identified by the XPath in its nodeset attribute, like this:
<!-- XForms user interface controls, like input and select1, go here -->
Inside the repeat is where you could use simple XForms input controls for typed answers or select1 controls for choice-based answers, or this is where you could use dynamic user interface techniques to show different kinds of user interface controls based on a type of survey item.
I learned something interesting about survey research recently. Given a survey question with a choice list answer, such as a list of icecream flavors or political parties, apparently there is a non-trivial segment of the population who read only the first few possible choices before rendering an answer. This segment can introduce significant bias into survey results unless the list of choices is randomized for each survey.
My recommended approach is to use web application code to put the choice list in the desired order and then push the list into the form before sending the form to an end-user for interaction. However, we increasingly see that people want the server side web application code to be as simple as "send form to user, and when it comes back, extract the data". As a result, front-end form logic is increasingly used in lieu of writing middle tier Java code.
So, here's a sample XForms single choice selection control that, due to the appearance attribute, would be presented as a group of radio buttons:
<xforms:select1 appearance="full" ref="/survey/icecream">
<xforms:label>Pick your favorite flavor of icecream:</xforms:label>
The set of choice items come from XForms instance data at the location given by the XPath location in the nodeset attribute of the itemset element. The label element then indicates what is shown to the user, and the value element indicates what internal data value corresponds to that display label.
<xforms:instance id="Flavors" xmlns="">
<choice value="CC">Cotton candy</choice>
Now, since we don't want to bias the results toward butterscotch and chocolate flavors, we want to randomize this list each time the question is presented to an end-user. The best solution would be if XForms had additional appearance keywords as custom namespaced extensions, something like this:
<xforms:select1 appearance="full xfdl:randomized" ref="/survey/icecream">
In current XForms, the appearance attribute only allows one keyword, and only the words minimal, compact and full are defined. The XForms working group agreed today to change the appearance attribute to a space-separated list to allow extension features like this ("xfdl:sorted" would be nice too, though we also recommend sorting a list before putting it in the form).
Supposing you do want to randomize a list of choices within an XForm, let's start by focusing on just creating a permutation when a form starts. Start with a list of any number of data values, such as 5 values, like this:
<xforms:instance id="Permutation" xmlns="">
<permutation count="" position="" random="" size="" swapvalue="">
Now, on form load, we'll create a permutation of them, like this:
<xforms:action id="GenPermutation" ev:event="xforms-model-construct-done">
<xforms:setvalue ref="instance('Permutation')/@random" value="random(true())"/>
<xforms:action while="instance('Permutation')/@count < instance('Permutation')/@size">
value="floor(random(false()) * (../@size - ../@count + 1))"/>
<xforms:setvalue ref="instance('Permutation')/@position" value="../@random + ../@count"/>
<xforms:action if="instance('Permutation')/@count != instance('Permutation')/@position">
<xforms:setvalue ref="instance('Permutation')/@count" value=". + 1"/>
The first line hooks the start of form processing, after the data model has been initialized. Next, we seed the random number generator and a couple of variables used in the main loop. The loop iterates through positions 1 to n-1 of an n-length permutation. For each position k, the value is swapped with a randomly determined position between k and n, inclusive.
Because XPath automatically concatenates the text nodes of a given XML tree or subtree, you can look at the whole permutation using a single XForms output, like this:
More importantly, you can use the permutation as a lens on the set of items shown by a selection control, such as an XForms select1, like this:
<xforms:select1 appearance="full" ref="/survey/icecream">
<xforms:label>Pick Favorite (random lens)</xforms:label>
Compared to the first code sample above, the only differences are the ref attribute expressions in the label and value elements, but they both do something interesting with the nodes produced by the itemset element. Typically, the itemset is used as a "for each" construct, and the label and value ref expressions drill *into* each node to obtain the label for an item and the data value corresponding to choosing that item. In this example, the itemset is really only used to obtain the correct number of items to generate and to set an initial context node within the set of items. The label and value ref expressions then pop up to the parent of that context node, and then drill down into a different choice element based on the order given in the permutation.
Below is a screen snap of the two select1 controls (and the XForms output for the permutation). On the left, you can see the radio button group with the items appearing in the original order of the data. At the bottom you can see the permutation that was generated when the form was rendered. Every time the form runs, a different order is selected, but this time, the order generated was 3, 1, 5, 2, 4. So, you can see on the left that "Cotton candy" is the 3rd choice, and so it appears first in the permuted choice list on the right. Similarly, "Butterscotch" is the first choice in the data, so it appears second in the permuted list on the right, followed by Vanilla, Chocolate and then Strawberry is last in the permuted list since it is 4th in the original list.
Finally, note that you only have to generate the one permutation of a particular size, which can be used as a lens on all selection controls that have the same number of elements as the permutation. If you have a survey in which different questions have different numbers of answers, then you'll need to generate a permutation for each different number of answers. However, once you have all the permutations you need, you can dynamically pick the permutation to use in the label and value ref expressions based on using the last() function to obtain the size of the itemset's node set.
A number of times recently, I've been asked how to create a pair of list boxes with arrow buttons between them that move the selected item from one list to the other in the direction of the arrow (if something is selected in the source list).
As I did, you can easily use the IBM Forms Designer to create a simple form that has the two list boxes for the two lists (see here
if you want to get a copy of my sample form). I used icecream flavors as the list items to be moved around between a list of available flavors and a list of chosen flavors. The Designer automatically generates the XForms markup, such as the xforms:select1 elements and the instance data to store the results of selections and the lists to select from. Here is the XForms markup from the form I created with a few drag and drop operations:
The data instance where the user selections for each list are stored:
<xforms:instance id="Generated" designer:type="generated" xmlns="">
Data instances where the actual choices are stored (the available flavors and the chosen flavors):
<xforms:instance id="FlavorsAvailable" xmlns="">
<xforms:instance id="FlavorsChosen" xmlns="">
Normal XForms-level user interface markup for showing the two lists above and for storing the choice made in each list:
<xforms:select1 appearance="compact" ref="instance('Generated')/page1/list1">
<xforms:select1 appearance="compact" ref="instance('Generated')/page1/list2">
Now we get to the dynamic part. I added two buttons to the form, one labelled >> and the other labelled <<. Each contained an xforms:trigger whose DOMActivate action performed an insert/delete/setvalue sequence to copy a selected choice from one list to the other. Given the above markup, here's what that looks like:
A trigger to copy from the available flavors (positioned on the left) to the chosen flavors (positioned on the right):
<xforms:action if="instance('Generated')/page1/list1 != ''" ev:event="DOMActivate">
<xforms:insert context="instance('FlavorsChosen')" nodeset="choice" origin="instance('FlavorsAvailable')/choice[@value = instance('Generated')/page1/list1]"></xforms:insert>
<xforms:delete at="" nodeset="instance('FlavorsAvailable')/choice[@value = instance('Generated')/page1/list1]"></xforms:delete>
A trigger to copy from the chosen flavors back to the available flavors:
<xforms:action if="instance('Generated')/page1/list2 != ''" ev:event="DOMActivate">
<xforms:insert context="instance('FlavorsAvailable')" nodeset="choice" origin="instance('FlavorsChosen')/choice[@value = instance('Generated')/page1/list2]"></xforms:insert>
<xforms:delete at="" nodeset="instance('FlavorsChosen')/choice[@value = instance('Generated')/page1/list2]"></xforms:delete>
Each trigger's action uses 'if' to test whether the source list has a selected choice. If so, then we do an insert first to copy the selected flavor's choice element to the target list. Then, we delete the choice from the source list. Lastly, we erase the selection from the data value of the select1 so that it does not report an out of range error relative to the its updated list, from which we just deleted the item that was selected. That's really all there is to it.
As an alternative to the method above, you could maintain a single list of choice elements, and have an extra boolean attribute on each choice element in the data. Call it selected. It would look more like this:
<xforms:instance id="Flavors" xmlns="">
<choice value="v" selected="false">Vanilla</choice>
<choice value="c" selected="false">Chocolate</choice>
<choice value="s" selected="false">Strawberry</choice>
<choice value="b" selected="true">Butterscotch</choice>
Then, the itemsets of the two select1 lists could use predicates, like this:
<xforms:select1 appearance="compact" ref="instance('Generated')/page1/list1">
<xforms:select1 appearance="compact" ref="instance('Generated')/page1/list2">
The advantage of this method is that you can then avoid using inserts and deletes, which cause rebuilds. Instead, you just change the selected attribute to true or false (and then also set the value of the source list to empty again):
<xforms:action if="instance('Generated')/page1/list1 != ''" ev:event="DOMActivate">
<xforms:setvalue ref="instance('Flavors')/choice[@value = instance('Generated')/page1/list1]/@selected">true</xforms:setvalue>
<xforms:action if="instance('Generated')/page1/list2 != ''" ev:event="DOMActivate">
<xforms:setvalue ref="instance('Flavors')/choice[@value = instance('Generated')/page1/list2]/@selected">false</xforms:setvalue>
I've recently realized that there are a few milestones to celebrate all at once here: This blog turned 5, my W3C XForms group age is 10, and this is the 100th exciting episode of the IBM Forms dW blog. I took a look back at what you, the readers, seem to like best, and those 2000-3000 hit entries are at the intersection of strong technical content and the confluence of open standards. How serendipitous!
The power and value of the XFDL markup language underlying IBM Forms comes from the way it brings together all the features of open standards needed to design sophisticated solutions, including XML, XML Schema, XForms, XML Signatures, and XPath. In this entry, I'd like to continue the schema driven design story by talking to you about the information architect's half of the equation. Specifically, we'll cover the XML Schema constructs that the IBM Forms Designer converts into various user interface controls like XFDL popups skinning XForms select1 elements and XFDL tables skinning XForms repeat elements. If you haven't yet watched the short demo videos mentioned in the prior blog entry, take a moment to do so now because I'd like for us to dig into parts of the "Medical Preapproval" schema used in that demo. The top-level element definitions for the schema look like this:
<element name="medicalPreapproval" type="health:medicalPreapprovalType"/>
<element name="patient" type="health:patientType"/>
<element name="contactInfo" type="health:contactInfoType">
<element name="details" type="health:detailsType"/>
In this sample, the data needed fora medical insurance preapproval consists of some basic patient data like name and date of birth, contact information like email and address, and details of the medical procedures required. When the IBM Forms Designer is provided with this schema, it can generate an XML data conformant to the schema in an XForms instance element, and then the form author can drag and drop the XML elements in the instance view onto the Design canvas. For example, if the form author drags and drops the "patient" element, then the IBM Forms Designer creates an XFDL pane that skins an XForms group element. Within the group, various XFDL items like text input fields and calendar pickers are created and mapped to the child elements of the "patient" element so that the textual content can be collected by the form. So you can get an idea of how to write XML schema for structured content, here's a part of the patientType definition:
<element name="lastName" type="xsd:string"/>
<element name="firstName" type="xsd:string"/>
<element name="SSN" type="health:serialNumberType"/>
<element name="dateOfBirth" type="xsd:date"/>
At design time, text entry fields are created for these data items, and any specialized schema rules, such as the regex pattern definition for the SSN, are applied at XForms run-time. In case you're curious, here's how you'd set up a regex pattern, in this case a very simple one that takes 9 digits but also allows an empty string:
For data types like the xsd:date (above) and xsd:boolean, you will get an alternative form control, such as calendar picker or checkbox. The IBM Forms Designer will even generate a calendar picker if you have a type derived from xsd:date. For example, you may want your schema to allow a date or an empty value. So, if you change the type of the dateOfBirth element above to health:dateOrEmpty, then you can use the declaration below to achieve that effect:
<element name="dateOfBirth" type="health:dateOrEmpty"/>
By default, the label text generated for each element is based on the element name. Camel-casing is used to determine when spaces should be added to the label. For example, the default label generated for the "patient" group is "Patient", and the default label for the "lastName" element is "Last Name". However, the information architect is provided greater control over the labels in the schema. In the example above, an appinfo annotation is used to define the desired label of "Contact Information:" for the contactInfo element. The same appinfo annotation can be used to control the labels of the UI controls bound to leaf (text content) nodes. For example, below is a definition that could go into the patientType definition, but it provides an alternative label for an element that has a name that is perhaps medically accurate but perhaps less desirable:
<element name="sex" type="health:genders">
Continuing with this example, the obvious choices of Male and Female would be defined in a schema enumeration, like this:
An enumeration like this in the schema causes an XFDL "popup" item (a dropdown menu) to be generated, along with an xforms:select1 element. The dropdown menu entries are Male and Female, and the user's choice at run-time places the word Male or Female in the "sex" element. But maybe you'd prefer to have the data be based on codes like M and F, but still show menu entry labels like Male and Female. Again, the appinfo annotation comes to the rescue:
I've used Gender in these examples for brevity, but you can use enumerations to describe other lists, such as for States and Provinces. The IBM Forms Designer will generate an XFDL popup item by default, but the Designer also lets you right-click convert the popup to other user interface items that can legally skin an xforms:select1, such as a radiogroup, checkgroup or list box.
Also, sometimes what you need is a combobox, so that the user can access a dropdown menu but also have the option to type an open-ended answer. For example, the open-ended response for a gender might provide medically relevant information. The information architect can control this in the XML schema by creating an open-ended enumeration, i.e. an enumeration unioned with a string:
Last but not least, we have the mechanism that signifies the need to create an XFDL table and xforms:repeat. Here's an excerpt of the health:detailsType definition:
<element name="reason" type="xsd:string"/>
<element name="recommendedProcedures" type="health:recommendedProceduresType"/>
<element name="procedure" minOccurs="0" maxOccurs="unbounded">
<element name="name" type="xsd:string"/>
<element name="qty" type="health:positiveIntegerOrEmpty">
<element name="price" type="health:decimalOrEmpty"/>
<element name="lineValue" type="health:decimalOrEmpty"/>
<element name="total" type="health:decimalOrEmpty"/>
Within the recommendedProcedures element, the schema indicates that a "procedure" element can appear more than once (maxOccurs is unbounded in this example, but any value greater than 1 will also work). The IBM Forms Designer responds with the XFDL table editor. It reads the child elements in the sequence to determine the default set of table columns. Notice the use of the appinfo annotation to control the column header label text. When the form author drags and drops the details element, XFDL fields sking xforms:input elements are created for elements like reason and total, but the table editor also produces an XFDL table and xforms:repeat whose nodeset attribute contains the XPath needed to bind to however many procedure elements are present in the data at form run-time. The table editor also generates XFDL buttons containing xforms:trigger elements that provide the "add row" and "delete row" capabilities using xforms:insert, xforms:delete and xforms:setfocus actions.
Now hopefully you have a more complete picture of how IBM Forms concentrates the combined value of all these standards-- XML schema, XPath, XForms, XML-- into one solution creation machine. Well, that may have been a tad long, but we're celebrating three milestones here :-) Besides, the length of this entry is a simple reflection of how much cool stuff you can now exploit in your XML Schemas to help your form authors create the forms that feed the right XML to your backend systems.
Engaging with customers (or employees) involves collecting information from them. To sell an insurance product, for example, you need to know what kind of insurance, how much, who's it for, their age, and a whole raft of other information. This is how we end up being able to provide a customer with an insurance quote or product, a financial preapproval, a medical appointment, and this is how we process IT requests, support requests, purchase orders, expense reports, etc.
If you're architecting a web-available XML-based information system, you will typically define the XML data required by the system. Specifically, what information does the organization need in order to process a transaction with a customer or an employee? A result of the design work will be an XML schema that describes the overall data structure and the data types for the back-end services that process the transactions. Then, a form is created to provide a web-based data collection interface for the data. Finally, there is the possibility of java servlet code in the middle to convert the data values collected by the form into the XML data structure required by your back-end services. Unless...
With IBM Forms 4.0 Designer, the XML schema describing the data structure can be consumed directly into the form design experience without the form author needing to understand anything about XML schemas. The "form author" skillset includes high precision GUI work to create either a UI that conforms to some regulations or a UI that captures and keeps the user's attention. The form author isn't and shouldn't be worried about the data structures. They want to drag and drop stuff onto a design canvas, and so much the better if the design environment makes sure that the GUI items are automatically mapped to the data structures needed on the back end to process the data collected by the form.
Here are two videos that give you an idea of what this design experience looks like: Schema-driven Forms Design
and Schema-driven Forms Design and Dynamic Tables
The form author has been provided with the data schema, and they just pull it into the design experience and ask for the data structure to be created based on the schema definition. Then, they drag and drop the GUI representations of the data elements to automatically create groups of user interface controls that are mapped onto those XML data elements. This leaves the form author free to focus on issues of colors, borders, GUI layout, and so on because the form automatically collects the data in the XML format defined by the XML schema
. The form author needs no knowledge of XML schema, nor XML markup for that matter
, to build up the form user interface.
The drag-and-drop operation is responsive to data types, so it creates different UI controls for various data types, such as calendar pickers for dates, check boxes for booleans, dropdown menus for closed enumerations, and comboboxes for open enumerations. The operation is also responsive to XForms label annotations on element and attribute definitions as well as enumeration items, so rather than using XML element and attribute names as the default values placed into the user interface, the data architect can assign friendly prompts or labels for element and attribute data to be collected as well as enumeration items appearing in lists.
The drag-and-drop operation is also responsive to structural definitions. If the form author drags and drops a piece of data for "Personal Details", then UI controls will be created for single child elements like "Name" and "Date of Birth" but whole subgroups of controls will be created for subtree elements like "Address", which may be comprised of child elements like "Street" and "City". Moreover, if an element being dropped includes a repeating subtree element, i.e. one bearing a schema maxOccurs value greater than 1, then a dynamic table element is created, including the insert and delete buttons that allow a user to insert or delete rows from the table.
So, have a look at the special demo videos above to see the work in action, then plug "IBM Forms" into your search engine, go to our site and download the Designer and Viewer trial software so you can give it a test drive, and stay tuned to this blog because in my next post I'll take you through a sample schema and show you exactly what bits of schema trigger the IBM Forms Designer to do something delightfully automatic for you.
One of the New IBM Forms 4.0 Demonstrations
has just won the top-rated video award at Lotusphere
. This is a fantastic victory for the Websphere Portal segment of Lotus, which focuses on products like IBM Forms that create and provide exceptional web experiences. It is also a victory for the W3C XForms standard
since the Wizard Creator
featured in the video is, as far as I know, the first point-and-click design experience for an XForms switch. This feature exemplifies the principle that declarative markup languages result in more powerful application design environments because they express what the author wants. With an imperative languages, a design environment must either operate at a much lower level (less powerful) or do some wicked reverse engineering/pattern matching to discern what the author wanted from how he did it.
So, XForms has definitely been my friend in helping to create an award-winning feature in IBM Forms. Wanna meet my newest friend? Here we are, just me and Watson, celebrating our victories at the closing session of Lotusphere 2011:
To put it in Watson's terms:
Category: Lotusphere Best Demo Video Winners
Answer: An IBM Form that uses XForms to express Wizard interfaces for forms.
Question: What is a Smarter Web Application?
MC: Congratulations Watson, you're ready for prime-time! Wanna join my social network?
The hit rate on my last blog announcement about IBM Forms 4.0 makes it clear that the release is pretty big news. I'm very pleased about that because it is a *major* release, both technically and in terms of the corporate branding. Corporate branding is not a decision lightly made, so it took the time it took, and meanwhile we didn't want to delay the technical features hitting the market, so you'll see some references to "Lotus Forms" under the covers of the main IBM branding. Not to fear, we're working on that for the next release.
Meanwhile, now that IBM Forms 4.0 is released, I've made a number of demonstration videos to show you key aspects of the IBM Forms Design and Run-time products. They're available on YouTube in HD, and they're broken into small segments, so you can really take them in pieces according to your interests.
To start, here's a 5-part series that takes you through run-time features with a demo form that ships with the product. These videos help you understand the value proposition of IBM Forms as well as new features of the release that will change your mind about what forms are and what they do.
- The New IBM Forms 4.0 - Introduction, Web 2.0 Look and Feel, AJAX Updates (LARGER YouTube HD)
- The New IBM Forms 4.0 - Dynamic Tables, Form Parts, Integrating Business Analytics (LARGER YouTube HD)
- The New IBM Forms 4.0 - Dojo Controls in Dialogs and on the Forms, Integrating Web Analytics (LARGER YouTube HD)
- The New IBM Forms 4.0 - High Precision Layout, Digital Signatures, Completing a Business Transaction (LARGER YouTube HD)
We also now have several videos that show off the IBM Forms Designer. This release really raises the bar on what level of data interactivity can be created through design GUI configuration rather than grunge coding, and many powerful XML data processing features of the IBM Forms Designer are directly attributable to W3C XForms. This includes Design by XML Schema, Dynamic Tables, Web Service Connections, Interaction Rules (e.g. constraints), and Wizard Creation:
Finally, here's a video series on the new FormParts feature (I didn't make these, but they're excellent and really belong in this post). FormParts allows you to create components that can be reused and kept up to date across a forms library.
If you want to know more about IBM Forms 4.0, then come to our wiki
or to our website
, where you can get more information, case studies and free trial downloads. Thanks for watching!