Modified by John M. Boyer
When I started on Java Server Pages (JSP) as a topic, I had intended it to be a blog topic. But it grew quite beyond blog size, so now that the technical work is finished, I can give you the meta-level on using JSP with Enterprise IBM Forms.
The work I'm telling you about here is intended to make it easy for you to exploit the powerful, simplifying JSP technique within the XFDL+XForms markup of IBM Forms documents. It took a some work to sort it all out, but with that done, it is easy for you to replicate what I did and gain the benefits. I wrote this wiki page on the IBM Forms product wiki to help you get set up, and the page references the developerWorks article I put together to show how to use JSP in your XFDL+XForms forms.
It was pretty challenging to get the JSP to talk to the Webform Server Translator module, so I was pretty happy when that started to work for me. It's one of those cases of only needing a line or two of code, but it being really hard to get exactly the right line or two. As Mark Twain once said, it's like the difference between lightning and the lightning bug. Anyway now that we know the smidge of code, it's easy for you to copy and use in your XFDL-based JSPs.
At first I thought, OK I have a good blog topic, but then I realized we weren't covering the full Forms information lifecycle. Put simply, a form is possibly prepopulated and then served, it collects data, but then it comes back and you have to do something with the data collected. So, back for more work sorting out how to receive a completed form into a JSP and use its values in JSP scriptlet code that helps prepopulate the next outbound form. This was a fair bit less challenging, as it maps very closely to how you start up the IBM Forms API in a regular Java servlet. Remember, JSP is just a convenient notation that the web application server knows how to turn into a Java servlet. JSP just makes it easier for you to focus on your special sauce application code.
Well, now that I could handle the whole Forms information lifecycle, I realized I hadn't covered the software development lifecycle. Back to the salt mines again. The problem was that JSP annotations are incompatible with XML. Although there is an alternative XML syntax for JSP, I devote a section in the article to explaining why it's a bit of a train wreck, and I focus instead on the normal JSP annotations. By representing them as XML processing instructions, we're able to maintain the XFDL and the JSP logic together using the IBM Forms Designer, and then use an XSLT to convert to actual JSP when it's time to deploy the IBM Form. This was really important to me because, quite frankly, if a new feature does not work in the Design environment for a language, then the feature essentially does not exist in the language.
Now, that's a wrap! I hope you like the article and get accelerated development benefit from it. JSP is really for building quick prototypes and demos, and also for solving simpler problems much more simply than using straight Java servlet coding. It's even a really nice complement to using Java servlet coding within a larger project. So don't delay, get ready to use JSP with XFDL today.
Something happened today that seemed like perfect fodder for this blog because it combines a humanitarian story, social business, and a good use case for IBM Forms Experience Builder in a social business context.
An esteemed colleague at my Canada software lab site is the organizer for our site's donations and efforts for a local charitable cause. Turns out, IBM has a special charitable grant program in which IBM makes a substantial additional donation to the charity if a set number of IBMers sign up as volunteers. I really like this model because it essentially gives IBMers a way to vote, not only on how much IBM corporately donates but also where the donation goes. It's the same reason I like charitable tax incentives, as it is essentially the same as the federal government allowing the people to vote on the charitable causes they support and how much tax money goes to them.
Anyway, IBM internally runs an enterprise social business platform called Connections, which includes profile pages for all IBMers. These are like Facebook or LinkedIn pages, only inside the corporate firewall. Still, there is information that would not normally show up on one's profile page, but which my colleague needs in order to complete our internal IBM charitable grant application. So, my colleague has found it necessary to reach out to a large number of people at our site to get this information; she's under a tight timeline, and she needs responses from everyone.
This is a great example use case for IBM Forms Experience Builder. One reason is that my colleague needs to quickly stand up a lightweight custom data collection solution, where we'd miss the grant deadline for our charity if we had to wait for a normal IT project to be completed. So, this is a great example of workforce agility to be gained. Yet, on a technical front, this is interesting because it is so easy to see that the solution needs two forms, and their corresponding backend DB storage, as well as a service to connect one to the other. The fact that you can make all this stuff and hook it up in your web browser without coding is truly a step forward in IT democratization.
In this solution, my colleague can create a first LIST form that simply allows her to enter the names of all the people that she needs to get information from. Each person's email address can then be automatically looked up by hooking up into the form the LDAP lookup service that is added to our installation of Forms Experience Builder.
Then, my colleague can create a second COLLECT form that allows each person to provide the information she needs.
Most importantly, my colleague really needs to be able to re-enter her LIST form at any time, not to add more names, but rather to see which people have submitted the COLLECT form-- and which haven't. To do this, she can automatically hook up the database for the COLLECT form as a service to her LIST form. As each person fills the COLLECT form, their entry on the LIST form can show their required information.
At any point, my colleague can look at her LIST form for blank entries to see those people with whom she has to take "secondary measures" to ensure 100% response before the deadline. Most importantly, her LIST form would already then have all the information she requires collected into one place.
Can you imagine how much more work she'd have to do if she did all this by email? She'd have a flood of emails mixed with her normal email, and she'd have to fish out the information from each email and put it into a list manually. Tedious, error prone, and lots more ways to miss the deadline. My colleague is be able to collaborate much more efficiently and effectively using a 2-form IBM Forms Experience Builder solution, and this is why lightweight data management solutions should be added as an essential ingredient of an enterprise grade social business platform. Specifically, if you're going to purchase IBM Connections, then add IBM Forms Experience Builder.
Last blog I told you how you could get a "Greenhouse" account that gives you access to a cloud-hosted build-without-coding environment for constructing and deploying data-centric IT solutions. The product is called IBM Forms Experience Builder (FEB), and I want to spend some time on this blog giving you a better sense of the solutions you can create.
The solution I'd like to cover is a bit more advanced than a "Hello, World" solution; maybe I'll do one of those in the next blog. But the first thing I wanted to be able to do was use my account to enable myself to share solution files with you over time. Every FEB solution has a body of code that represents it. You can export the serialization of any solution as a file so that it can be imported into another account or onto another FEB server. So, I wanted to create a FEB solution that allowed me to share FEB solution files with anyone. I created that "Simple File Sharer" solution and then used it to share the "Simple File Sharer" solution file. You can get the file here (requires Greenhouse account login).
Once you have the solution file, you can import it into your own FEB account within Greenhouse and then use it to share files, especially any sample FEB solutions.
The Simple File Sharer solution uses ordinary features in the form (user interface). I just dragged and dropped a table item, and then put a name, description and upload component into the table. I also dragged and dropped a textual label so I could provide some basic instructions for users of the solution.
The Simple File Sharer solution uses some more interesting FEB capabilities to implement behavioral features around the form user interface.
- I used the Access panel to indicate that only one user, yours truly, could create a new database record or update a database record.
- Often, once a user has used a Form to create a database record, subsequent visits to that Form should allow the user to update the record rather than creating a new record in the database. An example would be a vote or a survey response. You don't want the same person voting or answering the survey more than once. In my case, I wanted to restrict myself to one database record because I always want to update a single list of shared files, not create a new list of shared files each time I fill out the form. Regardless of your use case, you can do this by going to the Forms tab, clicking the properties of the form, going to the Advanced tab and then checking "Limit to single submission per Authenticated user".
- I used the Stages panel to add an Update stage to the solution so that database records can be updated after their initial creation.
- I used the Access panel to add a "Reader" role, and I assigned "all authenticated users" to that role. Then, in the Update stage, I gave only Read access to the Reader role. So, as soon as you do a Greenhouse account login, you can use the link I provided to read from the database record I created that includes the table of files that I've shared.
In conclusion, the Simple File Sharer solution may only allow me to create a record, but it still ends up demonstrating a number of FEB features that you would commonly use in solutions that allow multiple users to create and edit records. In the future, I'll share more solutions with you that highlight various FEB features that contribute to creating interesting solutions. Remember, it's not about just forms; it's really about the whole solution you can wrap around forms, including automatic database storage, access control, lightweight workflow stages, and even configuring web services-- all without coding. And that's what makes IBM Forms Experience Builder a platform-as-a-service capability that everyone from a line-of-business users up to an IT professional can appreciate!
How would you like to be able to construct, deploy and get results from IT solutions using only your web browser?
Don't believe me? Well, how about coming to the IBM Forms wiki
, where you can watch a few short videos that show you.
You'll be intrigued and want to go the next step. One of the prominently available wiki pages is a community article that gives you a starter pack
of prebuilt solutions like the ones you see in the video. You can download any one or all of them because they're just single files that describe the forms, access control, workflow stages and other resources of each solution. You can import any of them into your own IBM Forms Experience Builder
server, and then deploy them, use them, get results from them, and of course edit them to see how they work or to change them and redeploy them. All from your web browser.
Don't have an IBM Forms Experience Builder server to try it out? Well, now we've gotten to the main topic of this blog article. You can get your own free public access to an IBM Forms Experience Builder server. You can try out any of these starter pack solutions as well as build and deploy any of your own solutions.
Since you will be a builder
of forms experience solutions, we will need to be able to present your solutions to you, distinguished from everyone else's solutions. So, you'll have to start by registering yourself with the system that hosts the IBM Forms Experience Builder server. The system is called Lotus Greenhouse
, so click the link and then choose "Sign up" to get your account.
Once you're able to log in to Greenhouse, you'll get access to a number of software products including IBM's social business software (Connections), IBM Websphere Portal Server, and of course IBM Forms Experience Builder. However, you don't really need to log in to Greenhouse then menu navigate to IBM Forms Experience Builder when you can just bookmark the direct link to IBM Forms Experience Builder on Greenhouse
Once you log in with your Greenhouse user id and password, you'll see the "Manage" solutions page, which lists all of the Forms Experience Builder (FEB) applications that you have designed. This is the page that gives you the ability to create a "New Application" or "Import" one of those starter pack applications, all at the press of a button.
So, now you can try out and evaluate IBM Forms Experience Builder now to see for yourself that there really is a smarter web where you can construct valuable solutions without coding now. If you are building IT solutions for your organization, you owe it to yourself to see how much more effective you'll be at satisfying your organization's IT solution demands. But even more importantly, if you're competing for IT solution services contracts, you owe it to yourself to become an IBM business partner or to expand your partnership to include IBM Forms Experience Builder. And finally, if you like to build industry-specific data management products, then you should consider becoming an IBM value-added reseller (VAR) so you can build your products more efficiently with IBM Forms Experience Builder and go to market with IBM to sell the bundle. In all these cases, you now have the access you need above so you can learn more and get started today.
Forms exist to collect data from web users involved in business processes. Are you a business partner who wants to build solutions more quickly in order to make a higher margin? Then read on!
What if you could use a web browser to design not only the user interfaces of the multiple pages of a form, but also the whole solution for which it collects data?
Now, with IBM Forms Experience Builder, you finally can.
You can define the roles of users in the business process, and you can assign users and groups to those roles. You can even set up open roles whose users are defined dynamically during the business process once the right information is collected earlier in the process. For example, only once you take in a person's name can you access an LDAP service to look up his manager and then assign that person to the manager role for an approval step.
You can define the user interface of a Form, and have an automatic database created on the server side to store database records corresponding to completed instances of that Form. You can even define multiple Forms that work together within a solution that collects data according to different record schemas.
You can define the stages of a business process workflow that uses the Form or Forms to create and update database records. Stage transitions can branch forward, backward or even stay on the same stage to update a database record that still needs more work.
You can define access control for each workflow stage and determine which Forms, page, and UI elements are available in each stage.
You can even use the database records collected with one Form as a GUI configurable web service within the fill experience of a second Form. For example, you could have one Form of a solution that collects inventory data, and then use that data in a second Form that makes it possible to order from available inventory.
You can make the Form fill experience available within a portlet of an IBM Websphere Portal website.
You can create a solution with your web browser, you can save it to the server, you can hit Deploy in your web browser, and then your users can access the Forms of the solution from web links. If you later decide it is necessarily to add to or change the solution, you can edit the solution again using your web browser and hit Deploy again The data is retained for all the remaining form UI elements, and the database tables are altered as needed to make space to store data collected by any new form UI elements.
Via web links, users can access the list of database records collected by the solution. Only the records to which the user has access are presented. If you're the solution creator/administrator, you can get access to all the records. Whomever is given a link to view the records can also set up their own customized filters for the data, so a user can truly use the view as a business process task list, and even filter down to tasks of a particular type, from a particular person, having met or exceeded some value, etc.
Complete agile web solution creation.
Lose the custom coding, gain the market advantage, get IBM Forms Experience Builder now.
Continuing with the amazing stuff you can do with the eval() function: You can use it in a user interface binding to enable your form to programmatically control what the user sees.
As an exhibition of this capability, I'll give you the pertinent parts of an XML editor form that dynamically adjusts to XML structure and allows you to edit the content of any leaf nodes while giving you link buttons to allow you to drill deeper into element subtrees as well as a "back" button to go to the parent of any subtree whose leaves you may be editing. It starts with an XForms repeat, like this:
<xforms:repeat nodeset="eval(repeatexpr)" id="XMLEditor">
The repeat expression is computed by the form and is changed by user actions that drill deeper into the XML tree or go back to parent elements. The repeat expression will end with "/*" so that the controls in the repeat will show the children of whatever node the repeat expression selects before the "/*".
For simplicity, I've put the XML data to be edited as the first element of the instance that also manages the calculation of the repeat expression, but you could do this as two separate instances instead. Here's the instance structure I used in this example:
<xforms:instance id="data" xmlns="">
<Item-Cost type="currency" value="Price * Quantity"/>
<Subtotal type="currency" value="sum(Order-Details/Item/Item-Cost)"/>
<Tax type="currency" value="Subtotal * 0.05"/>
<Total type="currency" value="Subtotal + Tax"/>
The first element could be anything, but I used a "purchase order" data structure, so this form will magically morph into a purchase order editor. Further, it should now be clear why in the last blog I concentrated on data that carried its own formula calculations and data validation rules. If I replace the Purchase-Order element above with the loan calculation data below, then this same form will help calculate your monthly payment on a loan:
<Loan-Application label="ACME Used Car Loan Application">
<Name required="true">John Q. Public</Name>
<Address required="Name != ''"></Address>
<City required="Name != ''"></City>
<StatProv required="Name != ''" label="State/Province"/>
<PostalCode required="Name != ''"/>
<Employer required="../Principal > 20000"/>
<Principal required="true" constraint="Principal > 0 and Principal <= 50000"></Principal>
<Duration required="true" constraint="Duration > 0 and Duration <= 84"></Duration>
<Interest-Rate constraint1="Interest-Rate > 0"
constraint2="Interest-Rate <= 5+floor(Duration div 12)"></Interest-Rate>
<rate hidden="true" value="Interest-Rate div 1200.0"/>
relevant="Monthly-Payment > 0"
value="choose(rate > 0,
Principal * rate div (1.0 - power(1.0 + rate, -Duration)),
Principal div Duration)"
relevant="Monthly-Payment > 0"
value="Monthly-Payment * Duration"
Within the repeat, we can use different kinds of form controls to be responsive to the identified types of data and also to the issue of whether something is an input or an output based on whether it has a computed value. Here are two examples at the XFDL+XForms level:
<xforms:input ref="self::node()[not(*) and not(@value) and @type='currency']">
<xforms:label><xforms:output value="choose(./@label, ./@label, concat(translate(local-name(.), '-_', ' '), ': '))"/></xforms:label>
<xforms:output ref="self::*[not(*) and @value and @type='currency']">
<xforms:label><xforms:output value="choose(./@label, ./@label, concat(translate(local-name(.), '-_', ' '), ': '))"/></xforms:label>
<display compute="xformsenabled=='on' and value != 'NaN' ? 'on' : 'off'"/>
In the predicates of the form controls, "not (*)" ensures that these form controls are only relevant if the data node is a leaf that is to be filled with character content. The "value" attribute in the data provides a calculation formula, so that has been used to distinguish when to provide and input versus an output form control. The two examples above make relevant form controls for data elements annotated with a currency type attribute. Other form controls for checkboxes and dates can be created to bind to types like booleans and dates.
Next, let's look at how the repeat expression is computed:
<xforms:bind nodeset="firstelem" calculate="local-name(instance('data')/*)" />
<xforms:bind nodeset="path" calculate="concat(../firstelem, ../expr)" />
<xforms:bind nodeset="repeatexpr" calculate="concat(../path, '/*')" />
By design of this particular "magic morphing" (XML editor) form, the first element in the computed XPath expression is the first element of the instance. Then we compute the full path to the element whose children will become editable by the repeat and the form controls within the repeat. This is based on the "expr" data node that will be programatically manipulated. Because expr is initially empty in the instance data above, that means you will initially see the children of the first element of the instance, because the repeatexpr is calculated to the first element plus the initially empty "expr" plus "/*" to get the children.
OK, so how do we adjust what the XForms repeat presents to the user? Basically, we want to either add a child element name to drill down into a subtree or we want to subtract a child element name to go up a level. First, let's cover how to add an element, i.e. add a step to the location "path". Inside the repeat, each child element that is a subtree root (has children) gets an XForms trigger in a link style button. If you activate the trigger (press the button) then you drill down into the corresponding node. Here's what that looks like:
<xforms:label><xforms:output value="choose(./@label, concat(./@label, ' >>'), concat(translate(local-name(.), '-_', ' '), choose(*/* or *='', '', concat(' - ', *)), ' >>'))"/></xforms:label>
value="concat(., '/', local-name(context()),
choose(count(context()/../*[local-name(.)=local-name(context())]) = 1, '',
concat('[', 1 + count(context()/preceding-sibling::*[local-name(.)=local-name(context())]), ']')
<xforms:setindex repeat="XMLEditor" index="1"/>
The trigger ref binds to a node that has children, as tested by the predicate "[*]". The label shows the name of the child element whose subtree you will drill into if you activate the trigger. The action sequence simply chucks a slash plus that name onto the end of the "expr" as a new step in the location path. This adds to the "path" which adds to the "repeatexpr" which updates the XForms repeat to show the children of that subtree root.
The trigger to go back up to a parent from a child is something that would live outside of the repeat because you only need one "back" button. It's actually a bit trickier because you can't get the last slash in order to lop off the last location step in the path. Fortunately, XPath lets you find the first occurrence of a substring, and XForms actions include a loop. So, the way I did this was to construct a new expression out of all the location steps in the old one, except the last, which was detectable by there being no more slashes. Here's what that looks like:
<xforms:trigger id="GoBack" ref="expr">
<xforms:setvalue ref="../scratchexpr" value="substring(../expr, 2)"/>
<xforms:action while="contains(../scratchexpr, '/')">
<xforms:setvalue ref="." value="concat(., '/', substring-before(../scratchexpr, '/'))"/>
<xforms:setvalue ref="../scratchexpr" value="substring-after(., '/')"/>
<xforms:setindex repeat="XMLEditor" index="1"/>
The first setvalue copies the "expr" less the leading slash into the "scratchexpr". Then, we clear out the "expr" so we can build it up anew from the parts of the scratchexpr. Now, we execute while "scratchexpr" still contains a slash, so the loop stops short of copying the last location step from scratchexpr to expr. Once the processing is complete, then once again, the modifications made to expr, reverberate to "path" then to "repeatexpr" due to the XForms binds above, and so the XForms repeat updates to show and allow editing of the content of the parent element.
And that's it! Thanks to eval() used in combination with all other pre-existing features of XForms, you can make a form that edits any XML element data structure.
There are a number of new XPath extension functions available to XForms developers in the latest release of IBM Forms, and I'd like to draw your attention to two of them: eval() and eval-in-context(), and they are wicked cool!
The function eval(expr) evaluates expr in the context of the function call and returns the result. The eval-in-context(expr, contextExpr) function does a similar thing, except it first evaluates the contextExpr and uses the result as the context for the main expression. This is desparately needed for XPath 1.0 expressions to eliminate the infestation of pesky ".." operations that typically occurs. I've used it, rather than eval(), in the samples below.
One use of these functions is to enable the powerful capability to let XML data carry sophisticated dynamic metadata, which can then be implemented and enforced with singular XForms bind elements that attach the semantics of the metadata whereever it is found in the data.
It turns out that the xsi:type attribute from XML schema is already a rudimentary version of the metadata idea we're pursuing here, so the question becomes what if you could do it for all of the juicy metadata that XForms contains, like calculated data values, data validity constraints, and so forth? Let's look at what this "decorated" data might look like for a simple expense report:
<total value="quantity * price"/>
constraint="total < 10000"/>
It is really easy to use XPath capabilities to find all elements having a value attribute and then to bind an XForms calculate formula to those elements, and then eval-in-context() is used to determine the result according to whatever expression is given in the data, like this:
<xforms:bind nodeset="descendant::*[@value]" calculate="eval-in-context(@value, ..)"/>
The nodeset expression starts with descendant::* to explore all elements of the data, and then the predicate [@value] selects all elements that have a value attribute. For each such element node, a calculate formula is bound to it by the XForms bind. The eval-in-context() call uses ".." to go up a level so that the formulas in the value attributes can omit "../", e.g. so the expression can simply be "quantity * price" rather than "../quantity * ../price".
Similarly, a data validation constraint can be attached like this:
On the expense report data above, this binding evaluates the constraint expression attached to the total element, and then converts the result to a boolean. Due to the constraint, the expense form data cannot be submitted to a server for processing unless the total expense is less than 10000.
In other scenarios, you may want to control the other metadata properties like relevant, required and readonly. Here's an example of data where relevance and requiredness control is needed:
<parent relevant="age < 18" required="true">...</parent>
The xsi:type assignment for age already works in XForms without needing a bind. The required setting for name is statically true, not dynamically changeable, so it is very handy to be able to allow either a static boolean value or an expression, like this:
required="@required='true' or boolean-from-string(eval-in-context(@required, ..))"/>
Technically, the required property on the parent element is conditional on the age value, but that is an automatic feature of XForms, i.e. nodes marked required are only required if they are relevant. It should not be too surprising to see that the bind for relevance looks like this:
relevant="boolean-from-string( eval-in-context(@relevant, ..) )"/>
At this point, it's probably safe to leave the readonly bind as an exercise for the reader :-)
Information system architects, and even vertical industry standardization or government IT standardization bodies, derive immense value from XML schema definitions that describe the data structures and data types expected in valid transactions of the information system. However, these assets are focused on defining the completed transaction. In many a presentation, I've talked to potential customers about how IBM Forms documents express far more value because they are about the human interaction that takes place during the fill experience needed to produce those completed transactions.
As a bit of an aside, it's important for the technically minded reader to be familiar with the "sell" side of this equation. It is important to be able to easily justify technology adoption decisions with business owners who need to be able to understand how you will be able to better server your customers, reduce development and maintenance costs, increase competitiveness, eliminate vendor lock-in, etc.
In this blog, I'd like enumerate various benefits that you get from the standards basis of IBM Forms documents and their implementations, but I'd also like to separate the enumerations into two lists: 1) benefits above using XML Schema that you get just from using the XForms markup within an IBM Form, and 2) the benefits you get from the XFDL processor (XFDL is the XML vocabulary that IBM Forms provides as a presentation layer for XForms).
Firstly, an XForm is a clearly a superset of XML Schema since an XForm can incorporate the XML Schema if it is available and provide its validation information set to the fill experience and the submission experience. But an XForm provides many additional benefits, including the ability to:
Express data validation constraints that are based on other XML data values entered during the fill experience.
Automatically compute data values based on other input entered during the fill experience, rather than requiring users to perform the error-prone task of calculating and inputting summative results manually.
Describe the inputs, outputs and triggering mechanisms of web services to invoke, such as to use entered XML data to obtain database query results that fill other parts of the form or to invoke server-side validation or calculation logic from a business rules engine.
Control whether a data value is readonly or whether the user can enter a value, for example based on conditions related to a business process step or an access rule.
Express user interface controls that indicate which XML data values will be available to the presentation layer for input or output.
Conditionally show or hide the user interface controls (and hence their presentation) in response to conditions, such as those that may relate to a business process step or access rule.
Define a step-wise fill experience to guide a user systematically through incremental successes in entering valid data.
Provide customized help and validation error messages to help users fix data input errors when they occur.
Provide prompting label text to be associated with the presentation of each input or output control, both visually and aurally (for accessibility)
Associate a selection list with any XML data node in a way that constrains user input capability to the provided list
Define labels, help messages, validation error messages and selection lists in more than one human language within the same form, thereby ensuring citizens receive the same form logic and interaction behaviors regardless of which official language they select to request government services
Associate repeated data with a logically tabular set of user interface controls and encode the means by logical rows of the table are added or removed in response to insertions or deletions of data
While the above benefits indicate what additional behaviors and features of XML forms can be expressed above those that can be expressed by an XML Schema alone, XForms is also more interesting for standardization of a forms repository due to what it does not express. An XForm does not rigorously bind its many behavioral benefit to a specific presentation layer of the form. The intention of this language architecture was to address multimodal requirements of forms applications, e.g. rendition on a desktop, tablet, smartphone, telephone call-in voice service or instant messaging interaction. Different presentation layer implementations can address these requirements, and such implementations can even be provided by different vendors. The XForms working group also anticipated that there would be a wide array of varied technical requirements for presentation layers, and this language architecture allows XForms to be used with fundamentally different XML presentation languages that address these disparate requirements. Examples range from ODF for flowing text with fill-in-the-blanks fields to XFDL with its high-precision contract-style layout capability.
Due to the above mentioned language architecture, XForms markup does not comprise a well-formed XML document until it is incorporated into a presentation layer XML document. XFDL in the XML vocabulary used in IBM Forms to provide a presentation layer for XForms.
Of course, in a "baseline standard" version of a form, the default presentation layer associated with the XForms markup can be minimal in nature so that the only benefit is to provide a well-formed XML document to host the XForms markup. Interestingly, once you have this from XFDL, then the result is in fact an XML document, and so it can be processed by readily available XML processing tools like XSLT. These XML tools can be usd to automate creation of different versions of the baseline form that may have a richer presentation description, alternative natural language usage, or even different presentation layer markup. In addition, various consumers could use simple XML tooling to rebrand the forms.
The Extensible Forms Description Language (XFDL) in an XML vocabulary describing the presentation layer and richly interactive behaviors of modern web-based electronic forms. This XML vocabulary was first introduced to the W3C in 1998 (http://www.w3.org/TR/NOTE-XFDL
) and over the years, the versions of the language have consumed XML data-processing components of the W3C XML technology stack as they have been standardized by the W3C, especially including XML Schema, XForms, and XML Signatures. XFDL is a royalty-free open format whose current version specification, which can be obtained from -
XFDL is a host XML language for the XForms standard, and so the many benefits of XForms described above are inherited by XFDL. The implementations of XFDL as a presentation layer language add the following benefits over the core XForms processing:
very high precision control over the layout and rendition of the user interface
integration of XML Signatures with both XForms and a high precision user interface
comprehensive treatment for accessibility, localization and language support
dynamically computable presentation layer effects based on data value changes
multiple page support and ability to easily integrate web analytics within the form
ability to print or generate a PDF snapshot of the form or of just the agreement style pages
built-in ability to save the form locally and reload it later to continue the fill experience
a built-in file attachments folder system, including the ability of digital signatures to protect files attached into the form
a “public data” map for mapping XForms instance data to business process/workflow variables and case management properties
integrations to standard application server processing of form results and to run-time processing by JSR 168 and JSR 286 compliant portlets
the choice of zero-install operation within a web browser, using a server translator module
the choice of a client install to support both offline and online processing
In a larger sense, though, this only part of the benefit derived from XFDL. Still more benefit is derived from the availability of the XFDL forms visual design environment, which gives form authors integrated access to GUI features for XForms, for XML Signatures, for schema-driven design, for web sevice connections, and for the XFDL language benefits included in the list. The design environment even includes a converter that helps preserve the layout of PDF Forms that are brought over into XFDL. Finally, the XFDL forms visual design environment also provides features for maintaining a collection of forms, such as SVN and other team repository plugins and management of form parts.
Industry solutions is an important area of endeavour for IBM. An industry solution is an IT asset that helps solve an industry-specific problem and is easily reconfigurable to meet specific needs of each client. A solution often helps a client to reach out to and interact with their own customers or users.
An important segment within industry solutions is called case management, which takes the view that a customer/user interaction pattern can be orchestrated by a case. The definition of a case includes data structure and data type definitions, metadata definitions, business process and user access rules, and other possible resources. Based on an initial request by a customer or user, the case management system instantiates the case definition, and the resultant case orchestrates the interaction to achieve the goal or goals implicit in the defined pattern. For example, a case management system could be used to orchestrate the means by which a customer makes and successfully completes a warranty claim for a defective product. The process would begin with collecting initial information about the defective product, about the defect, and about the purchase. The process would include determining the legitimacy of the warranty claim, providing basic support to qualify the defect and determine a course of action, and ultimately to effect a repair or replacement of the product.
A case management user interface is a collection of interactive components that collect data from a user and store it in the data structure of a case (an instance of a case definition). The presentation of the user interface is also affected by the metadata of the case. A common piece of metadata is an enumeration of the valid values that a data item may take. If such a list is available, then it would be presented in a dropdown menu or list box, and the input would be collected via list selection rather than by free-form typing in a text entry field. Other common metadata are boolean flags such as for indicating whether a data node is readonly or required to fill in a step of the case processing. The user interface components would be affected by enforcing the readonly property or providing a sensory indication of the required property. Still other metadata can define validity constraints, such as a numeric datatype or a minimum or maximum value or length. The user interface of a component associated with a data node would be affected by indicating whether the current data value is or is not valid according to the constraints. A case management asset would also typically forbid progress to the next step of the orchestrated pattern when the data associated with the current step contains invalid values.
The term “case management solution” has been used to describe a software solution that supports the design, deployment, execution and reconfiguration of case management assets. As the field of case management matures, the term “advanced case management” has emerged as a way to characterize case management assets that have advanced feature requirements, such as the requirement to collect many dozens, scores or hundreds of fields of data. Some examples of advanced case management include: home or car insurance claims, credit card charge dispute resolution, citizen-facing ombudsperson cases, contagious disease outbreak tracking, and management of complex medical or psychological treatment cases. IBM Case Manager (ICM) is IBM's advanced case management solutions.
A problem arises in advanced case management solutions with respect to the expected maturity of the user interface. The typical case management solution generates a user interface presentation layer for the data of a step using a simple linear columnar approach or a column of expandable stacks of related data values. The advantage of this approach is that it most easily adapts to a reconfiguration of the case management asset in which the data structure is amended to add or remove data nodes. However, there are a number of disadvantages to this approach. For one, it provides a one-size-fits-all approach to the user interface layout in which usability substantially degrades in quality as the size of the data set grows.
As well, larger data sets tend to correlate to more advanced requirements in an overall solution that a case management asset simply cannot begin to address using only a simple user interface approach. There are many such features, including creating multipage guided interview style wizards, creating mutiple print-style pages to reflect a “document of record” for the case, and of course adding the ability to digitally sign the “document of record” as a way to create a legally binding agreement or a record that can stand up to rigorous auditability requirements.
The new release of IBM Forms is the strategic IBM forms technology that now solves this problem for IBM Case Manager (ICM). An IBM Form combines an XML data structure with a template describing interaction behavior rules and a comprehensive user interface definition. The how-to for connecting an IBM Form to an ICM solution is as simple as going into the IBM Forms Designer, right-clicking on any number of XML data nodes, making them “public” and giving them public names equal to the ICM case property names they must map to, and then ataching the IBM Form into the ICM solution.
The IBM Form can then be used in the Case eForm widget anywhere in the ICM solution where the Case Data Widget would have been used. During execution of a case under the ICM solution, the case property values are automatically injected into the XML data of the IBM Form as it is rendered to the user, based on the public data mapping mentioned above. When the user completes or saves the form, the updated data values from the IBM Form are injected back to case properties so that the Form and the Case are in synch.
Oh, but the story is so much cooler than just adding high precision, multipage user interface control for the data. And it's cooler than having a “document of record” for the data that can be digitally signed. IBM Forms contains this fabulous technology for defining live interaction with XML. It's called... you guessed it... XForms. XForms manages not just XML data, but also metadata pertaining to each data node, such as the node's datatype, or whether the node is required, or whether it must be valid according to some constraint expression. XForms also allows a list of values to be associated with the input mechanism for a data node. Remember above where I said that case management assets define metadata just like this for case properties? Well, when you design a case solution that includes this metadata, and then map a XML node to a case property by assigning the public data name, the IBM Forms integration automatically injects not just the case property values, but also any lists as well as XForms binds for the metadata. During the form run-time, the XForms processor then automatically combines these XForms bind results with any metadata settings that might be defined within the form itself. In effect, there is a seamless bridge from ICM case processing to the user's interaction with the IBM Form.
Finally, this seamless bridge works in the other direction too, from the IBM Form back to the ICM case solution. In addition to synchronizing data updates from the form back to the case, the key lifecycle operations of saving or completing a form interaction are gated by a validation operation. IBM Case Manager delegates the validation operation to the IBM Form technology, which executes a validation operation based on the behavior of an XForms submission. This means that non-relevant nodes are automatically pruned from the validation, and the validation result is the sensible combination validation rules injected from the case solution and validation rules expressed directly in the form.
Netting it out, IBM Forms is a first-class citizen of IBM Case Manager solutions, and the case data and metadata of an IBM Case Manager solution are handled as first-class citizens of the user experience provided by an IBM Form.
Recently, I was experimenting with one of the features planned for the next version of XForms. The feature is the iterate attribute for XForms actions, which will perform a for-each loop operation based on a nodeset obtained from the xpath expression in the iterate attribute value. XForms 1.1 already has a while loop, but iterate makes many data processing loops easier to write and also more performant (subject of a future blog). There were lots of iteration use cases to choose from, but I decided to experiment with sorting because it is a well-known benchmark algorithm.
Before we go any further, let me say up front that XForms action scripting is intended for very lightweight data manipulation, like adding or deleting a data node corresponding to a table row or copying data results to or from the SOAP envelopes of a web service. By the time you get to nested iterations like those needed sorting, you should be considering alternatives expressed in full-blown imperative languages available in the information system within which the form is being used. For example, in the case of sorting, it is a better idea to request sorting in the database query whose results are returned from a web service into your form so that your form logic does not even have to do the sorting.
So, with the disclaimer out of the way, let's abuse the technology a bit to get a better sense of what is feasible in those customer-needs-it-yesterday circumstances. It turns out that XForms 1.1 does allow full nodeset processing in the insert action's origin attribute and the delete action's nodeset attribute. Without even needing the new iterate attribute, this is just enough iteration capability to perform efficient sorting -- so there are some kinds of iterations that can be done now without the iterate attribute.
We're going to do a divide-and-conquer "partition" sort that I personally created as a university freshman after my 1st semester instructor told our class that linked lists could only be sorted slowly. At the time, the usual computer languages only allowed static allocation for arrays, and even though I didn't know what a "quick sort" was, I had seen the light of dynamic allocation, and I was never going back! I later learned how great a merge sort is on a linked list, but the effort of turning an array quicksort into a linked list partition sort comes in handy now because a merge sort cannot be efficiently expressed in XForms until the iterate attribute is added.
The way a quicksort works on an array (or subarray) is that you pick a random element to be the 'pivot' value. Then you run two index variables at the same time, one from the start of the array upward and the other going from the end of the array downward. The 'up' index is advanced until it finds a value greater than the pivot, the 'down' index is decremented until a value less than the pivot is found, and then the values at the 'up' and 'down' locations are swapped. This keeps happening until 'up' and 'down' meet somewhere near the middle of the array. At this point you've partitioned the array into a subarray of values less than the pivot value and a subarray of values greater than the pivot value. The quicksort is then invoked recursively to sort both subarrays.
The main challenge with this approach is the 'down' index, which is a reverse iteration. In a singly linked list, you can only go forward. XForms insert and delete actions have a similar limitation: they can only identify a nodeset of nodes to insert or delete, but not really a direction of iteration. But the important bit is what the quicksort is doing, not how it is done. Think of the list content as being completely messy, and each partitioning stage must make it somewhat less messy by dividing the content into a partition of lesser elements and a partition of greater elements. Then, the next partitioning stage is invoked recursively to do a better job of cleaning up the mess within each partition.
Let's explore this concept by sorting a list of elements, such as sorting a list of <person> elements by a <lastname> child element. We begin by copying the list into an initial partition element of a temporary instance called 'sortdata', like this:
1) <xforms:insert context="instance('sortdata')" origin="instance('partition')"/>
2) <xforms:insert context="instance('sortdata')/partition" origin="instance('data')/list/person"/>
3) <xforms:delete nodeset="instance('data')/list/person"/>
Next, we initialize the random number generator so we can randomly select pivot values for all the partitioning stages:
4) <xforms:setvalue ref="instance('sortpivot')" value="random(true)"/>
Next, we start up a simple while loop that continues to process partitions until none are left.
5) <xforms:action while="instance('sortdata')/partition">
Within the loop, we grab the last partition from the sort data and determine whether it is non-trivial or trivial (only 1 or 2 elements). A non-trivial partition is subjected to further divide-and-conquer processing.
5.1) <xforms:action if="count(instance('sortdata')/partition[last()]/*) > 2">
The first part of dividing and conquering is to create a new empty <partition> element, which we obtain from an instance that expresses a template empty partition element.
5.1.1) <xforms:insert nodeset="instance('sortdata')/partition[last()]"
Step 5 and step 5.1.1 are more interesting than they seem at first. The list of partition elements in the sortdata actually implements the recursion stack, and we just pushed a new element into that stack at the second-to-last position. Because we have an explicit stack, we only need a loop in step 5 to implement recursion.
The next thing we do here is grab a random last name to serve as a pivot value for the partitioning. The first setvalue just picks a random location, and then the second step uses the location to get the value. Notice also that I use * rather than partition before the [last()] predicate because the sort data only contains partition elements, so there is no point in doing a name test for partition.
5.1.2) <xforms:setvalue ref="instance('sortpivot')"
value="1+floor(random(false) * count(instance('sortdata')/*[last()]/*))"/>
Now the magical part happens. All elements in the last partition whose key element (lastname) is less than or equal to the pivot value are moved to the newly created second-to-last partition. By combining the nodeset processing capability of XForms insert and delete actions with the predicate-based node selection capability of XPath, the matching nodes can be selected and moved using two single XForms actions, i.e. without using an XForms while loop.
5.1.3) <xforms:insert context="instance('sortdata')/*[last()-1]"
origin="instance('sortdata')/*[last()]/*[compare(lastname, instance('sortpivot')) <= 0]"/>
<xforms:delete nodeset="instance('sortdata')/*[last()]/*[compare(lastname, instance('sortpivot')) <= 0]"/>
If the last partition is now empty due to the move operation in step 5.1.3, then the new second-to-last partition received all its elements. If all the moved elements are equal to the pivot value, then we can output them back into the original data list and then remove the last two partitions. Note that the insert is configured to prepend the elements into the data list, and we're copying them from the last non-empty partition, which has the elements with the greatest key value.
5.1.4) <xforms:action if="count(instance('sortdata')/*[last()]/*) = 0">
<xforms:action if="not(instance('sortdata')/*[last()-1]/*[compare(lastname, instance('sortpivot')) != 0])">
<xforms:insert context="instance('data')/list" origin="instance('sortdata')/*[last()-1]/*"/>
<xforms:delete nodeset="instance('sortdata')/*[position() >= last()-1]"/>
</xforms:action> <!-- End of 5.1 non-trivial partition handler -->
Now, we've finished with the non-trivial partition handler, and we turn our attention to processing a trivial partition containing at most 2 elements. The content of the partition is moved to the original data list and the partition is removed. Again, note that we're processing the last partition, which has the greatest key values, and the insert prepends to the data list, so the sorted data list starts with the greatest values and grows as lesser and lesser values are prepended over time as all the partitions are processed.
5.2) <xforms:action if="count(instance('sortdata')/*[last()]/*) <= 2">
If the partition contains two elements and the greater one is first, then it is removed from the partition and put in the original list:
5.2.1) <xforms:action if="count(instance('sortdata')/*[last()]/*) = 2 and
instance('sortdata')/*[last()]/*/lastname) > 0">
<xforms:insert context="instance('data')/list" origin="instance('sortdata')/*[last()]/*"/>
Now the partition contains either no elements, one element, or two elements in sorted order, so we move the content to the data list and then delete the partition.
5.2.2) <xforms:insert if="count(instance('sortdata')/*[last()]/*) > 0"
</xforms:action> <!-- End of step 5.2 trivial partition handler -->
</xforms:action> <!-- End of step 5 recursion while loop -->
As a final note on all this algorithmic fun, the question arises whether this sort achieves optimal O(N log N) performance. The answer is no, not quite, due to hidden costs of data instance management and data node selection. However, the sort will be much faster than a "simple" sort because it does perform only O(N) XForms actions.