Well, no blogs from me next week, so seems a good idea to knock off another one this week...
XML namespaces rec (pardon the pun) states that an attribute which is namespace unqualified
is 'local' to an element and is uniquely identified by a combination of the attribute local name and the type and namespace URI of the containing element.
The word identify
really should be used more sparingly, as here is a case where its misuse has caused years of confusion and acrimony in the XML community. An identifier is something that established the identity of something else. You cannot have two things associated with the same identifier unless they are identical things.
I am often frustrated by seasoned W3C folks who say that "this depends on your definition of identify" and honestly
believe this is a defense of the confusion in the namespaces rec. This is like saying ot me, "Well you're right unless you have a definition for identify that doesn't identify things, which is what we
did at the W3C."
To see the problem, you have to look earlier in the spec where a namespace n1 is associated with a URI and then the following code appears:
<good a="1" n1:a="2" />
The problem is that the element 'good' is in the same namespace as n1. So now you have local attribute a
that is essentially given meaning by an element that is in the same namespaces as the 'global' attribute n1:a
. Yes the two attributes have different values.
From this we have to infer that, although a local attribute is given meaning based on the containing element and its namespace, a global attribute with the same local name and namespace qualified into the same namespace can actually mean something totally different.
In other words, we have two attributes with the same local name, contained by the same element, and given meaning by the same namespace URI, but they are not identical. This is the local attribute not
being 'identified' (in my sense) by local name and containing element type and namespace.
The technically subtle W3Cer will tell you that there was no reason to spell out using words in the normative part of the spec the fact that the two attributes are different things because the spec says that local attributes are in a different partition than global ones. Problem is, this partitioning info is in a non-normative part of the spec, and particularly in the same part that has the language about how local attributes are 'identified' by local name and containing element type and namespace.
Anyway, the upshot is that an XML vocabulary does not need to but is allowed to say that local and global attributes with the same local name can mean different things. If they're supposed to mean the same thing, then the XML language has to define a precedence rule for what happens if the two attributes differ in value. Here's an example:
<data xmlns="http://example.org" xmlns:ex="http://example.org">
<price currency="USD" ex:currency="EUR">10</price>
Question: Is the price in USD or Euros?
Answer: Depends on who designed the language.
Second answer: Don't do that.
Interestingly, the XHTML working group came up with a fascinating example where it is legitimate and sensible to have a global and local attribute with the same local name but completely different meanings. It has to do with the next version of XML events. After hours of discussion, the decision was that they weren't going to do that (second answer above). Not because it's illegal, but because it's too subtle for most XML people.
Well, the XHTML group may change their minds, but even if they don't, the example is really worth understanding because it actually makes sense why you'd want to have the two attributes mean something different. Stay tuned, I'll tell you all about it when I get back...[Read More
The hit rate on my last blog announcement about IBM Forms 4.0 makes it clear that the release is pretty big news. I'm very pleased about that because it is a *major* release, both technically and in terms of the corporate branding. Corporate branding is not a decision lightly made, so it took the time it took, and meanwhile we didn't want to delay the technical features hitting the market, so you'll see some references to "Lotus Forms" under the covers of the main IBM branding. Not to fear, we're working on that for the next release.
Meanwhile, now that IBM Forms 4.0 is released, I've made a number of demonstration videos to show you key aspects of the IBM Forms Design and Run-time products. They're available on YouTube in HD, and they're broken into small segments, so you can really take them in pieces according to your interests.
To start, here's a 5-part series that takes you through run-time features with a demo form that ships with the product. These videos help you understand the value proposition of IBM Forms as well as new features of the release that will change your mind about what forms are and what they do.
- The New IBM Forms 4.0 - Introduction, Web 2.0 Look and Feel, AJAX Updates (LARGER YouTube HD)
- The New IBM Forms 4.0 - Dynamic Tables, Form Parts, Integrating Business Analytics (LARGER YouTube HD)
- The New IBM Forms 4.0 - Dojo Controls in Dialogs and on the Forms, Integrating Web Analytics (LARGER YouTube HD)
- The New IBM Forms 4.0 - High Precision Layout, Digital Signatures, Completing a Business Transaction (LARGER YouTube HD)
We also now have several videos that show off the IBM Forms Designer. This release really raises the bar on what level of data interactivity can be created through design GUI configuration rather than grunge coding, and many powerful XML data processing features of the IBM Forms Designer are directly attributable to W3C XForms. This includes Design by XML Schema, Dynamic Tables, Web Service Connections, Interaction Rules (e.g. constraints), and Wizard Creation:
Finally, here's a video series on the new FormParts feature (I didn't make these, but they're excellent and really belong in this post). FormParts allows you to create components that can be reused and kept up to date across a forms library.
If you want to know more about IBM Forms 4.0, then come to our wiki
or to our website
, where you can get more information, case studies and free trial downloads. Thanks for watching!
new release of our forms software, version 4.0, is now only a few weeks from shipping. And as of this release, the product line will be known as IBM Forms
! This is an incredibly important indicator of the strategic value IBM sees in the Forms business as a key component in building a Smarter Planet. The feature set coming in this new major release combines the best of Web 2.0 client application behaviors and design experience with the traditional strength of interactive XML data collection for which the prior releases our product line are well known. IBM Forms documents are interactive web application instances that have many traditional capabilities that we have been building into them since 1993.
- They can always be serialized to provide a user with their own saved copy of their web application experience, to archive, to email, or to pass to the next step of a business process or workflow. They can be reinstantiated at any time to continue the interactive fill experience.
- They can be instantiated directly into a web browser without using a plugin, and they can also be instantiated in a client application for an offline fill experience, dramatically simplifying IT maintenance and platform support, .
- They provide a multipage high precision visual layout with comprehensive accessibility support and localization
- They operate over XML payloads that directly conform to the needs of back-end business functions, enabling straight-through XML processing systems.
They enable rich web application interaction and behaviors based on the W3C XForms 1.1
- They allow users to attach supporting documents into the form in support of their business function, including spreadsheets, word processing files, images, videos, etc.
- They secure the business function implemented by the form with a comprehensive interactive document digital signature system that includes sectional signing, multiple signatures, and overlapping signatures, all of which protect not just the form data, but the form behaviors, the form appearance and the attached documents as well.
Now lets add to that what's coming in the new release:
- The ability to drive customer satisfaction and employee productivity through more efficient, compelling interfaces that can include
- gradients, border styling, thematic color control, combined text and images
- AJAX-driven XML web service updating without web page refreshing,
- formatted HTML text for bold, italic, underline, text colors, font control, lists, hyperlinks, etc.
- Embedded custom HTML extensions and popup dialogs that can show maps or movies or even interact with the running form.
- The ability to deliver integrated solutions faster and spend less time and money maintaining them through a comprehensive new set of Design Experience features including
- Form Parts that make it easy to reuse and maintain common components consisting of one or many user interface controls and corrresponding behavioral markup
- Data-driven form design with XML schemas that define back-end data formats; data type sensitive automatic user interface creation, including tables and label annotations
- Web service integration wizard that provides point-and-click, drag-and-drop web service consumption and user interface mapping.
- The "wizard of wizards" that enables rapid development or addition of a step-by-step wizard experience for a form
- SVN source code repository interaction, Integrated Design Time spell-checking, etc.
- The ability to use your Forms solutions how you want and where you want, including
- An automatic forms view portlet and support for Portal 7.0 and Mashup 2.0
- Updated browser and OS support and improved integrations
- Improved workflow system integrations with Filenet and Websphere BPM
- Webform Server support for the iPad
The real achievement here is not just these new features, but getting them to work well with all the traditional features. IBM Forms 4.0
, coming soon to a Smarter Web Application near you.
The purpose of XForms is to express the core XML data processing asset used in sophisticated data collection scenarios.
In fact, it would be better if XForms were called the XML data processing language (XDP or XDPL) because XML is about standardizing data and about 80% of business transactions are based on filling out some kind of form to collect the transactional data.
An XForm contains one or more XML data instances. An instance is an arbitrarily structured XML data document that is typically an instance of some XML schema that expresses the static validation rules for a target namespace.
One can write an XForm without an XML schema by just expressing the XML data in an instance. This is because XForms provides other channels of data validity checking that can be easier to work with when only simple data type validation is needed. For example, you can use an XForms
type declaration to associate an
xsd:date or similar data type to an XML data node without writing an XML schema for your XForm.
But XForms validity checking is also dynamic, in recognition of the fact that validity of some values can be based on other values or the aggregation of other values. For example, in an interlibrary article request, the upper bound page number in the journal must not be less than the lower bound page number. Or, the user is only authorized to make a purchase order with less than $10,000 total value.
The latter example is important because it leads to the conclusion that we not only need a way of testing data values relative to other data values, but also that we need a way of calculating data values that are then used in validity tests.
From there, it is not a big leap to conclude that generalized XML data processing requires some way to indicate dynamically whether further changes should be allowed to certain pieces of data or whether certain parts of the data are still applicable to the transaction based on other data values. A good example is a mortgage preapproval form that can handle both single and joint applications. The co-applicant data is only relevant if the user selects the joint application mode.
XForms allows the form author to express formulae for these aspects of data, which are called model item properties, or just MIPs. Not too surprisingly, the names of these MIPs are
Of course, there is no point in representing data, calculating values over data, and validating data if you have no way to change the data. XForms allows simple content data values to be changed, but it also allows insertion and deletion of larger blocks of data that contain internal structure because this is essentially what's needed to add or delete a row from a table.
Most importantly, XForms offers form controls that expose data to the surrounding application context. If the data changes, the form controls change. This includes not only exposing a changed simple content data value, but if a set of form controls are associated with a repeated sequence of structured data, and the number of data nodes in the sequence changes, then form controls are created or destroyed as needed to respond to the change of data.
XForms is all about thinking of the data first and driving outward to how that data gets exposed to applications. Perhaps the most prevalent of such applications are for presenting the data to a human user, though even human users have highly varied capabilities. For example, the desktop user and the PDA user have very different visual capabilities. Of course, this argument extends easily to meeting the far greater accessibility needs of the sight-impaired.
For this reason, the XForms form controls represent what I've often called an intent-based user interface. It's kind of neat to see the term popping up more frequently now. It gets to the heart of the matter: XForms does not provide a presentation layer. XForms relies for presentation on a host language like XFDL (in Workplace Forms) or XHTML (in web browsers). I am certainly hoping that VoiceXML will come to the conclusion that they should soak up the benefits of XForms rather than reinventing all of this stuff over again (partly because almost everybody underestimates how much work goes into it until it's too late; but maybe they will prove to be wiser than the rest).
I sometimes get asked whether XForms will next extend itself to standardizing the actual presentation layer. Clearly, from above the answer is no. XForms standardizes the core XML data processing asset, and more work will go into doing a better job of that. The key issue we want is to address interoperability and reusability across applications and user contexts of the data processing behaviors that are fundamental to completing a transaction.
As an example of the powerful data-driven dynamism available in Lotus Forms due to features of XForms, I'd like to take you through a brief conceptual tour on the focused example of creating a Lotus Form template for a Questionnaire or Survey. This template is able to handle not just any number of questions and any amount of question text, but also any kind of answer type. And all of this would be controlled by the data so that the actual design of the Lotus Form template is the same.
The power of being purely data-driven should not be glossed over. You can easily have web application servlet code that obtains the questionnaire template and then prepopulates it with specific questionnaire data so that the client side receives a specific questionnaire selected in a previous step of the web application. But, XForms-based Lotus Forms also have that AJAX property of being responsive during run-time to new data obtained by a form via web services or other http submissions. So, you could even have a Lotus Form that obtains and adds new questions on the fly in response to answers provided to initial questions.
This post will focus on the main repeating template that provides the dynamic presentation layer for each question of a questionnaire or survey. As this is an example of a purely data-driven questionnaire, let's start by looking at a sample data format. Suppose you have a survey consisting of any number of items, each of which can contain a question text, an indication of the type of question being asked, a place for an answer, and optionally some possible choices for those answers. Something like this:
<question type="yesno">Do you like apples?</question>
<question type="likertscale">It is OK for apples to have a powdery texture.</question>
<question type="closedselection">What is your physical gender?</question>
<choice label="Female" code="F"/>
<choice label="Male" code="M"/>
In the XFDL presentational language that Lotus Forms combines with the XForms data processing layer, every XForms user interface element has a container XFDL element
for presentation. The survey format consists of a number of <item> elements, so an XFDL <table> containing an <xforms:repeat> is the correct top-level presentation element:
... <!-- UI for showing one item of data -->
... <!-- More XFDL options for styling the whole table -->
The table has a scope identifier (sid) attribute that allows the table to be programmatically referenced, but we won't be using that feature in this example. The table can also have XFDL options outside of the <xforms:repeat> to control presentational aspects like borders and background colors, and we aren't focusing on that either.
The <xforms:repeat> has an attribute called "nodeset" which uses an XPath expression to make a reference to however many <item> elements are in the <survey>. This is an automatic or "declarative" loop construct. For each <item> node in the data, no matter how many there are, the template content of the <xforms:repeat> is generated to present that <item> to the user. Even if new <item> elements are added at run-time, e.g. by a web service or an <xforms:insert> action, the XFDL table in the Lotus Form will dynamically grow to present the new <item> elements. And even if some <item> elements are removed from the data, e.g. by an <xforms:delete> action associated with an XFDL <button> by an <xforms:trigger>, the XFDL table will dynamically and automatically remove the corresponding user interface elements that were presenting those removed <item> elements.
So, the magic really happens in the template inside the <xforms:repeat>. In Lotus Forms, you can put any and all kinds of XFDL items in the <xforms:repeat>, including more XFDL table items. In this example, we will be showing a few variations that present different kinds of user interface controls for collecting a few different kinds of answers to questions.
First off, though, presenting the actual question text for an <item> is a simple matter of using an XFDL label item with an <xforms:output>, like this:
... <!-- more XFDL options for presentational styling -->
... <!-- more XFDL items for collecting answers -->
For each survey <item>, an XFDL <label> item is generated, and it binds to the <question> child element of that associated <item> using the "ref" attribute. The XFDL label item presents the text of the bound <question> node, and other XFDL options can be used to provide styling such as the block layout flow as well as alternative font color, background color, font selection and so forth.
More XFDL items can be added to the <xforms:repeat> to collect the answer for the given question. In many cases of XFDL tables, each XFDL item within the <xforms:repeat> template is actually presented to the user. An example would be using each XFDL item in the <xforms:repeat> to represent one column of a purchase order table. However, it is not necessary to show all of the XFDL items within the <xforms:repeat> template. In fact, XForms user interface controls have a selective binding feature that XFDL items support, since the XFDL items are wrappers for the XForms user interface elements.
The selective binding feature of XForms will be used to help easily choose one XFDL item from among many to collect the user's answer to the question. Each question can have a different type of answer, so each "row" of the table can make a different choice of user interface control used to collect the answer. The selective binding feature uses an XPath predicate to decide whether or not the XForms user interface element binds to a node of data or not, and the control is invisible if it is not bound to a data node.
In the example survey data above, the first <item> contains a <question> whose type attribute indicates it is a "yes/no" question. Inside the <xforms:repeat> we can create a checkbox item that can collect a (schema valid boolean) true/false answer, as follows:
The above checkbox widget only binds to <answer>, and therefore is only visible, if the corresponding question type is 'yesno'. Otherwise, the XPath in the ref attribute of the <xforms:input> does not select any nodes, so the XFDL <check> item is not visible.
The second <item> of sample data above has a type of 'likertscale', so we would like to show a 5-point radio button group rather than a checkbox. As explained above, the check box on the second row of the survey table automatically hides itself due to selective binding, so all we have to do is add an XFDL <radiogroup> item to the <xforms:repeat> to provide the interface for collecting the 'likertscale' type of answer, as follows:
<xforms:select1 ref="answer[../question/@type='likertscale']" appearance="full">
<xforms:label>Neither agree nor disagree</xforms:label>
The third survey <item> in the sample data above provides a closed selection of choices. That could be styled using a pair of radio buttons, a pair of mutually exclusive checkboxes, a list box, or a popup control that provides a simple dropdown list. The answer types in the survey format could be made to distinguish these possibilities using more keywords, but for this example we'll just assume that a <popup> control is the desired presentation for a closed selection. The XFDL markup below shows how this can be done, and it is also interesting because it is shows that the data can also dynamically control the choices, rather than having only static choices as shown in the <radiogroup> above.
<xforms:select1 ref="answer[../question/@type='closedselection']" appearance="minimal">
It seems a useful, now, to round out this blog post by presenting a few more examples for other common types of input, such as single-line strings, multiline text, and dates. Here's what the data would look like:
<question type="oneline">What's your name?</question>
<question type="multiline">Do you have any other comments?</question>
<question type="date">What is your date of birth?</question>
The corresponding XFDL items that would be added to the <xforms:repeat> content template for these types of questions would be:
So, hopefully you now have the idea that a completely dynamic and completely data-drive survey or questionnaire can be created using the features of XForms in XFDL (Lotus Forms). Any number of XFDL items can be added to the <xforms:repeat>, XPath predicate selection can be used to choose one XFDL item from among many to collect an answer for a survey question, and most importantly that a different choice of user interface control can dynamically selected for each survey question.
Modified by John M. Boyer
Ever since my first blog entry in this recent series on artificial intelligence, I've been highlighting the lesser, calculational nature of machine intelligence and learning-- as well as the valuable role it nonetheless can play in driving more effective human understanding and decisions. I've been doing this by articulating mainly what machines do, as that is the primary interest of mine and most who would read a developerWorks blog. Still, our interests will be served by taking an entry to discuss human learning as a counterpoint or contrast.
The multiple linear regression example in my last post is a good example to start with because it highlights the difference between accuracy versus understanding. If there is a linear relationship among the data, then an MLR can have a very high predictive accuracy, but it has no explanatory power whatsoever. The MLR model does not have, nor does it convey, any understanding as to why the relationship exists.
Let's see how this predictive accuracy rates in terms of human intelligence and learning. In this case, we can benefit from an instance of that delightful human propensity to apply ideas to themselves. Specifically, we humans have applied our learning abilities to the phenomenon of our learning abilities, with many useful results including Bloom's Taxonomy.
According to Bloom's taxonomy, the very lowest level of cognitive learning is the knowledge level, or the ability to remember and recall what is learned. When you think about it, you realize that an MLR model, like many predictive analytics, is really a storage mechanism for something that has been machine learned from data. In MLR, we store the constants of a linear formula as the representation of what has been learned from linearly related data.
The next higher level of Bloom's taxonomy is comprehension, which is where understanding and true explanatory power begin to surface. But human learning is so much more sophisticated than the knowledge level of machine learning that there are a number of levels above comprehension. There's the application level, in which we can use our knowledge to solve new problems, including being able to explain why the new solution works. The analysis level drills deeper into our ability to make inferences and generalizations. The synthesis level begins to get at our ability to be creative with what we've learned and come up with new ideas and solutions. Finally, the evaluation level gets at our ability to be subjective and judge quality and creativeness of ideas and solutions. We are beginning to see some faint glimmers of some elements of some of these levels in cognitive computing efforts like IBM Watson, but it is early days indeed.
While we're on the subject of human learning and Bloom's Taxonomy, it makes sense to digress for a bit and mention the IBM Social Learning product. This is a SaaS educational platform intended to help enterprises achieve a Smarter Workforce. A few reasons for the digression are
this is a product I currently work on,
both it and the whole Smarter Workforce initiative are being featured at the IBM Connect 2014 conference being held right now, and
learning is a key ingredient of how a human workforce becomes smarter.
The IBM Social Learning product has a very nice feature that enables educational administrators to implement Bloom's Taxonomy in their learning materials. A component of the product is the Kenexa LCMS, or learning content management system, which includes various subcomponents like a course designer and a metadata dictionary. The educational administrator can add any metadata tag, such as "Learning Goal", and any tag values, such as "Basic Knowledge", "Comprehension", "Application", etc. Once this is done, the educational administrator can use the metadata tag values to classify any learning item in the LCMS accord to Learning Goal. Once these classified learning materials are published, learners can use the "Learning Goal" as a new faceted search criterion in the platform's learning library. A learner would be able to isolate and focus on "knowledge" level learning in a subject area before proceeding to comprehension and then application, for example. This will enable learners to effectively use the natural way in which their learning blooms, i.e. Bloom's Taxonomy.
Finally, there is an aspect of human learning that goes beyond Bloom's taxonomy, and it's an area that is highlighted by the IBM Social Learning product. There is a very important word in the product title: Social. This is crucial because it underscores the central role of communication and collaboration in the human learning process. We are an order of magnitude more effective at learning based on our interconnectedness to others who think and learn, rather having access to just data. This is pertinent to the advancement of artificial intelligence because "social" goes quite beyond the computing architecture underlying a lot of today's machine learning efforts.
A lot of talks about XForms are a bit technical in nature because the people who manufacture XForms processors tend to be technical people who understand the business value of XForms in terms like "model-view-controller" architectures, a superset of AJAX, and software engineering benefits like abstraction. But the C-level executive cares not about these things. It is important to connect them to what the C-level executive does care about.
The C-level exec is about efficiency, flexibility and accountability.
The CEO wants
- efficiency via reduced operating costs and decreased time to close deals
- flexibility to react to new business processes and changes of business partners
- accountability for control of expenditures and compliance with regulations
The CIO wants to achieve
- efficiency through end-to-end business process integration and ability to leverage business objects as IT assets
- flexibility through a malleable system architecture that can be rapidly reconfigured with replaceable, reusable components
- accountability through transaction record auditability
We need to speak about how XForms helps the organization to achieve better results along these metrics. This is where the global picture of what XForms does against the backdrop of the classic 3-tier architecture comes in handy.
The C-level exec is not up at night worrying about the server tier, where databases, content managers and workflow engines live. These may be expensive systems, but they are designed to be robust and highly scalable systems whose metrics relative to the size of the anticipated user base of a system are easier to quantify. In other words, they are low risk numbers that are easy to budget for.
The C-level exec is not as concerned about the client tier, where the browser and OS live, again because the metrics are easy and stable.
The C-level exec has the most uncertainty and risk at the middle tier. There are two parts to it. There is the part that uses APIs to talk to DBs, CMs and workflow engines to implement custom application logic as needed. This part is not as scary because we have reliable, robust, scalable, stable APIs, and because the components we're talking to are those reliable, robust, scalable, stable, expensive systems sold by companies with important people's ties to yank when there is a support problem. Then there's the scary, assembly-language-of-the-web-part of the middle tier. This is the part that has to juggle a dynamic, multistep end-user experience on the anything-goes client side, where the web browsers are free and you get what you pay for. The upfront cost of the thin client is low, but the time to market with new offerings is the most highly affected because a lot of unpredictable, time-consuming work lives here.
- It allows you to standardize and consolidate the end-user experience into a single business object that represents the overall transaction.
- It allows that business object to access the web services of an SOA as a natural part of the process of going from empty transaction data to completed transaction data.
- It fully represents the transaction and therefore can function as an integral part of the records needed for auditability.
As a final thought, it should be clear from the exponential scales of complexity and cost that the diagram above argues that XForms-based systems have business value through complexity containment, and that said value is rightly reflected in software product cost.
Yesterday I presented a full paper on a new method and research prototype that describes our thinking on the strategic technical direction of the Lotus Forms document format and processing model. The paper is now available from the ACM Digital Library and is entitled "A REST Protocol and Composite Format for Interactive Web Documents". If you have one, use your institutional access to the ACM Digital Library. For IBMers, you can get access using KnowledgeGate.
As you may know, the document format for Lotus Forms is called XFDL, and it has grown and changed many times in the last 15 years to become more and more comformant to open standards. Once upon a time, back in the 1990's, XFDL was not even an XML format. Currently, the XFDL format is an XML vocabulary that describes a high precision multipage layout for an interactive Web 2.0 application, and XFDL incorporates other standards including XForms, XML Schema and XML Signatures to express dynamic behaviors, business rules, client-side data validation, web service access, and digital signature security.
This open standards transition has represented an important and substantial evolution of the Lotus Forms document format and processing model over time, and this new paper on "Interactive Web Documents" articulates the next step in the evolution of Lotus Forms. As with any strategic operation, it is the kind of thing that plays out over years in the product line, but I am used to that, and I am very excited by our progress.
The initially empty interactive web document template would describe the web application. To initiate a business process, the template is instantiated, becoming an interactive web document capable of holding XML data and file attachments provided by end-user participants and machine agents in the business process. The instantiation sets up a REST service keyed by a unique ID to the instantiated document, and simple GETs, PUTs and DELETEs are used to access and update the resources in the composite document. In particular, XForms submissions are used to synchronize data and change pages within the interactive web document. When a user completes their work on the document, it is the edit token, not the whole document that is passed along through the business process. In this way, the interactive web document and its REST interaction session become the central artifacts over which the entire business process operates, with document serialization only being required to transmit the interactive web document (i.e. the web application combined with its current state) across enterprise domain boundaries.
This is the general idea, and the paper contains numerous additional technical explanations and markup examples. It should be clear from this description, though, that this next evolutionary step will enable Lotus Forms to more easily adapt to and incorporate emerging Web 2.0 technologies, via its page basis in HTML and XHTML, and it will provide a quantum leap in performance of the Webform Server by avoiding a single large XML DOM for large forms applications, by offloading XForms processing to the client-side, and by avoiding serialize/parse operations during the course of the business process.
Modified by John M. Boyer
Machine learning today is every bit as calculated, as simulated, as is machine intelligence. It is easier to use machine intelligence to highlight how much greater human cognition is, which is why I've been using a machine intelligence algorithm over the last several entries. However, the conclusion drawn so far is that, while machine intelligence is only simulated, it is still quite effective and valuable as an aid to human insight and decision making. Machine learning offers another leap forward in the effectiveness and hence value of machine intelligence, so let's see what that is.
Machine learning occurs when the machine intelligence is developed or adapted in response to data from the domain in which the machine intelligence operates. The James Blog entry only does this degenerately, at a very coarse grain level, so it doesn't really count except as a way to begin giving you the idea. The James Blog entry plays a game with you, and if he loses, he adapts by increasing his lookahead level so that his minimax method will play more effectively against you next time. In some sense, he learned that you were a better player. However, this is only a single integer of configurability with only a few settings of adjustment that controls only one aspect of the machine intelligence algorithm's operation. To be considered machine learning, a method must typically have a more profound impact on the operation of the algorithm, with much more adaptation and configurability based on many instances of input data. An example will clarify the more fine grain nature of machine learning.
The easiest example of which I can think is a predictive analytic algorithm called linear regression. Let's say you'd like to be able to predict or approximate the purchase price of a person's new car based on their age. Perhaps you want to do this so that you can figure out what automobile advertisements are most appropriate to show the person. Now, as soon as you hear this example, your human cognition kicks in and you rattle off several other likely variables that would impact the most likely amount of money a person is willing to spend on a car, such as their income level, debt level, nuclear familial factors, etc. This analytic technique is typically called multiple linear regression (MLR) exactly because we humans most often dream up many more than two variables that we want to simultaneously consider. Like most machine learning techniques, MLR does not learn of new factors to consider by itself. It only considers those factors that a human has programmed it to consider. When they are well chosen, additional variables typically do make an MLR model more effective, but for the purpose of discussing the concept of machine learning, the simple two-variable example suffices since your mind will have no problem generalizing the concept.
Suppose you have records of many prior car purchases, including a wide and nicely distributed selection of prices of the cars and ages of their buyers. This is referred to as "training data". If you plotted the training data, it might look something like the blue points in the image below. Let purchase price be on the vertical Y axis since it is the "dependent" variable that we want to predict, and let age be on the X-axis since it is a predictor, or "independent" variable. MLR uses a standard formula to compute a "line of best fit" through the given data points, again like the one shown in red in the picture.
A line has a formula that looks like this: Y=C1X1+C0, where C1 is a constant that governs the slant (slope) of the line, and C0 is a constant that governs how high or low the line is (C0 happens to be the point where the line meets the Y-axis, and the line slopes up or down from there). If we had more variables, then MLR would just compute more constants to go with each of them. For example, if we wanted to use two variable predictors of a dependent variable, then we'd be using MLR to create a line of the form Y=C2X2+C1X1+C0.
Technically, MLR computes the constants like C1 and C0 of the line Y=C1X1+C0 in such a way that the line minimizes the sum of the squares of the vertical (Y) distances between each data point and the line. For each point, we take its distance from the line as an amount of "error" in the prediction. We square it because that gets rid of the negative sign (and, less importantly, magnifies the error resulting from being further from the line). We sum the squares of the errors to get a total measure of the error produced by the line, and the line is computed so as to minimize that total error.
Once the constants have been computed, it is a trivial matter to use the MLR model as a predictor. You simply plug the known values of the predictor variables into the formula to compute the predicted Y-value. In the car buying example, X1 is the age of a potential buyer, and so you multiply that by the C1 constant, then add C0 to obtain the Y-value, which is the predicted value of the car.
In this way, hopefully you can see that the MLR "learns" the values of the constants like C1 and C0 from the given data points. Furthermore, the actual algorithm that produces the machine intelligence only computes the result of a simple linear equation, so hopefully you can also see that the predictive power comes mainly from the constants, which were "learned" from the data. In the case of the minimax method, most of the machine intelligence came from the algorithm, but with MLR-- as with most machine learning-- the machine intelligence is for the most part an emergent property of the training data.
Lastly, it's worth noting that there are a lot of "best practices" around using MLR. However, these are orthogonal to topic of this post. Suffice it to say that just like the minimax method has a very limited domain in which it is effective as a machine intelligence, MLR also has a limited domain. For example, the predictor variables (the X's) do need to be linearly related to the dependent variable in reality. However, within the limited domain of its linearly related data, MLR is quite effective and an excellent example of a simple machine learning technique that produces machine intelligence within that domain.
Just ran across a good developerWorks article on Integrating Lotus Forms with Lotus Domino.
Domino is a web application and web services platform that is often used in combination with the Lotus Notes rich client platform. However, as this article shows, it is possible for Domino to have broader reach out to all web browsers without a large client-side installation. The intelligence and interactivity of XForms are combined with the high precision presentation layer of XFDL to describe a rich client experience, but the Lotus WebForm Server is used to convert that to HTML and AJAX that is natively understood by the web browser. The net result is that XML data processing and web services from Domino servers are extended right out to the webtop.
Of course, if you do have the Lotus Notes client platform installed, then the story gets better because the Notes replication capabilities can be brought to bear for when a user needs to work in offline/disconnected mode. You can also use the Notes Composite Application Framework to create mashups involving Lotus Forms and other application components deployed to the Notes client. Whether you use the Lotus Forms Viewer or the Lotus Forms WebForm Server to render a Lotus Form in Notes application component, you have access to a running Lotus Form using an API. This gives you getters and setters, of course, so you can push data from other components into the Lotus Form, but you can also set up event listeners that are notified of changes made within the Lotus Form, so you can push changes from the Lotus Form to any other component in your mashup.
But net-net, if you want to extend your Domino applications and services beyond the usual enterprise IT boundaries and get access to new B2C and G2C market opportunities, using Lotus Forms is a way to do it. See the new article for technical details.
Forms exist to collect data from web users involved in business processes. Are you a business partner who wants to build solutions more quickly in order to make a higher margin? Then read on!
What if you could use a web browser to design not only the user interfaces of the multiple pages of a form, but also the whole solution for which it collects data?
Now, with IBM Forms Experience Builder, you finally can.
You can define the roles of users in the business process, and you can assign users and groups to those roles. You can even set up open roles whose users are defined dynamically during the business process once the right information is collected earlier in the process. For example, only once you take in a person's name can you access an LDAP service to look up his manager and then assign that person to the manager role for an approval step.
You can define the user interface of a Form, and have an automatic database created on the server side to store database records corresponding to completed instances of that Form. You can even define multiple Forms that work together within a solution that collects data according to different record schemas.
You can define the stages of a business process workflow that uses the Form or Forms to create and update database records. Stage transitions can branch forward, backward or even stay on the same stage to update a database record that still needs more work.
You can define access control for each workflow stage and determine which Forms, page, and UI elements are available in each stage.
You can even use the database records collected with one Form as a GUI configurable web service within the fill experience of a second Form. For example, you could have one Form of a solution that collects inventory data, and then use that data in a second Form that makes it possible to order from available inventory.
You can make the Form fill experience available within a portlet of an IBM Websphere Portal website.
You can create a solution with your web browser, you can save it to the server, you can hit Deploy in your web browser, and then your users can access the Forms of the solution from web links. If you later decide it is necessarily to add to or change the solution, you can edit the solution again using your web browser and hit Deploy again The data is retained for all the remaining form UI elements, and the database tables are altered as needed to make space to store data collected by any new form UI elements.
Via web links, users can access the list of database records collected by the solution. Only the records to which the user has access are presented. If you're the solution creator/administrator, you can get access to all the records. Whomever is given a link to view the records can also set up their own customized filters for the data, so a user can truly use the view as a business process task list, and even filter down to tasks of a particular type, from a particular person, having met or exceeded some value, etc.
Complete agile web solution creation.
Lose the custom coding, gain the market advantage, get IBM Forms Experience Builder now.
The XML Schema language is a good language for capturing the syntactic constraints of an XML vocabulary. But let's face it, it is really designed more for describing data. It's just not powerful enough to be used for making normative contributions to W3C Recommendations like, say, XForms.
This is not to say that a disagreement between the recommendation and the schema won't sometimes be resolved in favor of what the schema says, but there is a reasonably popular view that the schema is also normative. Actually, the recommendation is normative. In fact, the recommendation is typically provided in two or three formats (sliced up version, single HTML file, diff marked version), and only one of those is considered normative. Even more to the point, English is, for better or worse, the only language in which the normative recommendation is expressed. Any other versions are also considered to be informative.
Quite apart from what anyone tells you about the normativeness of a schema, you can determine that the schema associated with a recommendation is informative, not normative, if it does not appear in TR space. This is a subdirectory of the W3 website named 'tr' and used to publish all technical recommendations of the W3C. In the case of XForms, the schema lives in the working group space, not TR space, so it is informative.
However, the location of the schema isn't really the deciding factor for me. Even if a schema did appear in TR space, it still would only be informative in my opinion because there are lots of language constraints that you just cannot express in schema but which are expressed in the recommendation. A number of great examples of this can be found in XForms, two of which are explained below.
Perhaps the easiest is the use of XPath expressions in XForms. In the XForms schema, a number of the attributes like
ref have values that must be XPath expressions. The schema declaration for those attributes declares a type of
xforms:XPathExpression. But here is what the XForms schema defines for the XPathExpression type:
<xsd:simpleType name="XPathExpression"> <xsd:restriction base="xsd:string"/> </xsd:simpleType>
Shock of shocks, it's a no-restriction restriction from the base type of
xsd:string. You can just see two spec writers hands getting tossed way up in the air on this one and being caught, if only tentatively, by wrists not anxious to end up with carpal tunnel syndrome from the attempt to express the XPath BNF rules as an XML schema restriction. (Those with formal language training can groan a little louder than the rest).
The point is that schema just can't touch this. But, the rules of XPath are clearly and normatively defined in the XPath recommendation, and the XForms recommendation dutifully cites XPath in its list of normative references. That's why the recommendation text is normative and the schema isn't.
Not to put too fine a point on this, but you might claim that the XPath example is a perverse one. Well, it's not the only one. For example, a number of XForms elements like
select1 have a required single node binding. For the two elements I just mentioned, the single node binding attaches the user interface control to a node of instance data so that user input can be placed into the data model of the form. A single node binding can be expressed using either a
ref attribute or a
bind attribute. Unfortunately, XML schema doesn't have a way to express the requirement that one of two attributes must appear. So, in the schema for XForms, we say that both
bind are optional even though that's not very accurate. Separately, each is optional, but that's only the half of the story that XML schema can tell. The full story appears in the normative text of the XForms Recommendation.
A Lotus Form is a document currently expressed in an XML vocabulary called XFDL (html version, pdf version).
This is significant enough, in and of itself, to be set off in a paragraph. XML is a de facto industry standard from the W3C, and this means that widely deployed, interoperable, industry standard software toolkits can be used to introspect and manipulate the content of XFDL, i.e. Lotus Forms documents, thus mitigating issues of vendor lock-in.
But the story gets better...
The XFDL format internally employs W3C standard XForms, so without further reference to any vendor-specific documents, the standard indicates where in the XFDL document to look for the data content created by an end-user who fills in the Lotus Forms document. Anyone with access to a Java reference manual could write code to prepopulate data into an XFDL (Lotus Forms) document before it is delivered to an end-user, and they can write code to extract data from the document when it is returned to the server for processing.
But the story gets even better...
XFDL incorporates the W3C XML Signatures standard, so widely available industry standard tools are available from multiple vendors for validating the security of digital signatures in XFDL documents. In addition to the Apache XML Security implementation, it is notable that Java itself now natively contains support for XML Signatures (JSR 105).
These standards mean that the entire server-side lifecycle of the Lotus Form can be achieved without being locked into using any particular vendor's API. Any application server, any portal server, any server-side environment that can receive HTTP POSTs and process XML in the POST data can be used in combination with Lotus Forms.
And if you ever hear a vendor try to play up high availability of a client-side browser plugin for processing their favorite file format, ask them "Plugin?? What plugin??" With Lotus Forms, you don't need a browser plugin at all because the Lotus Forms Web Form Server converts the XFDL into dynamic HTML that can be processed directly by the browser with no plugins at all. Best of all, when the user finishes interacting with the Lotus Form, the resulting XFDL document is delivered to the application server endpoint, where it can be processed as XML using the standard APIs.
Despite all the excitement about the possibilities that HTML5 offers for a better world wide web, I believe there is a critical flaw in the go-forward plan of those who are feeling the momentum.The problem is that they still haven't got ~80% of the web browser makers on board!By which I mean they haven't got you-know-who.
There's really only one way to break the loggerjam and move forward with advancing thestate of the web. We need to get you-know-who out of the way of progress by showingthat we can innovate on the web with or without their participation. We have now shown that this is feasible by way of the ubiquity strategy based on our abilityto deliver an interactive web application language in multiple web browsers with no client-side plugins and no server-side processing of the language. Click here to visit the project.
For all web advancement technologies, including HTML5, the longer term effect of the ubiquity strategy should be that the late bloomers are shamed into implementing the advancements, thereby obviating the need for the ubiquity code. But meanwhile the ubiquity strategy is the way to enable web advancement technologies including HTML5 to emerge (in the proactive verb sense of the word).
The annual World Wide Web Conference is being held next week in Banff, Canada. It's about an hour's flight from Victoria, where I live, so of course I'm going!
I served on the program committee of the XML Web Data track this year. Competition to get into the conference is quite stiff, so recommending a paper for acceptance is a low probability event. Of the nine papers I reviewed, most were really good, and a couple that I had hoped would make it to the program will indeed appear. I am pleased to be chairing a session on Thursday May 10 that includes one such paper (I did not receive the other two for review).
I will also be presenting in the W3C Track on Friday at 1:30 as part of an Architectural Integration session organized and chaired by Steven Pemberton. I will be presenting on The W3C Rich Web Application Backplane, a forward-looking view of the possibilities for integration and composition of web applications leveraging W3C formats and APIs.
Finally, I'll be presenting in the Dev Track on Saturday at 1:30. My talk will demonstrate a schema-initiated drag-and-drop design experience for XForms using the Lotus Forms Designer.
Hope to see you there!