Probably all ten of you following this blog know that I invented, designed, and wrote the PageAutomation system, with only small parts written by others. (Well, to be perfectly not-humble, I wrote most of the code for the Generation Engine, too, though I can't claim the invention or design -- those were joint efforts.) The purpose of the PageAutomation system is to tie Pages to Variables, and remove from the developer all the messy details of seeing that the pieces of data flow properly from the server to the browser and back. It also includes the ability to create all the UI, though in reality that was not considered to be part of its original goals.
From the very beginning I knew that the design would have to be a multi-stage approach. First the system should create a semantic model of what it is going to generate, then modifier builders can change that semantic model until it represents what the developer really wants created. The system then tries to match up the Fields and the Groups to existing HTML on the page, and it generates HTML for the ones it couldn't match up. Finally, it invokes Builders for each Field to generate the JSP. (It was Martin's idea to invoke other Builders to generate the actual controls. Brilliant!)
Step 1. Create the initial semantic model.
There are actually a variety of ways that Data Page creates the original model of the data. The most common one -- from a schema -- is actually just a deep clone of the internal representation of a schema because schemas now use the same Java representation. The Public API to the this data is in the interfaces com.bowstreet.webapp.DataDefinitionElement and its descendants DataDefinitionContainer and DataDefinitionLeaf. (Note that this internal Schema representation is what a Rich Data Definition Builder modifies. When you apply an RDD file to a schema, then you are indirectly modifying every Data Page that is based on that schema.)
Once the elements are cloned and the new copies associated with a Page, then they can be accessed through the Java classes PageDataElement, PageDataContainer, and PageDataLeaf. These are the elements that are modified by the PageAutomation modifier Builders like Data Column Modifier, Data Field Settings, etc. The settings control everything from how the HTML for the element will be constructed (if necessary) to validation and formatting information to the text for its label.
This internal representation of the data structure is reflected directly in what you see in the Application Tree when you click on a Group, Table, Field, or Column. Each of those is a single object in the tree. If you switch the Designer to show Source View instead of Design View, you'll see some (extremely useful) info about the element and its children. Below is an example taken from the Create page of the OrdersServiceConsumer example (with a few fields removed for brevity).
Also during this step, all the internal names of the Elements are changed so that they do not conflict with any other element names on the page (even if there is more than one Data Page Builder applied to the page). Also, they are modified to be legal Java names, because they might be used as variables in the _SaveData method that will be generated later.
Step 2. Modifier Builders act upon the semantic model
As mentioned above, these PageDataContainers and PageDataLeafs are the elements that the modifier Builders like Display Manager and HTML Data Layout. Since this is considered to be a public API, you could safely make your own Builder which modified these elements. (There's a little bit of a secret to making a PageAutomation Builder, however. Your Builder class must implement PageAutomationBuilder and your BDEF must declare PageAutomationGenHandler as its GenHandler.)
Step 3. HTML elements are located or generated.
Once the modifier Builders have all run, the tree of PageDataElements might look very different -- even a different arrangement or with some elements set to be skipped over. However, operating on this changed tree, the PageAutomation system will try to match up all the nodes to elements on the page. If it can't match them all, then it will make a depth-first tree walk through the elements, creating HTML for each one. The default PageGenerationAgent finds the right snippet of HTML from the HTML Template file being used, but note that the PageGenerationAgent can be changed at any level. When you apply an HTML Data Layout Builder on an element in the tree, what you are doing is changing the PageGenerationAgent for that element to be one that uses a canned block of HTML when it is your element's turn to generate HTML.
During this step, also, the PageDataElements receive references to all the IXml nodes on the page that they care about. A PageDataContainer cares about where its label goes, its Repeat location (if it repeats), and where its wrapper is. A PageDataLeaf cares about where to put its label and its control and where its overall wrapper is. Plus, if it is a Data Entry Leaf, it cares about where its "Required" indicator and its validation error message goes.
Also during this step the default RDD for the model is applied to each field. This allows you to control anything about a field based only on its type. You can change the default RDD for a single model with a Data Field Settings Builder, or you can change across your entire project with a setting in override.properties.
Step 3a. HTML might be exported.
If the Designer has asked for some or all of the page to be exported, either for an HTML Data Layout Builder or for the Export button in the Data Page Builder, it will be done now.
Step 4. JSP is generated.
The PageAutomation system again walks through the tree and calls the Repeated Region Builder for the repeated elements, and a control Builder and possibly some modifier Builders for every Leaf. (By "modifier Builders," I mean ones added through an RDD, such as Visibility Setter or Attribute Setter, not PageAutomation modifiers.)
Step 5. _SaveData method might be generated.
If the page has any Data Entry elements on it, then an _SaveData method is generated. This is done by, once again, recursively walking the tree of elements and inserting some code for every Data Entry element. This code takes into account its validation operation, whether it is a text node or an attribute in the XML schema, etc. If there is a Post-Save operation defined for the Data Page, then a call to it is added. A reference to the method is them added to the Page Submit event for the form. This is done by adding a hidden field to the page with a magic name that the Factory runtime code knows means it should run the method referenced before it runs the action.