Super-size your webapp: Building highly scalable web apps using dynamic loading and browser-side caching
ChrisLaffra 060000KCEQ Comments (11) Visits (6143)
Whether you write a webapp by hand, or migrate it from an existing green screen application, scalability will be a concern sooner, rather than later. Your application will grow to incorporate different features or concerns. It will get bigger and bigger over time. At some point, loading the application will become noticeably slow, especially over slower networks.
Instead of loading one monolithic application into the browser, it would be more efficient to load modules piecemeal and thereby increase the responsiveness of the browser application. This brings us to the topic of incremental loading of program modules.
Incremental loading is nothing new. Java has had it from day one in the form of class loaders that load classes only when they are needed. More recently, full-fledged component models were added to Java, such as the Eclipse plugin model and the OSGi bundle framework. In each, the intent is to split up an application into smaller sub-components, reuse the components, and compose a running application dynamically at runtime.
Dynamic loading is also used other domains, such as desktop applications, running on platforms such as Windows. Executable applications load modules in the form of DLLs. The executable file gets smaller, and DLLs are only loaded once into main memory and their code segments are reused by multiple applications. The main result is a reduction in memory by sharing executable code across processes.
In a graph the difference is more striking:
This table shows that for dynamically loaded web apps, the initial cost to bring up the application is dramatically smaller. Only 100K + 2K needs to be downloaded to show the first page. For a statically generated web app, we need to load all hundreds or thousands of pages plus the main page, before we get to show any individual page. Each subsequent page is then free in the monolithic case as we already downloaded them all, but by then our customer probable gave up already.
Dynamically loading web apps
2. decouple modules so that dependencies between modules become symbolic, rather than hard-coded
3. figure out how 2 modules can communicate with each other when they actually never met before
EGL Rich UI has the perfect built-in abstraction for modules and it is called the RUIhandler. A RUIHandler is a standalone program that can be run in the browser all by itself. In that case it is parented to the document body. However, a RUIHandler can also be attached to another parent RUIHandler. This is a very natural decomposition of complexity, and almost all Rich UI applications I have seen use this mechanism automatically.
When a RUIHandler is compiled, the EGL build tools use the static type information that is available in EGL to determine the transitive closure of all the components required by this given RUIHandler. This is an automatic process, and the EGL developer need not worry about this process at all. The tooling creates the deploy XML file when compiling the RUIHandler.
When the RUIHandler is deployed, the transitive closure, captured in the deploy XML file, is used to generate an HTML file to host the application. That closure is a perfect candidate for use as a module definition.
This is what a typical deploy.xml file might look like:
It is essential that MainHandler does not directly refer to the type Page1, but instead uses a String referring to the type. Again, this is to avoid the deep closure computation the linker performs when our application is published.
Whenever a RUIHandler is completely loaded and instantiated, the dynamic loader sends an event on the InfoBus, and the "attach" function will be called. In the sample above, we simply add the UI elements in the handler to the current document, thereby activating the newly loaded RUIHandler and enabling it for user interaction.
The InfoBus can also be used between components, of course. One module might publish an InfoBus event to indicate the current selection has changed. Numerous other dynamically loaded modules may subscribe to this very event and update their own UI, generate some trace events, start a timer, or do whatever suits them.
Also notice that the dynamic loader is asynchronous, as we use Ajax calls to load the modules over the network.
Debugging the dynamic loader
Now that we abstracted out the actual loading of modules in an asynchronous manner, we have the perfect hook for easily implementing concerns such as feature coverage, tracing, and debugging information. The EGL Rich UI dynamic loader already comes with a very useful visual tracer that uses the InfoBus events generated by the dynamic loader to provide insight into how the various components are loaded and added to the document. It generates the following information when Page1 is loaded:
When we trigger the loading of Page2, its deploy file is used as shown above:
Browser-side caching is more experimental, and requires the use of not yet widely available HTML5 or unsupported Google Gears. Therefore, it is not included in the attached version of the dynamic loader.