Comment lines: Scott Johnson: Lazily loading your Dojo Dijit tree widget can improve performance

Populating a tree widget's nodes lazily, rather than all up front, will render the tree more quickly and enable it to perform better. This real-world example shows how you can use REST calls to lazily load JSON data for populating a Dojo Dijit tree widget. This content is part of the IBM WebSphere Developer Technical Journal.

Scott Johnson (scottjoh@us.ibm.com), Software Engineer, WebSphere Business Monitor, EMC

Author photoScott Johnson has been a software developer for 26 years. Before joining IBM he worked in the areas of banking, cash management, survey research, and healthcare scheduling and staffing. He joined IBM in 2000 as the JavaServer Pages component lead for WebSphere Application Server. He currently uses Java, Javascript, Dojo and other technologies to develop widgets for WebSphere Business Monitor.


developerWorks Contributing author
        level

14 May 2008

Also available in Japanese

Lazy can be faster

Using REST calls from the browser client to retrieve data for a Dojo tree can be expensive in terms of performance. Populating a tree widget's nodes lazily will enable the tree to perform better than if all its nodes are populated up front: the tree will render more quickly. Only the nodes that are expanded will ever require REST calls to populate them.

This article provides a sample of a Dojo data store that uses REST calls to lazily load JSON data for populating a Dojo Dijit tree widget. The sample uses more complex JSON data structures in its REST responses than you're likely to find in other samples for Dojo data stores. The REST responses do not map directly into tree node structures, and in that regard are more "real world" than other samples. This sample also works equally well on both Dojo 1.0 and Dojo 1.1. Look at the function _createStoreAndTree() in AccountTreeWidget.js to see how this was accomplished.

Working with the sample

The tree is shown in Figures 1 and 2.

Figure 1. The tree's two top-level nodes
Figure 1. The tree's two top-level nodes
Figure 2. The tree with some nodes expanded
Figure 2. The tree with some nodes expanded

Files and data

The files that make up this sample are:

  • AccountTreeDemo.html: Contains the node to which the Dijit tree is attached (_treeWidgetAttach), and bootstraps AccountTreeWidget. Launch this file in your browser to see the demo.
  • AccountTreeWidget.js: Provides the glue for connecting the AccountTreeStore to a Dijit tree. The store's initial data structure is created by AccountTreeStoreUtil, which makes multiple REST calls in order to create that structure. Only when AccountTreeStoreUtil has finished creating that JSON data structure can AccountTreeWidget create the store and tree.
  • AccountTreeStoreUtil.js: Makes multiple REST calls in order to create the initial store structure.
  • AccountTreeStore.js: The custom data store. This hands data to the tree and lazily loads data as nodes are expanded.
  • Accounts/Accounts.json: A directory directly beneath the other files. Accounts.json contains a JSON structure describing Accounts and nested child Accounts.
  • Transactions/*.json: A directory on the same level as Accounts. It contains multiple files that contain JSON structures describing arrays of transactions.

Install the files and directories in a directory called "ibmtreedemo," at the same level as Dijit, Dojo, and DojoX. In other words, situate the demo so it exists in the Dojo directory structure as a peer to the Dojo directories.

The data for the sample consists of accounts and transactions for the fictional US National Bank. To populate the tree, two distinctly different JSON structures (accounts and transactions) must be parsed and combined.

  • Account data: The Account data is retrieved by a single REST call that returns the complex JSON structure shown in Listing 1. The accounts data consists of top-level accounts with possibly deeply nested child accounts. Joe Smith's accounts make up the first half of the file, and Mary Bradley's accounts are shown in the second half.
    Listing 1. Account data
    {bankId: "NationalBank", bankName:"NC National Bank", location: "US_NC", 
    "accountArray": [
        {accountName: "Joe Smith's US National Bank Account", 
            accountId: "JoeSmith_US_NC", dataType: "Account",
            childAccounts: 
                [{accountName: "Joe Smith's Bahamas Account", 
                  accountId: "JoeSmith_BS",  dataType: "Account",
                  childAccounts: 
                      [{accountName: "Joe Smith's Turks and Caicos Account", 
                        accountId: "JoeSmith_TC",  dataType: "Account", 
                        childAccounts: []}]},
                 {accountName: "Joe Smith's Swiss Account", 
                  accountId: "JoeSmith_SZ",  dataType: "Account",
                  childAccounts: []}]},
        {accountName: "Mary Bradley's US National Bank Account", 
            accountId: "MaryBradley_US_NC", dataType: "Account",
            childAccounts: 
                [{accountName: "Mary Bradley's Bahamas Account", 
                  accountId: "MaryBradley_BS",  dataType: "Account",
                  childAccounts: 
                      [{accountName: "Mary Bradley's Turks and Caicos Account", 
                        accountId: "MaryBradley_TC",  dataType: "Account", 
                        childAccounts: []}]}]}
    ]}
  • Transaction data: Each account's transaction data is found in the file whose name identifies the account holder and account ID. Listing 2 shows a shorterned version of Joe Smith's Swiss account transactions from the file Transactions/JoeSmith_SZ.json. The transactions are inside the array transactionArray.
    Listing 2. Transaction data
    {bankId:"NationalBank",location:"US_NC",accountId:"JoeSmith_SZ",
    transactionArray:[
    {transactionAmount:"4050.00",  
       dataType: "Transaction", 
       currency:"USD",
       transactionDate:"2006/10/24"},
    {transactionAmount:"2050.00",  
       dataType: "Transaction", 
       currency:"EUR",
       transactionDate:"2006/10/25"},
    {transactionAmount:"404.00",  
       dataType: "Transaction", 
       currency:"GBP",
       transactionDate:"2006/10/26"},
    {transactionAmount:"98300.00",  
       dataType: "Transaction", 
       currency:"EUR",
       transactionDate:"2006/10/27"}
    ]}

Initial data structure for the data store

The data store is called AccountTreeStore, and is a custom store that extends dojo.data.ItemFileReadStore.

ItemFileReadStore can be created with either a "url" parameter or a "data" parameter. For this sample, the url parameter can't be used because the initial structure must be created from multiple URLs, and the data must be parsed before it can be used to populate the tree. Therefore, the data parameter is used to create the store.

The initial data structure for the store is created dynamically by the widget called AccountTreeStoreUtil. A shortened version of this data structure is shown in Listing 3.

Listing 3. Initial data structure for the data store
{
identifier: 'name',  
label: 'label' 
items: [
{ name:"Joe Smith's US National Bank Account", 
  type:'topLevelAccount', 
  accountId: 'JoeSmith_US_NC', 
  label:"Joe Smith's US National Bank Account", 
      children:[
          {_reference:"Joe Smith's Bahamas Account", 
            accountId:"JoeSmith_BS", 
            label:"Joe Smith's Bahamas Account", 
            parent: "JoeSmith_US_NC", type: "childAccount"}, 
          {_reference:"Joe Smith's Swiss Account", 
            accountId:"JoeSmith_SZ", 
            label:"Joe Smith's Swiss Account", 
            parent: "JoeSmith_US_NC", type: "childAccount"},
          {_reference:"JoeSmith_US_NC1", 
            parent: "JoeSmith_US_NC", 
            type: "transaction"},
          {_reference:"JoeSmith_US_NC2", 
            parent: "JoeSmith_US_NC", 
            type: "transaction"}]},
  { type: 'stub', 
        name: "Joe Smith's Bahamas Account",
        accountId:"JoeSmith_BS", 
        dataType:"Account", 
        label: "Joe Smith's Bahamas Account", 
        parent:"JoeSmith_US_NC"},
  { type: 'stub', 
        name: "Joe Smith's Swiss Account",
        accountId:"JoeSmith_SZ", 
        dataType:"Account", 
        label: "Joe Smith's Swiss Account", 
        parent:"JoeSmith_US_NC"},
  { type: 'stub', 
        name:"JoeSmith_US_NC1", 
        label:"2006/10/24 USD 4050.00", 
        dataType: "Transaction", 
        parent:"JoeSmith_US_NC", 
        transactionAmount: "4050.00", 
        currency: "USD", 
        transactionDate:"2006/10/24"},
  { type: 'stub', 
        name:"JoeSmith_US_NC2", 
        label:"2006/10/25 EUR 2050.00", 
        dataType: "Transaction", 
        parent:"JoeSmith_US_NC", 
        transactionAmount: "2050.00", 
        currency: "EUR", 
        transactionDate:"2006/10/25"},
// end of top-level item  -   name:"Joe Smith's US National Bank Account
    
//  Mary Bradley's account data would follow the same pattern as Joe Smith's.

  { name:"Mary Bradley's US National Bank Account", 
    type:'topLevelAccount', dataType: 'Account',........................}
    // end of top-level item  -   name:"Mary Bradley's US National Bank Account
]};

This structure only resides in memory; it is never persisted to disk.

The bold elements in the structure above are the keys to making lazy loading work in this sample:

  • type:'topLevelAccount'
    'topLevelAccount' matches the query with which the Dijit Tree is created in AccountTreeWidget.js: query:{type:'topLevelAccount'}. This tells the tree that items of type topLevelAccount are the top-level nodes of the tree. Any other type will be subnodes. You can see in Figure 1 that only Joe's and Mary's US National accounts are top nodes. This is because only those accounts are identified as type:'topLevelAccount' in the initial structure for the data store (Listing 3).

  • _reference:
    In the children array for top-level accounts, each child is identified as a _reference. The value of the _reference, such as "Joe Smith's Bahamas Account," matches up to a name element in a stub structure.

  • type: 'stub'
    name: 'somevalue'

    The _references in the top-level accounts are defined in structures identified as type: 'stub'. The name value in a stub structure matches one _reference value in the children array. For example, _reference:"JoeSmith_US_NC1" in Joe Smith's account's children array matches up to name:"JoeSmith_US_NC1" in one of the stub structures.

Once AccountTreeStoreUtil has created this initial structure, the store and tree can be created. AccountTreeWidget creates the store and tree, as shown in Listing 4.

Listing 4. Creating the data store and the tree
_createStoreAndTree: function(data){
    if (dojo.version.major == 1 && dojo.version.minor == 0) {
        this.store = new ibmtreedemo.AccountTreeStore({
            data: data["treeStructure"], 
            cachedTransactionObjects: data["cachedTransactionObjects"],
            cachedChildAccounts: data["cachedChildAccounts"]});
        this.tree = new dijit.Tree({store: this.store, 
            label: &Accounts and Transactions for US National Bank&, 
            labelAttr: &name&, 
            typeAttr: &type&, 
            id:&accounttree&, 
            query:{type:&topLevelAccount&}});
        
        this.treeAttachPoint.appendChild(this.tree.domNode);
        this.tree.startup();
    } else if (dojo.version.major == 1 && dojo.version.minor >= 1) {
        this.store = new ibmtreedemo.AccountTreeStore({
            data: data["treeStructure"], cachedTransactionObjects: 
            data["cachedTransactionObjects"],
            cachedChildAccounts: data["cachedChildAccounts"]});
        var myModel = new dijit.tree.ForestStoreModel({
            store: this.store,
            query:{type:&topLevelAccount&},
            rootId: "accts",
            rootLabel: &Accounts and Transactions for US National Bank&});                
        this.tree = new dijit.Tree({model: myModel});
        this.treeAttachPoint.appendChild(this.tree.domNode);
        this.tree.startup();
    } else {
        alert("Unsupported dojo version.  dojo 1.0 or greater required");
    }   
}

The data parameter to the store is: data["treeStructure"]. This is the initial data structure described above and was created by AccountTreeStoreUtil.js.

After the store has been loaded with the initial data, and the calls above have executed, the tree is rendered. (The initial data contains all the top-level accounts and their transactions. The store is one step ahead of the tree at all times.) When a tree node is expanded, the store looks at child nodes and only then loads their transactions.

Lazy loading of child nodes in the tree

When a node is expanded, the Tree widget calls the store's isItemLoaded() function. For example, if the two top level nodes (Joe's and Mary's top level bank accounts) are not expanded when the tree is first displayed, then clicking on the expand icon (+) for Joe's top level account will result in the store's isItemLoaded() being called by the tree in order to resolve the stubs in the initial store data structure. These stubs were described earlier.

Remember that the store is always one step ahead of the tree in retrieving data. The store is handed a cache for accounts, so the store never needs to make the accounts' REST query again. More significantly, the store is also handed a cache of transactions for the top-level accounts. This is efficient because the initial data structure for the tree must contain stubs for all children of the top-level accounts, which includes transactions. Therefore, AccountTreeStoreUtil must make a query to get the transactions for both Joe's and Mary's top level accounts. These account objects are then handed to the store in a cache when the store was created. This means that when Joe's US National Bank account node is expanded, its transactions can be retrieved from the cache. The cache is then supplemented with Joe's Bahamas account transactions, even though the Bahamas node hasn't yet been expanded. When the Bahamas node is expanded, the transactions are retrieved from cache, and the cache is supplemented with Joe's Turks and Caicos account transactions. This pattern is repeated until an account has no more child accounts and therefore no more transactions.

The function isItemLoaded() is small and looks like this:

Listing 5. isItemLoaded() function
isItemLoaded: function(item) {
    //  summary:
    //      Overload of the isItemLoaded function to
	 //      look for items of type 'stub', which indicate
    //      the data hasn't been loaded in yet.
    //
    //  item:
    //      The item to examine.
    // For this store, if it has the value of 'stub' for its type attribute,
    // then the item hasn't been fully loaded yet.  It's just a placeholder.
    if (this.getValue(item, "type") == "stub") {
        return false;
    }
    return true;
},

When Joe's top level account node is expanded, the first item that needs to be resolved is Joe's Bahamas account, which was defined as a child of his top-level account, and which had the type 'stub' in the initial data structure. Because the Bahamas account is type 'stub', isItemLoaded() will return false.

The tree calls isItemLoaded() for the other stubs in Joe's top level account; these are his Swiss account, and the transactions for the top level account. These were all listed in the children array for his US bank account in the store's initial data structure. However, none of these will be resolved until the first child, the Bahamas account, is resolved.

Handling account stubs

The store's loadItem() function is then called by the tree in order to resolve the Bahamas account stub. loadItem() now does the following things:

  1. The store's loadItem() function determines that the item needing to be resolved is an account, not a transaction. It calls the store's addAccountNodes() function.
  2. addAccountNodes() does three things:
    1. Creates a new node in memory representing Joe's Bahamas account. This node's type is 'stub.' The complete structure of this node is shown in Listing 6.
    2. Creates an array within the new node for children of the Bahamas account – children can be the Bahamas account's immediate child accounts, and transactions for the Bahamas account. The first children added to the array are immediate child accounts, which are looked up in a cache that the store was given when it was created. (Since the REST query for accounts was a single query, it is more efficient to cache the results.)
    3. Calls the function getTransactionsForAccount() which will perform a REST query to get the transactions for the Bahamas account (remember, the store is staying one step ahead of the tree). getTransactionsForAccount() will add the transactions to the children array of the new Bahamas account node. It will also add these transactions to a transaction cache; when the Bahamas node is expanded, these transactions will be available in the cache.

When the store's getTransactionsForAccount() function is finished getting transactions for the Bahamas account, it calls the store's gotData() function and passes the new Bahamas account node. In memory, this structure looks like Listing 6 below. (Only a sample of all transactions are shown here for space reasons):

Listing 6. Stub structure for an account node, including its transactions and child accounts
{
   "name":["Joe Smith's Bahamas Account"],
   "label":["Joe Smith's Bahamas Account"],
   "type":["stub"],
   "accountId":["JoeSmith_BS"],
   "dataType":["Account"],
   "parent":["JoeSmith_US_NC"],
   "children":[
      {
         "name":"Joe Smith's Turks and Caicos Account",
         "label":"Joe Smith's Turks and Caicos Account",
         "stub":"Joe Smith's Turks and Caicos Account",
         "type":"childAccount",
         "dataType":"Account",
         "accountId":"JoeSmith_TC",
         "parent":["JoeSmith_BS"]
      },
      {
         "stub":"JoeSmith_BS1",
         "type":"transaction",
         "parent":"JoeSmith_BS",
         "dataType":"Transaction"
      },
      {
         "stub":"JoeSmith_BS2",
         "type":"transaction",
         "parent":"JoeSmith_BS",
         "dataType":"Transaction"
      }]}

Note that the account has a type of "stub," but the children have an element called "stub" and their type is either "childAccount" or "transaction." This is important because gotData() looks for an element called 'stub,' and turns the structure into a stub element for later resolution!

Handling transaction objects

When the store's loadItem() function detects that the item is a transaction, it simply looks up the transaction object in the cache, and passes that object to gotData(). The structure is not a stub, since it was cached. It looks like this:

Listing 7. Structure completely resolving an account node
{
   "name":"JoeSmith_US_NC6",
   "label":"2006/10/29 GBP 100000.00",
   "parent":"JoeSmith_US_NC",
   "dataType":"Transaction",
   "currency":"GBP",
   "transactionAmount":"100000.00",
   "transactionDate":"2006/10/29",
   "type":"transaction"
}

What this all means

The key to achieving lazy loading of child account and transaction data, using the approach described in this article, is correctly creating the in-memory JSON data structures that are used to populate the tree. Examine the Javascript code files to see in detail how this sample achieves its goals. AccountTreeStore.js is the workhorse of the sample.

The data store's initial data structure, which is created dynamically from multiple REST calls, describes stubs for children of top level accounts. The initial data structure uses the notation type: 'stub' for child accounts and transactions. The JSON structures that are created at run time when tree nodes are expanded describe stubs in a different way. The notation used in those structures is "stub":"Joe Smith's Turks and Caicos Account." The notation "stub": "value" is a flag for the store's gotData() function. The gotData() function recognizes items using that notation and transforms them into items with type: 'stub' notation, and calls onItem() in order to load these new stub items into the tree.


Download

DescriptionNameSize
Code samplelazyloadtreedemo.zip14 KB

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Select information in your profile (name, country/region, and company) is displayed to the public and will accompany any content you post. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere, Web development
ArticleID=307595
ArticleTitle=Comment lines: Scott Johnson: Lazily loading your Dojo Dijit tree widget can improve performance
publish-date=05142008