Using HTML5 database and offline capabilities, Part 2: Leveraging the IndexedDB API in HTML5

HTML5 won't be an official World Wide Web Consortium (W3C) standard until 2014, but web browser vendors are already adding and marketing HTML5 features. Two such features — offline application support and local persisted storage — can deliver the same rich user experiences online and offline that were previously available only in proprietary desktop application development frameworks. Building on the foundation that is provided in Part 1, this article explains how to leverage the Indexed Database (IndexedDB) API to build an offline application with locally persisted data.

Brian J Stewart, Principal Consultant, Aqua Data Technologies, Inc.

Photo of Brian StewartBrian J. Stewart is a principal consultant at Aqua Data Technologies, a company that he founded to focus on content management, XML technologies, and enterprise client/server and web systems. He architects and develops enterprise solutions based on the Java EE and Microsoft .NET platforms.



27 November 2012

Also available in Chinese Russian Japanese

"Using HTML5 database and offline capabilities, Part 1" introduced the offline application and local data persistence options available in the HTML5 specification, focusing on localStorage. This installment introduces the Indexed Database (IndexedDB), a robust data persistence technology that is part of the HTML5 standard, and discusses how to integrate an IndexedDB data provider with the Contact Manager application you created in the first article.

The sample Contact Manager application (for names, addresses, and phone numbers) has an online and offline mode. When offline, the data resides in local persisted storage. Upon switching to online mode, a simple data-synchronization function synchronizes local data changes to the server. The application supports the four basic persisted storage functions (create, read, update, and delete [CRUD]) in both online and offline modes.

Overview of the application architecture

The server interface consists of two servlets: ContactServlet and DictionaryServlet. You can find a table showing a summary of the interfaces in Part 1. The implementation of these servlets and the corresponding business services and data providers is not the focus of this article series.

As a recap, first look at Figure 1, which shows the Contact Manager architecture. The server architecture consists of two servlets that correspond to business services and data providers. The UI consists of a single HTML file and four JavaScript modules, with an external reference to the latest version of the jQuery library.

In this article, you replace the offline database provider with one based on the IndexedDB API. Specifically, you replace the localdb.js JavaScript module.

Figure 1. Contact Manager application architecture
Image showing the application architecture, with boxes showing the servlets under client architecture and server architecture

Overview of the data model

The data model consists of two data entities: contact and state (see Figure 2). The contact table contains the actual contact data. The state table contains the dictionary values for the state selection list.

Figure 2. Contact Manager application data model
Image showing the data model

The IndexedDB API

The HTML5 specification contains several persisted storage technologies. IndexedDB is the preferred HTML5 browser database; it replaces the deprecated WebSQL database.

Web browser support and the implementations of the IndexedDB API are not always consistent in web browsers. At the time of this writing, IndexedDB is supported by Google Chrome 11+, Mozilla Firefox 4+, and Windows® Internet Explorer® 10.

A website can contain one or more databases that are identified by a unique name. Each database can contain one or more object stores. An object store is similar to a table in a relational database in that it's identified by a unique name and is a collection of records. However, an object store handles data storage, access, and querying differently from the way that a table in a relational database does. The data within an object store is stored with a key and a value. The key must be unique within an object store and can be specified or generated by a key generator.

The IndexedDB API specification includes support for common database constructs such as transactions, indexing, querying data, and cursors. The specification includes both a synchronous and asynchronous API. The synchronous API is meant to be used within web workers. However, not all web browsers support web workers and the IndexedDB synchronous API. The asynchronous API uses requests and callbacks. All database operations, such as opening a database, retrieving data, querying data, and deleting data, have request API calls. Each request has an onsuccess and onerror callback, which are called when the operation is successful or unsuccessful, respectively. The callbacks provide an event result parameter for returned data.

The sample Contact Manager application consists of a single database with two object stores. The first store is the contacts object store, which contains the actual contact records. The second object store is the states object store, which contains the values for the states selection list. This article demonstrates the use of the asynchronous API for data access.


Connecting to and disconnecting from a database

Because each web browser implements IndexedDB slightly differently, it's best to create a global variable (localDatabase) and initialize it based on the different web browser implementations. This global variable provides a reference to the IndexedDB API.

The next step is to open the database using the open method. If the open database request is successful, the onsuccess callback is invoked. All database operation code should occur within the onsuccess callback. The onerror callback is invoked if an error occurs opening the database. Listing 1 shows the code to initialize and open a database.

Listing 1. Opening a database
var localDatabase = {};

localDatabase.indexedDB = window.indexedDB || window.mozIndexedDB || 
   window.webkitIndexedDB || window.msIndexedDB;
localDatabase.IDBKeyRange = window.IDBKeyRange || window.webkitIDBKeyRange;
localDatabase.IDBTransaction = window.IDBTransaction || window.webkitIDBTransaction;

console.log('opening local database');
var openRequest = localDatabase.indexedDB.open(dbName);
openRequest.onerror = function(e) {
   console.log("Database error: " + e.target.errorCode);
};
openRequest.onsuccess = function(event) {
   console.log("open database request succeeded ");

   console.log('set db');
   db = openRequest.result;

...
};

To disconnect from a database, you simply invoke the close method of the IndexedDB database object.


Creating object stores

The next step is to create the object stores within the database. You can create an object store at any time, but after you create it, you can modify it only within the onupgradeneeded callback on the open database request. This callback indicates that a database upgrade is necessary.

The code in Listing 2 shows how to create the two object stores.

Listing 2. Creating the object stores
openRequest.onupgradeneeded = function (evt) {   
console.log('creating object stores');
var contactsStore = evt.currentTarget.result.createObjectStore
(contactStore, {keyPath: "id"});
var statesStore = evt.currentTarget.result.createObjectStore
(stateStore, {keyPath: "itemId"});
console.log('object stores created');
};

The keyPath identifies the key field for the object store.


Creating the dictionary for selection lists

Records within an object store are created and modified using transactions. The IndexedDB API provides three modes for transactions:

  • readonly — Provides read-only access to the object store.
  • readwrite — Provides read-write access to the object store.
  • versionchange — In addition to read-write access to the object store, provides the ability to create and delete object stores.

You create a transaction using the following syntax:

var transaction = db.transaction("states", "readwrite");

After creating a transaction, the next step is to obtain a reference to an object store. The transaction object contains the objectstores property, which provides access to the object stores associated with the database. To obtain a reference to the states object store, use the following statement:

var store = transaction.objectStore("states");

When you're connected to the server, you want to replace all local states data with the latest server data. To accomplish this, you must clear the states object store before populating it with the latest values from the server. Use the clear method on the object store to do that.

If the object store was cleared successfully, the onsuccess callback is invoked. Within this callback, you can now loop through and add each value to the local states object store. The data that you want to add to the object store is contained in the stateArray string array. Use the jQuery $.each method to iterate through the array. For each value in the states array, call the put method to add the record to the states object store.

The code fragment in Listing 3 shows how to populate the object store with the values in the states array for local and offline access:

Listing 3. Saving state data in the object store
try {
   console.log('saving local state data');

   var openRequest = localDatabase.indexedDB.open(dbName);

   openRequest.onerror = function(e) {
      console.log("Database error: " + e.target.errorCode);
   };

   openRequest.onsuccess = function(event) {
      db = openRequest.result;

      console.log('opening states store');
      var transaction = db.transaction("states", "readwrite");
      var store = transaction.objectStore("states");                    
     
      var clearReq = store.clear();
      clearReq.onsuccess = function (ev) { 
         console.log('cleared state store');
      
         $.each(stateArray, function(i,item){
            var itemId = generateUUID();
   
            var request = store.put({
               "text": item,
               "itemId" : itemId
            });
   
            request.onsuccess = function(e) {
            };
            
            request.onerror = function(e) {
               console.log(e.value);
            };
         });
      };

      db.close();
   };
}
catch(e){
   console.log(e);
}

Now it's time to move on to adding and updating contact records while offline.


Adding and updating a contact

This article uses the same approach to create and update records as that used in Part 1 (when you stored the offline data in localstorage). New records are identified by a uniquely generated negative number for the ID field. The negative number indicates that the record is new and must be created in the server database table. In addition, the isDirty flag indicates that the record was modified or created while offline. Use the put method to save the record in the object store (similar to the way you populated the states object store). Listing 4 shows the full code to create and update a record in an object store:

Listing 4. Creating and updating a contact record
openRequest.onsuccess = function(event) {
   db = openRequest.result;
   
   var transaction = db.transaction(contactStore, "readwrite");
   var objectStore = transaction.objectStore(contactStore);
    
   var id = $('#contactId').val();
   var firstName = $('#firstName').val();
   var lastName = $('#lastName').val();
   var street1 = $('#street1').val();
   var street2 = $('#street2').val();
   var city = $('#city').val();
   var zipCode= $('#zipCode').val();
   var state= $('#state').val();;
   
   if (contactId > 0) {
      var getRequest = objectStore.get(parseInt(contactId));
      
      getRequest.onsuccess = function(event)
      {  
         var contact = event.target.result;

         contact.firstName = firstName;
         contact.lastName = lastName;
         contact.street1 = street1;
         contact.street2 = street2;
         contact.city = city;
         contact.zipCode = zipCode;
         contact.isDirty = true;
         contact.lastModifyDate = "";
         contact.isDeleted = false;
         
         var addRequest = objectStore.put(contact);
         
         addRequest.onsuccess = function(event) {
            recordUpdated=true;
         };
   
         addRequest.onerror = function(e) {
            console.log(e.value);
         };
      };
      
      getRequest.onerror = function(e) {
         console.log(e.value);
      };
   } // if update
   else {
      var newContactId = (-1) * Math.floor(Math.random()*100000);

      var lastModifyDate = "";
      var newContact = {
         "timeStamp": "", 
         "id":newContactId,
         "firstName": firstName,
         "lastName": lastName,
         "street1": street1,
         "street2": street2,
         "city": city,
         "zipCode": zipCode,
         "state": state,
         "isDirty":true,
         "lastModifyDate": "",
         "isDeleted":false };
      
      var request = objectStore.put(newContact);
      
      var nextIndex = data.length;
      data[nextIndex] = newContact;
      recordUpdated=true;
   } // if create

Next, I show how to delete a contact record while offline.


Deleting a contact while offline

When you delete a contact while offline, you don't want to remove the record. Instead, you want to prevent it from being displayed in the offline contact listing and indicate that the record was deleted. You do this by using the isDeleted flag (similar to the method described in Part 1 of this series). The isDirty flag is also set to true to indicate that the record was modified offline. After the record is altered in the object store, the contact listing is refreshed to remove the record from the listing (records with the isDeleted flag set to true are not displayed). The code in Listing 5 shows how to accomplish this task.

Listing 5. Deleting a contact record
try {
   console.log('deleting local contact');
   
   var openRequest = localDatabase.indexedDB.open(dbName);
   console.log('after open');
   openRequest.onerror = function(e) {
      console.log("Database error: " + e.target.errorCode);
   };
   openRequest.onsuccess = function(event) {
      db = openRequest.result;
   
      console.log('opening contacts store');
      var transaction = db.transaction(contactStore, "readwrite");
      var store = transaction.objectStore(contactStore);

      var getRequest = store.get(contactId);  
      getRequest.onsuccess = function (ev) {
         var item = getRequest.result;
         item.isDeleted=true;
         item.isDirty=true;

         var request = store.put(item);

         request.onsuccess = function(e) {
            alert('Contact deleted');
            loadOfflineContacts();
         };

         request.onerror = function(e) {
            console.log(e.value);
         };
      }

      getRequest.onerror = function(e) {
         console.log(e.value);
      };

      db.close();
   };              

   loadOfflineContacts();
}
catch(e){
   console.log(e);
}

Querying contacts

To retrieve the contact records from the contacts object store, you use an IndexedDB cursor. Like database cursors in relational databases, an IndexedDB cursor provides a way to iterate through the records within an object store. While iterating through the records, you build an array containing the contact records. Any record with the isDeleted flag set to true is ignored. Listing 6 shows how to build a contact array with the data contained in the contacts object store.

Listing 6. Querying contacts
var data = new Array();
...
var cursorRequest = objectStore.openCursor();
cursorRequest.onsuccess = function(evt) {  
   var cursor = evt.target.result;  
   if (cursor) {  
      if (!cursor.value.isDeleted) {
         var newContact = {
            "timeStamp":cursor.value.timeStamp,
            "id":cursor.value.id,
            "firstName": cursor.value.firstName,
            "lastName": cursor.value.lastName,
            "street1": cursor.value.street1,
            "street2": cursor.value.street2,
            "city": cursor.value.city,
            "zipCode": cursor.value.zipCode,
            "state": cursor.value.state,
            "lastModifyDate": cursor.value.lastModifyDate,
            "isDeleted": cursor.value.isDeleted,
            "isDirty": cursor.value.isDirty
         };
         //console.log('adding ' + newContact.toString());
         //console.log("adding contact to array: " + data.length);
         data[data.length]= newContact;
      }
   
      cursor.continue();
   } // more records
   else {
      displayContactData(data);    
   } // no more records
};  // open cursor

Next, I demonstrate a simple algorithm and approach to synchronizing offline additions and modifications with the server.


Synchronizing local data with the server

When you're working online, all CRUD operations use the servlets, and the server database is updated instantly. The local (IndexedDB) database is also updated with online database changes to ensure that the latest data is always available online or offline.

When you're offline, all CRUD operations update the data in the IndexedDB database. Upon reconnecting with the server:

  • Any records that were created in the local database are persisted to the server.
  • Any records that were modified in the local database are updated on the server.
  • Any records that were deleted in the local database are deleted on the server.

The code in Listing 7 shows the complete synchronization method. The same online functions that are described in Part 1 are used for create, update, and delete operations.

The first step is to iterate through the records in the contacts object store using a cursor and build an array containing all records that need to be posted to the server. Records that were updated or created locally have the isDirty property set to true. The save operation is identified as new if its unique record ID is negative (that is, not assigned by the MySQL database). Records that are deleted locally are flagged using the isDeleted property.

When you have the array containing all the records that need to be posted to the server, use the jQuery $.each method to iterate through the array and post each change to the server.

Finally, when the data-synchronization method finishes, it refreshes the local contacts object store with the latest data in the server. This includes any changes you (and other users) made while you were offline:

Listing 7. Synchronizing local data with the server
...
cursorRequest.onsuccess = function(evt) {  
var cursor = evt.target.result;  

if (cursor) {  
   var isDirty = cursor.value.isDirty;
   var curId = cursor.value.id;
   console.log(curId + ' isDirty = ' + isDirty);
   if (isDirty) {
      var newContact = {
         "timeStamp":cursor.value.timeStamp,
         "id":curId,
         "firstName": cursor.value.firstName,
         "lastName": cursor.value.lastName,
         "street1": cursor.value.street1,
         "street2": cursor.value.street2,
         "city": cursor.value.city,
         "zipCode": cursor.value.zipCode,
         "state": cursor.value.state,
         "lastModifyDate": cursor.value.lastModifyDate,
         "isDeleted": cursor.value.isDeleted,
         "isDirty": cursor.value.isDirty
      };
      //console.log('adding ' + newContact.toString());
      //console.log("adding contact to array: " + data.length);
      data[data.length]= newContact;
   }

   cursor.continue();
} // more records
else {
   console.log("no more records");

   console.log('number of modified records: ' + data.length);
   var recordsUpdated = 0;
   var recordsCreated = 0;
   var recordsDeleted = 0;
   
   $.each(data, function(i,item){
      console.log("processing record " + item.id);
      if (item.isDeleted) {
         deleteOnlineContact(item.id, true);
         recordsDeleted++;
      }
      else if (item.isDirty && !item.isDeleted) {
         $('input[name="contactId"]')[0].value = item.id;
         $('input[name="firstName"]')[0].value = item.firstName;
         $('input[name="lastName"]')[0].value = item.lastName;
         $('input[name="street1"]')[0].value = item.street1;
         $('input[name="street2"]')[0].value = item.street2;
         $('input[name="city"]')[0].value = item.city;
         $('select[name="state"]')[0].value = item.state;
         $('input[name="zipCode"]')[0].value = item.zipCode;

         var dataString = $("#editContactForm").serialize();
         postEditedContact(dataString, true);
         if (item.id > 0) {
            recordsUpdated++;
         }
         else {
            recordsCreated++;
         }   
      }
   });

   var msg = "Synchronization Summary\n\tRecords Updated: " + recordsUpdated 
      + "\n\tRecords Created: " + recordsCreated 
      + "\n\tRecords Deleted: " + recordsDeleted;
   alert(msg);

Conclusion

This article builds on the foundation that is described in Part 1. It uses the same pattern for online and offline support and for maintaining a consistent user experience. It introduces the IndexedDB API, the data-synchronization algorithm that handles synchronizing the offline creation, deletion, and modification of records. Because the code doesn't have robust error handling and conflict resolution (when the same record is modified locally and on the server by another user), it is not production-ready. However, it does provide a good foundation for future work.

Resources

Learn

Get products and technologies

Discuss

  • The developerWorks community: Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Web development on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Web development
ArticleID=846182
ArticleTitle=Using HTML5 database and offline capabilities, Part 2: Leveraging the IndexedDB API in HTML5
publish-date=11272012