Using HTML5 database and offline capabilities, Part 1: Provide offline data editing and data synchronization

HTML5 will reshape the web experience and line-of-business applications. With the offline capabilities and local persisted storage features, you can deliver the same rich user experiences online and offline that were previously available only in proprietary desktop application development frameworks. In this article, learn how to leverage HTML5 offline capabilities and local persisted storage features. An example application illustrates how to avoid common problems.

Share:

Brian J Stewart, Principal Consultant, Aqua Data Technologies, Inc.

Photo of Brian StewartBrian J. Stewart is a principal consultant at Aqua Data Technologies, a company that he founded to focus on content management, XML technologies, and enterprise client/server and web systems. He architects and develops enterprise solutions based on the Java EE and Microsoft .NET platforms.



17 July 2012

Also available in Chinese Russian Japanese

Introduction

HTML version 5 (HTML5) is not expected to reach World Wide Web Consortium (W3C) Recommendation status until 2014. Though it's not officially a standard, web browser vendors are adding and marketing HTML5 features. HTML5 is already reshaping the web experience for Internet websites and line-of-business (LOB) applications. Many websites, such as the Amazon Kindle Cloud Reader, are already taking advantage of HTML5. Two key HTML5 features will dramatically change LOB applications: offline application support and local persisted storage. Because HTML5 is not an official standard, browser support is inconsistent at best.

In this article, learn about the offline application support and the different persisted storage capabilities provided by the proposed HTML5 standard. An example application helps illustrate the features.


Example application

The example Contact Manager application provides for the management of contact information (names, addresses, and phone numbers). It provides an online mode, an offline mode, and a simple data-synchronization function to synchronize local data changes to the server upon switching to online mode. When offline, the data resides in local persisted storage. The application supports the four basic persisted storage functions—create, read, update, and delete (CRUD)—in both online and offline modes.

Architecture

Figure 1 shows an overview of the Contact Manager application architecture. The server architecture consists of two servlets: business services and data providers. The user interface (UI) consists of a single HTML file, four JavaScript modules, and an external reference to the latest version of the jQuery library.

Figure 1. Application architecture overview
Image showing the application architecture

The data model

The data model consists of two data entities, contact and state, as shown in Figure 2. The contact table contains the actual contact data; the state table contains the dictionary values for the state selection list.

Figure 2. Data model
Image showing the data model

The server interface

The server interface consists of two servlets: ContactServlet and DictionaryServlet. Table 1 summarizes the servlets. (Implementation of the servlets and the corresponding business services and data providers are outside the scope of this article.)

Table 1. Summary of servlets
Servlet nameOperationParametersDescription
DictionaryServletcode>getstatesN/AReturns an array of states in JavaScript Object Notation (JSON) format.
ContactServletgetallcontactsN/AReturns an array of contact records in JSON format.
ContactServletdeletecontactId - ID of the contact to be deleted.Deletes the specified contact record; returns a JSON object with a Boolean flag to indicate whether the operation was successful.
"{"result": true/false"}
ContactServletsave
  • contactId - ID of the contact to be saved. (If greater than zero, it's an update operation.)
  • firstName - Value of the first name field.
  • lastName - Value of the last name field.
  • street1 - Value of the street 1 field.
  • street2 - Value of the street 2 field.
  • city - Value of the city field.
  • state - Value of the state field.
  • zipCode - Value of the zip code field.
Returns a JSON object with a Boolean flag to indicate whether the operation was successful and the new or updated contact ID.
"{"contactId": <id>, "result": <true/false>"}

Calling the server interface

The code in Listing 1 shows how to make the asynchronous call to the contact servlet to retrieve the contacts stored in the online database. The code uses the jQuery getJSON function to invoke the contact servlet.

Listing 1. Retrieving contacts from the server
function loadOnlineContacts() 
{
	$('#contactList').empty();
	$('#contactList').append('Loading contact data...');
	
	var url = '/html5app/contact?operation=getallcontacts';
	
	$.getJSON(url, function(data) {
		saveOfflineContactData(data);
		displayContactData(data);		
	});
}

The code in Listing 2 shows how to save a new or updated contact to the server. It uses the jQuery ajax function. The code sends the data to the contact servlet using HTTP POST.

Listing 2. Saving contacts to the server
function postEditedContact(dataString) {
	postEditedContact(dataString, false);	
}

function postEditedContact(dataString, suppressAlert) {
var contactId = $('input[name="contactId"]')[0].value;

$.ajax({  
type: "POST",  
url: "/html5app/contact",  
data: dataString,  
cache: false,
dataType: "json",
success: function(data) {
	var result = data.result;
	
	if (result) 
	{
		if (contactId > 0)
		{
		if (!suppressAlert) {
		alert("Contact was successfully updated.");
		}
		var lastModifyDate = data.lastModifyDate;
		$('input[name="lastModifyDate"]')[0].value = lastModifyDate;
		}
		else 
		{
		if (!suppressAlert) {
			alert("Contact was successfully created.");
		}
		
		var lastModifyDate = data.lastModifyDate;
		$('input[name="lastModifyDate"]')[0].value = lastModifyDate;
		$('input[name="contactId"]')[0].value = data.contactId;
		}
		
		loadOnlineContacts();

		hideEditForm();
	}
	else 
	{
		alert('An error occurred saving contact ' + contactId + '.');
	}
}
});  
}

The last function is to delete a record from the online database. Listing 3 shows how to delete a record from the server. The code uses the jQuery getJSON function to invoke the contact servlet.

Listing 3. Deleting a contact on the server
function deleteOnlineContact(contactId, suppressAlert){
	var url = '/html5app/contact?operation=delete&contactId=' + contactId;
	
	$.getJSON(url, function(data) {
		var result = data.result;
		if (result) {
			if (!suppressAlert) {
				alert('Contact deleted');
			}
			loadOnlineContacts();
		}
		else {
			alert('Contact ' + contactId + 'not deleted');
		}
	});		
}

Building the local data provider

The local data provider locally persists all selection list and contact data. The HTML5 specification contains a few options for persisted storage. Selecting which technology to use depends on your data storage and browser support requirements. The following sections discuss the three persisted storage technologies as well as an implementation of a local data provider using the persisted storage technology that all major web browsers support.

Three persisted storage technologies are associated with HTML5:

  • localStorage - localStorage provides for the simple storing of data using a flat key-value store. All major web browsers, including Apple® Safari®, Google Chrome™, Microsoft® Windows® Internet Explorer®, Mozilla® Firefox®, and Opera™, support localStorage. HTML5 localStorage is synchronous and is currently the only database storage mechanism that is supported cross-platform and cross-browser.
  • WebSQL - WebSQL was originally aimed at bringing a Transact-SQL-based database to the web browser. The learning curve is low based on its similarity to common relational databases such as IBM® DB2®, Microsoft SQL Server®, Oracle® MySQL® Server, and Oracle Database. WebSQL is supported by a few browsers, including Safari, Chrome, and Opera. It is not supported by Firefox or Internet Explorer. It will presumably be phased out because the WebSQL proposed specification is no longer being developed.
  • Indexed database (Indexed DB) - Indexed DB is an indexed hierarchical key-value store similar to many commercial cloud data storage offerings. WebSQL was dropped in favor of Indexed DB, and Indexed DB is currently supported by Firefox, Chrome, and, in the future, Internet Explorer 10. The application programming interface (API) for Indexed DB is asynchronous and supports indexing, querying, and transactions.

The example solution in this article leverages JSON and localStorage for persisted storage primarily because of the wide browser support for localStorage.

Local data provider

The localStorage approach persists the contact and dictionary data by serializing it as a JSON string and saving the string to localStorage. When retrieving the data, it is deserialized as an array of JSON objects and processed accordingly.

Locally saving data

Listing 4 shows how to save the contact data to localStorage. The JavaScript function JSON.stringify is used to serialize the JSON data that the server returns to a string so it can be stored in localStorage.

Listing 4. Saving data to localStorage
// fetch data from server
...
			
// convert JSON data to a string
var contactDataString = JSON.stringify(data);

// persist contact data to localstorage
localStorage.setItem("contactData", contactDataString);

Locally retrieving data

Listing 5 shows how to retrieve data from localStorage. The first step is to get the JSON string from localStorage. The string is then converted to JSON objects using the JavaScript function eval, which deserializes the string to an array of JSON objects. The data is displayed using the custom JavaScript function displayContactData.

Listing 5. Reading data from localStorage
function loadOfflineContacts() 
{
	$('#contactList').empty();
	$('#contactList').append('Loading contact data...');
	
	var dataStr = localStorage.getItem("contactData");
	var data = eval('(' + dataStr + ')');
	
	displayContactData(data);
}

Locally deleting a record

Listing 6 shows how to delete a record from localStorage.

Listing 6. Deleting a record from localStorage
function deleteOfflineContact(contactId) {
	var dataStr = localStorage.getItem("contactData");
	var data = eval('(' + dataStr + ')');
	
	var recordUpdated = false;
	$.each(data, function(i,item){
		if (item.id == contactId) {
			item.isDeleted=true;
			recordUpdated = true;
			return false;
		}	
	});
	
	if (recordUpdated) {
		dataStr = JSON.stringify(data);
		localStorage.setItem("contactData", dataStr);
		alert("Contact was successfully deleted.");
		
		loadOfflineContacts();
	}
}

The code:

  • Reads the database from local storage and deserializes it.
  • Iterates through the records until the contactId record is found.
  • Sets the isDeleted flag to true.
  • Uses the isDeleted flag in the data synchronization function. (See the "Data synchronization" section.)
  • Persists the data to localStorage, and the data grid is refreshed.

Locally updating and creating records

Listing 7 shows how to update or create a record in localStorage.

Listing 7. Updating a record in localStorage
function updateLocalContact() {
	var dataStr = localStorage.getItem("contactData");
	var data = eval('(' + dataStr + ')');
	
	var contactId = $('input[name="contactId"]')[0].value;
	
	var recordUpdated = false;
	if (contactId > 0) {
		$.each(data, function(i,item){
			if (item.id == contactId) {
				item.isDirty=true;
				item.firstName = $('input[name="firstName"]')[0].value;
				item.lastName = $('input[name="lastName"]')[0].value;
				item.street1 = $('input[name="street1"]')[0].value;
				item.street2 = $('input[name="street2"]')[0].value;
				item.city = $('input[name="city"]')[0].value;
				item.state = $('select[name="state"]')[0].value;
				item.zipCode = $('input[name="zipCode"]')[0].value;	
				recordUpdated = true;
				return false;
			}	
		});
	}
	else {		
		var newContactId = 0;
		var nextId = 0;
		while(newContactId == 0) {
			var found = false;

			nextId = nextId - 1;
			$.each(data, function(i,item){
				if (item.id == nextId) {
					found = true;
					return false;
				}
			});
			if (!found) {
				newContactId = nextId;
			}
		}
		var lastModifyDate = "";
		var newContact = {"street2": $('input[name="street2"]')[0].value,
				"id":newContactId,
				"street1":$('input[name="street1"]')[0].value,
				"lastName":$('input[name="lastName"]')[0].value,
				"isDirty":true,
				"zipCode":$('input[name="zipCode"]')[0].value,
				"state":$('select[name="state"]')[0].value,
				"lastModifyDate": lastModifyDate,
				"isDeleted":false,
				"firstName":$('input[name="firstName"]')[0].value,
				"city":$('input[name="city"]')[0].value};
		var nextIndex = data.length;
		data[nextIndex] = newContact;
		recordUpdated=true;
	} 
	
	if (recordUpdated) {
		dataStr = JSON.stringify(data);
		localStorage.setItem("contactData", dataStr);
		alert("Contact was successfully updated.");
	}
	
	hideEditForm();
}

As shown in Listing 7, you:

  • Read the database from localStorage and deserialize it.
  • If the contactId of the record to be saved is not zero (an update occurred), iterate through the records until the contactId record is found. It is then updated accordingly.
  • Alternatively, if the record is new (contactId is not zero), find the next unused negative contactId.
  • Assign it to the new record.
  • Append the new record to the database.

The data is then serialized as a JSON string and saved to localStorage. A valid (greater than zero) contactId will be assigned during server synchronization. The negative ID is a temporary ID used to identify the record as new.

It's important to know that localStorage:

  • Is limited to 5MB. (Indexed DB should be used when more data storage is required.)
  • Is supported by all major web browsers.
  • Only works with string values.

The next step is to build the UI using HTML5.


Building the UI using HTML5

The example Contact Manager application has a simple UI with a single page. It supports edit and delete for each record and provides the ability to create new records. Cascading style sheets (CSS) and dynamic HTML (through jQuery) are used to hide and show the create/edit subforms as necessary.

To provide a consistent user experience, the same page is used whether online or offline; the only difference is which data provider is called when an operation is performed. Figure 3 shows the application.

Figure 3. Contact Manager application
Image showing the Contact Manager application

JavaScript modules

The application consists of four custom JavaScript modules:

  • core.js provides common JavaScript functions and is used by the other modules.
  • formEvents.js provides the form and button event handlers. It dispatches database operations to the correct data provider based on the online or offline status.
  • onlinedb.js provides the functions for communicating with the server when online.
  • offlinedb.js provides local data storage functions.

All modules also use the latest version of the jQuery library for traversing data, making asynchronous web requests, and dynamic HTML. The client communicates with the server using JSON.

Offline application manifest

HTML5 offline capabilities provide for the caching of static files and resources. The offline application manifest file (.appcache) is the key file for enabling offline application support for a web application. The manifest file defines the following information:

  • Which resources and pages are available while offline.
  • Which resources are only available online.
  • The fallback page displayed for resources that aren't available when offline.

The manifest file consists of three sections: CACHE, NETWORK, and FALLBACK. The pages and resources under CACHE are cached locally. The pages and resources under NETWORK are never cached and are only available when online. The page specified in FALLBACK is displayed if the requested page is unavailable offline. The asterisk (*) in the NETWORK section ensures that all other pages and servlets are only available when online. If the * is missing, the servlet calls will fail (even online). Listing 8 shows the manifest file for the Contact Manager.

Listing 8. Offline application manifest
CACHE MANIFEST
# Revision 1
CACHE:
default.html
list.html
scripts/core.js
scripts/localdb.js
scripts/onlinedb.js
scripts/formEvents.js
http://code.jquery.com/jquery-1.7.2.min.js
NETWORK:
*
FALLBACK:
/ offline.html

When working with offline applications, it's important to know:

  • The offline application manifest file extension .appcache must be mapped to the text/cache-manifest Multipurpose Internet Mail Extension (MIME) type. In Apache Tomcat, do this by adding the mime-mapping entry to the server's web.xml file (not the web application's web.xml file). Most browsers silently ignore the offline application manifest if the MIME type is incorrect.
  • If an offline application manifest file is present, the locally cached resource is always used (even if online).
  • A local resource is only updated when the offline application manifest file changes, typically by changing a revision number in a comment within the manifest file. Changes to HTML or CSS resources are not reflected in the web browser until the application manifest file is changed.
  • Any page that supports offline usage must have the following:
    <html lang="en" manifest="app.appcache">

Online or offline

With JavaScript, you can detect whether the application is online or offline by using the navigator.onLine Boolean. It returns True if the application is online.

Form events (online/offline handling)

In the Contact Manager, the same form is used whether online or offline. The key to making this solution work is in the button and form event handlers. Check navigator.onLine to determine which operation to invoke (local or online). Listing 9 shows an example for the loading of contact data.

Listing 9. Loading data (in the onLoad event of the HTML BODY)
if (navigator.onLine) 
{	
	// selection list needs to be populated prior to synchronizing data
	// the list is updated from the online dictionary later
	populateOfflineStates(); 
	
	setStatusText("Synchronizing contact data with server...");
	synchronizeContacts();

	setStatusText("Loading dictionary data from server...");		
	populateOnlineStates();
	
	setStatusText("Loading contact data from server...");
	loadOnlineContacts();
}
else 
{
	alert('You are currently offline.');
	populateOfflineStates();
	setStatusText("Loading contact data from local storage...");
	loadOfflineContacts();
}

Data synchronization

When online, all CRUD operations use the servlets for the create, modify, and delete operations. The local cache is also updated when the online database changes.

When offline, all CRUD operations use the local data provider to persist the changes. Upon reconnecting with the server:

  • Any records that were created locally are persisted to the server.
  • Any records that were modified locally are updated on the server.
  • Any records that were deleted locally are deleted on the server.

Listing 10 shows the complete synchronization method. During synchronization, the same online functions are used for the create, update, and delete operations. The first step is to iterate through the local records using the jQuery $.each function.

Records that were locally updated or created are flagged using the isDirty property. The Save operation is identified as new if its unique record ID is negative (that is, not assigned by the MySQL database). Records that are deleted locally are flagged using the isDeleted property.

Listing 10. Synchronizing offline changes to the server
var recordsUpdated = 0;
var recordsCreated = 0;
var recordsDeleted = 0;

$.each(data, function(i,item){
	if (item.isDeleted) {
		deleteOnlineContact(item.id, true);
		recordsDeleted++;
	}		
	else if (item.isDirty && !item.isDeleted) {
		$('input[name="contactId"]')[0].value = item.id;
		$('input[name="firstName"]')[0].value = item.firstName;
		$('input[name="lastName"]')[0].value = item.lastName;
		$('input[name="street1"]')[0].value = item.street1;
		$('input[name="street2"]')[0].value = item.street2;
		$('input[name="city"]')[0].value = item.city;
		$('select[name="state"]')[0].value = item.state;
		$('input[name="zipCode"]')[0].value = item.zipCode;
		
		var dataString = $("#editContactForm").serialize();
		postEditedContact(dataString, true);
		if (item.id > 0) {
			recordsUpdated++;
		}
		else {
			recordsCreated++;
		}			
	}
});

var msg = "Synchronization Summary\n\tRecords Updated: " + recordsUpdated + 
"\n\tRecords Created: " + recordsCreated +"\n\tRecords Deleted: " + recordsDeleted;
alert(msg);

The latest data is fetched from the database using the getcontacts operation and is displayed. Any changes that other users made would be reflected. The data is then persisted locally to ensure that it's available when offline.


Conclusion

In this article, the example application showed a good pattern for online and offline support. A consistent user experience was maintained by using a single HTML page for online and offline mode and by invoking the corresponding online/offline data providers in the form event handlers based on the online/offline status.

The data-synchronization algorithm provides a good foundation; it handles synchronizing the offline creation, deletion, and modification of records. It is not production-ready code, however. For example, it doesn't handle conflicts where the same record is modified locally and on the server by another user.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Web development on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Web development
ArticleID=825665
ArticleTitle=Using HTML5 database and offline capabilities, Part 1: Provide offline data editing and data synchronization
publish-date=07172012