Creating mobile Web applications with HTML 5, Part 2: Unlock local storage for mobile Web applications with HTML 5

Improve the speed of your mobile apps with standardized local storage

One of the most useful new features in HTML 5 is the standardization of local storage. Finally, Web developers can stop trying to fit all client-side data into 4 KB Cookies. Now you can store large amounts of data on the client with a simple API. This is a perfect mechanism for caching, so you can dramatically improve the speed of your application—a critical factor for mobile Web applications that rely on much slower connections than their desktop brothers. In this second article in this series on HTML 5, you will see how to use local storage, how to debug it, and you will see a variety of ways to use it to improve mobile Web applications.

Michael Galpin, Software architect, eBay

Michael Galpin's photoMichael Galpin is an architect at eBay. He is a frequent contributor to developerWorks. He has spoken at various technical conferences, including JavaOne, EclipseCon, and AjaxWorld. To get a preview of what he is working on next, follow @michaelg on Twitter.



29 June 2010 (First published 25 May 2010)

Also available in Chinese Russian Japanese Vietnamese

02 Jun 2010: Added links to Part 3 of this series in About this series, Summary, and Resources sections.

08 Jun 2010: Added links to Part 4 of this series in About this series, Summary, and Resources sections.

29 Jun 2010: Added links to Part 5 of this series in About this series, Summary, and Resources sections.

About this series

HTML 5 is a very hyped technology, but with good reason. It promises to be a technological tipping point to bring desktop application capabilities to the browser. As promising as it is for traditional browsers, it has even more potential for mobile browsers. Even better, the most popular mobile browsers have already adopted and implemented many significant parts of the HTML 5 specification.

In this five-part series, you will take a closer look at several of those new technologies that are part of HTML 5 and can have a huge impact on mobile Web application development. In each part, you will develop a working mobile Web application showcasing an HTML 5 feature that you can use on modern mobile Web browsers like the ones found on the iPhone and Android-based devices.


Prerequisites

Frequently used acronyms

  • API: Application Programming Interface
  • CSS: Cascading stylesheet
  • DOM: Document Object Model
  • HTML: Hypertext Markup Language
  • HTTP: Hypertext Transfer Protocol
  • JSON: JavaScript Object Notation
  • JSONP: JSON with padding
  • SDK: Software Developer Kit
  • UI: User Interface
  • URL: Uniform Resource Locator
  • W3C: World Wide Web Consortium

In this article, you will develop Web applications using the latest Web technologies. Most of the code here is just HTML, JavaScript, and CSS—the core technologies of any Web developer. The most important thing you will need are browsers to test things on. Most of the code in this article will run on the latest desktop browsers, with some noted exceptions. Of course you must test on mobile browsers too, and you will want the latest iPhone and Android SDKs for those. In this article iPhone SDK 3.1.3 and Android SDK 2.1 were used. See Resources for links.


Local storage 101

Web developers have struggled with storing data on the client for ages. HTTP Cookies have been arguably abused for this purpose. Developers have squeezed amazing amounts of data into the 4KB allocated by the HTTP specification. The reason is simple. Interactive Web applications need to store data for a variety of reasons, and it is often inefficient, insecure, or inappropriate to store that data on a server. There have been several alternative approaches to this problem over the years. Various browsers have introduced proprietary storage APIs. Developers have also made use of expanded storage capabilities in the Flash Player by exposing this through JavaScript. Similarly, Google created the Gears plugin for various browsers, and it included storage APIs. Not surprisingly, some JavaScript libraries attempted to smooth out these differences. In other words, these libraries provide a simple API, and then check to see what storage capabilities were present (which might be a proprietary browser API or might be a plugin like Flash).

Fortunately for Web developers, the HTML 5 specification finally contains a standard for local storage that is implemented by a wide variety of browsers. In fact, this standard has been one of the most quickly adopted ones and is supported in the latest versions of all major browsers: Microsoft®, Internet Explorer®, Mozilla Firefox, Opera, Apple Safari, and Google Chrome. Even more importantly for mobile developers, it is supported in WebKit-based browsers like those found in the iPhone and phones using Android (version 2.0 or later) phones as well as other mobile browsers like Mozilla's Fennec. With that in mind, let's take a look at the API.

The Storage APIs

The localStorage API is quite simple. Actually, per the HTML 5 specification, it implements the DOM Storage interface. The reason for this distinction is that HTML 5 specifies two distinct objects that implement this interface, localStorage and sessionStorage. The sessionStorage object is a Storage implementation that only stores data during a session. More precisely, as soon as there is no script running that can access the sessionStorage, the browser can delete sessionStorage data at its leisure. This is in contrast to localStorage, which spans multiple user sessions. The two objects share the same API, so I shall focus on localStorage only.

The Storage API is a classic name/value pair data structure. The most common methods you will use are getItem(name) and setItem(name, value). These work exactly as you might expect: getItem returns the value associated with the name or null if nothing exists, while setItem either adds the name/value pair to localStorage or replaces an existing value. There is also a removeItem(name) that, as the name suggests, removes a name/value pair from localStorage if one exists or does nothing otherwise. Finally, for iterating over all of the name/value pairs, there are a two APIs. One is the length property which gives you the total number of name/values pairs being stored. To go along with this, a key(index) method returns a name from an array of all names used in storage.

With these simple APIs, you can accomplish a lot. Some examples can be personalization or tracking user behavior. These can be important use cases for mobile Web developers, but there is a much more significant one: caching. With localStorage, you can easily cache data from your servers, on the client's local machine. This allows you to avoid waiting for potentially slow calls back to your servers and minimizes the amount of data needed from your servers. Now take a look at an example that demonstrates how to use localStorage to achieve this kind of caching.


Example: Caching with local storage

This example builds on an example that you first started t0 develop in Part 1 of this series. That example showed how to perform a local search of Twitter by getting the location of a user with geolocation APIs. Begin with that example, simplify it, and then give it a performance boost. To start, strip down the example to just a Twitter search without geolocation. Listing 1 shows the simplified Twitter search application.

Listing 1. Most basic Twitter search
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name = "viewport" content = "width = device-width"/>
<title>Basic Twitter Search</title>
<script type="text/javascript">
    function searchTwitter(){
        var query = "http://search.twitter.com/search.json?callback
=showResults&q=";
        query += $("kwBox").value;
        var script = document.createElement("script");
        script.src = query;
        document.getElementsByTagName("head")[0].appendChild(script);
    }
    // ui code deleted for brevity
    function showResults(response){
        var tweets = response.results;
        tweets.forEach(function(tweet){
            tweet.linkUrl = "http://twitter.com/" + tweet.from_user 
+ "/status/" + tweet.id;
        });
        makeResultsTable(tweets);
    }
</script>
<!--  CSS deleted for brevity -->
</head>
<body>
    <div id="main">
        <label for="kwBox">Search Twitter:</label>
        <input type="text" id="kwBox"/>
        <input type="button" value="Go!" onclick="searchTwitter()"/>
    </div>
    <div id="results">
    </div>
</body>
</html>

In this application, you use the Twitter search API's support for JSONP. When the user submits a search, an API call is made by dynamically adding a script tag to the page and specifying the name of a callback function. This allows you to make a cross-domain call from a Web page. Once the call returns, the callback function (showResults) is invoked. You add a link URL to each tweet returned by Twitter and then create a simple table displaying the tweets. To speed this up, you can cache the results from a search query and use these cached results whenever a user submits a query. To begin with, look at how to use localStorage to store the tweets locally.

Saving locally

The basic Twitter search will provide an array of tweets from the Twitter search API. If you can save these tweets locally and associate them with the keyword search that generated them, then you have a usable cache. To save the tweets, you only need to modify the callback function that will be invoked when the call to the Twitter search API returns. Listing 2 shows the modified functions.

Listing 2. Search and save
function searchTwitter(){
    var keyword = $("kwBox").value;
    var query = "http://search.twitter.com/search.json?callback
=processResults&q=";
    query += keyword;
    var script = document.createElement("script");
    script.src = query;
    document.getElementsByTagName("head")[0].appendChild(script);
}
function processResults(response){
    var keyword = $("kwBox").value;
    var tweets = response.results;
    tweets.forEach(function(tweet){
        saveTweet(keyword, tweet);
        tweet.linkUrl = "http://twitter.com/" + tweet.from_user + "/status/" + tweet.id;
    });
    makeResultsTable();
    addTweetsToResultsTable(tweets);
}
function saveTweet(keyword, tweet){
    // check if the browser supports localStorage
    if (!window.localStorage){
        return;
    }
    if (!localStorage.getItem("tweet" + tweet.id)){
        localStorage.setItem("tweet" + tweet.id, JSON.stringify(tweet));
    }
    var index = localStorage.getItem("index::" + keyword);
    if (index){
        index = JSON.parse(index);
    } else {
        index = [];
    }
    if (!index.contains(tweet.id)){
        index.push(tweet.id);
        localStorage.setItem("index::"+keyword, JSON.stringify(index));
    } 
}

Start with the first function, searchTwitter. This is called when a search is submitted by the user. The only thing that has changed compared to Listing 1 is the callback function. Instead of just displaying the tweets when they come back, you need to process them (save them as well as display them). Thus, you specify a new callback function, processResults. You take each tweet and call saveTweet. You also pass the keyword that was used to generate the search results. That is because you want to associate these tweets with that keyword.

In the saveTweet function, start by checking to make sure that localStorage is indeed supported by the browser. As mentioned earlier, localStorage is widely supported in both desktop and mobile browsers, but it is always a good idea to check when using a new feature like this. If it is not supported, then you simply return from the function. Nothing will be saved obviously, but no errors will be given—the application will simply not have a cache in this case. If localStorage is supported, then first check to see if the tweet is already stored. If it is not stored, then save it locally using setItem. Next, retrieve an index object that corresponds to the keyword. This is simply an array of IDs of tweets associated with the keyword. If the tweet ID is not already part of the index, then add it and update the index.

Note that when you save and load JSON in Listing 3, you have used JSON.stringify and JSON.parse. The JSON object (or more accurately, window.JSON) is part of the HTML 5 specification as a native object that is always present. It's stringify method will turn any JavaScript object into a serialized string, while its parse method does the opposite. It restores a JavaScript object from its serialized string representation. This is necessary since localStorage only stores strings. However, the native JSON object is not as widely implemented as localStorage. For example, it is not present on the latest Mobile Safari browser on the iPhone (version 3.1.3 at the time that this was written.) It is supported on the latest Android browsers. You can easily check to see if it is there, and, if not, load an extra JavaScript file. You can obtain the same JSON object that is used natively by going to the json.org Web site (see Resources). To see what these serialized strings look like locally, you can use various browser tools for inspecting what is stored in localStorage for a given site. Figure 1 shows some cached tweets stored locally and viewed using Chrome's Developer Tools.

Figure 1. Locally cached tweets
Screen capture of a list of locally cached tweets (with Key and Value fields)

Both Chrome and Safari have built-in developer tools that allow you to view any data being saved in localStorage. This can be very useful for debugging applications that use localStorage. It shows you the key/value pairs that are stored locally in plain text. Now that you have started saving the tweets coming from Twitter's search APIs so that they can be used as a cache, you just need to start reading them from localStorage. Look at how to do this next.

Rapid local data loading

In Listing 2, you saw some examples of reading from localStorage using its getItem method. Now when a user submits a search, you can check for a cache-hit and immediately load the cached results. Of course, you will still query against the Twitter search API, since people tweet all the time and add to the search results. However, now you are also armed with a way to make the querying even more efficient by only asking for results that you do not already have in cache. Listing 3 shows the updated search code.

Listing 3. Searching locally first
function searchTwitter(){
    if ($("resultsTable")){
        $("resultsTable").innerHTML = ""; // clear results
    }
    makeResultsTable();
    var keyword = $("kwBox").value;
    var maxId = loadLocal(keyword);
    var query = "http://search.twitter.com/search.json?callback=processResults&q=";
    query += keyword;
    if (maxId){
        query += "&since_id=" + maxId;
    }
    var script = document.createElement("script");
    script.src = query;
    document.getElementsByTagName("head")[0].appendChild(script);
}
function loadLocal(keyword){
    if (!window.localStorage){
        return;
    }
    var index = localStorage.getItem("index::" + keyword);
    var tweets = [];
    var i = 0;
    var tweet = {};
    if (index){
        index = JSON.parse(index);
        for (i=0;i<index.length;i++){
            tweet = localStorage.getItem("tweet"+index[i]);
            if (tweet){
                tweet = JSON.parse(tweet);
                tweets.push(tweet);
            }
        }
    }
    if (tweets.length < 1){
        return 0;
    }
    tweets.sort(function(a,b){
        return a.id > b.id;
    });
    addTweetsToResultsTable(tweets);
    return tweets[0].id;
}

The first thing that you will notice is that when a search is submitted, you first call the new loadLocal function. This function returns an integer that is the ID of the newest tweet found in cache. The loadLocal function takes a keyword, as this is used to find the relevant tweets in the localStorage cache. If you have a maxId, then use it to modify the query to Twitter, adding the since_id parameter. You are telling the Twitter API to only return tweets that are newer than the ID given in this parameter. Potentially, this can reduce the number of results coming back from Twitter. Any time you can optimize server calls for a mobile Web application, it can really improve the user experience over slow mobile networks. Now look more closely at loadLocal.

In the loadLocal function, you make use of the data structures stored back in Listing 2. You first load the index associated with the keyword by using the getItem method. If no index is found, then there are no cached tweets, so there is nothing to show and no optimization can be made on the query (and you return a value of 0 to indicate this). If an index is found, then you get the list of IDs from it. Each of these tweets is cached locally, so you just have to use the getItem method again to load each of them from the cache. The tweets that are loaded are then sorted. Use the addTweetsToResultsTable function to display the tweets, and then return the ID of the newest tweet. In this example, the code that gets new tweets directly calls functions for updating the UI. You might be bristling at this as it creates coupling between the code for storing and retrieving tweets and the code for displaying them, all through the processResults function. Using storage events provides an alternative, less coupled approach.

Storage events

Now expand the example application to also show a Top 10 of the search terms that have the most cached results. This might represent the searches that the user submits the most. Listing 4 shows a function for calculating this top 10 and displaying it.

Listing 4. Calculating the top 10 searches
function displayStats(){
    if (!window.localStorage){ return; }
    var i = 0;
    var key = "";
    var index = [];
    var cachedSearches = [];
    for (i=0;i<localStorage.length;i++){
        key = localStorage.key(i);
        if (key.indexOf("index::") == 0){
            index = JSON.parse(localStorage.getItem(key));
            cachedSearches.push ({keyword: key.slice(7), numResults: index.length});
        }
    }
    cachedSearches.sort(function(a,b){
        if (a.numResults == b.numResults){
            if (a.keyword.toLowerCase() < b.keyword.toLowerCase()){
                return -1;
            } else if (a.keyword.toLowerCase() > b.keyword.toLowerCase()){
                return 1;
            }
            return 0;
        }
        return b.numResults - a.numResults;
    }).slice(0,10).forEach(function(search){
        var li = document.createElement("li");
        var txt = document.createTextNode(search.keyword + " : " + search.numResults);
        li.appendChild(txt);
        $("stats").appendChild(li);
    });
}

This function shows off more of the localStorage API. You start by getting the total number of items stored in localStorage and then iterating over them. If the item is an index, then you parse the object and create an object representing the data you are about to process: the keyword associated with the index and the number of tweets in the index. This data is stored in an array called cachedSearches. Next, sort cachedSearches, putting the searches with the most results first, and then using a case-insensitive alphabetical sort if two searches have the same number of cached results. Then take the top 10 searches, create HTML for each search, and append them to an ordered list. Let's call this function when the page first loads, as shown in Listing 5.

Listing 5. Initialize the page
window.onload = function() {
    displayStats();
    document.body.setAttribute("onstorage", "handleOnStorage();");
}

The first line calls the function in Listing 4 when the page loads. The second load is where things get more interesting. Here you set an event handler for the onstorage event. This event fires whenever the localStorage.setItem function finishes executing. This allows you to re-calculate the top 10 searches. Listing 6 shows this event handler.

Listing 6. Storage event handler
function handleOnStorage() {
    if (window.event && window.event.key.indexOf("index::") == 0){
        $("stats").innerHTML = "";
        displayStats();
    }
}

The onstorage event will be associated with the window. It has several useful properties: key, oldValue, and newValue. In addition to these self-explanatory properties, it also has a url (the URL of the page that changed the value) and source (the window that contains the script that changed the value). These last two are more useful if a user might have multiple windows or tabs or even iFrames to your application, none of which are particularly common in mobile applications. Going back to Listing 6, the only property you really need is the key property. You use this to see if it was an index that was modified. If it was, then you reset the top 10 list and re-draw it by calling the displayStats function again. The advantage of this technique is that no other functions need to know about the top 10 list as it is self contained.

I mentioned earlier that DOM Storage in general, which includes both localStorage and sessionStorage, is an HTML 5 feature that has seen widespread adoption. However, storage events is an exception to this—at least on desktop browsers. At the time of writing, the only desktop browsers that support storage events are Safari 4+ and Internet Explorer 8+. It is not supported in Firefox, Chrome, or Opera. However, in the mobile world things are a little better. Both the latest versions of the iPhone and Android browsers fully support storage events, and all of the code presented here runs perfectly in them.


Summary

You, as a developer, can feel very liberated when you suddenly have a huge amount of storage space on the client. For long-time Web developers, it opens up things that they might have wanted to do for years, but did not have a good way to do them that did not rely on buggy tricks. For mobile developers, it's even more exciting as it really opens up the local caching of data. In addition to dramatic improvement of the performance of your application, local caching is key to enabling another new exciting capability of mobile Web applications: going offline. That will be the topic for the next article in this series.


Download

DescriptionNameSize
Article source codelocal.zip8KB

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into XML on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=XML, Open source, Mobile development
ArticleID=491401
ArticleTitle=Creating mobile Web applications with HTML 5, Part 2: Unlock local storage for mobile Web applications with HTML 5
publish-date=06292010