02 Jun 2010: Added links to Part 3 of this series in About this series, Summary, and Resources sections.

08 Jun 2010: Added links to Part 4 of this series in About this series, Summary, and Resources sections.

29 Jun 2010: Added links to Part 5 of this series in About this series, Summary, and Resources sections.

About this series

HTML 5 is a very hyped technology, but with good reason. It promises to be a technological tipping point to bring desktop application capabilities to the browser. As promising as it is for traditional browsers, it has even more potential for mobile browsers. Even better, the most popular mobile browsers have already adopted and implemented many significant parts of the HTML 5 specification.

In this five-part series, you will take a closer look at several of those new technologies that are part of HTML 5 and can have a huge impact on mobile Web application development. In each part, you will develop a working mobile Web application showcasing an HTML 5 feature that you can use on modern mobile Web browsers like the ones found on the iPhone and Android-based devices.

Back to top

Prerequisites

In this article, you will develop Web applications using the latest Web technologies. Most of the code here is just HTML, JavaScript, and CSS—the core technologies of any Web developer. The most important thing you will need are browsers to test things on. Most of the code in this article will run on the latest desktop browsers, with some noted exceptions. Of course you must test on mobile browsers too, and you will want the latest iPhone and Android SDKs for those. In this article iPhone SDK 3.1.3 and Android SDK 2.1 were used. See Resources for links.

Back to top

Local storage 101

Web developers have struggled with storing data on the client for ages. HTTP Cookies have been arguably abused for this purpose. Developers have squeezed amazing amounts of data into the 4KB allocated by the HTTP specification. The reason is simple. Interactive Web applications need to store data for a variety of reasons, and it is often inefficient, insecure, or inappropriate to store that data on a server. There have been several alternative approaches to this problem over the years. Various browsers have introduced proprietary storage APIs. Developers have also made use of expanded storage capabilities in the Flash Player by exposing this through JavaScript. Similarly, Google created the Gears plugin for various browsers, and it included storage APIs. Not surprisingly, some JavaScript libraries attempted to smooth out these differences. In other words, these libraries provide a simple API, and then check to see what storage capabilities were present (which might be a proprietary browser API or might be a plugin like Flash).

Fortunately for Web developers, the HTML 5 specification finally contains a standard for local storage that is implemented by a wide variety of browsers. In fact, this standard has been one of the most quickly adopted ones and is supported in the latest versions of all major browsers: Microsoft®, Internet Explorer®, Mozilla Firefox, Opera, Apple Safari, and Google Chrome. Even more importantly for mobile developers, it is supported in WebKit-based browsers like those found in the iPhone and phones using Android (version 2.0 or later) phones as well as other mobile browsers like Mozilla's Fennec. With that in mind, let's take a look at the API.

The Storage APIs

The localStorage API is quite simple. Actually, per the HTML 5 specification, it implements the DOM Storage interface. The reason for this distinction is that HTML 5 specifies two distinct objects that implement this interface, localStorage and sessionStorage. The sessionStorage object is a Storage implementation that only stores data during a session. More precisely, as soon as there is no script running that can access the sessionStorage, the browser can delete sessionStorage data at its leisure. This is in contrast to localStorage, which spans multiple user sessions. The two objects share the same API, so I shall focus on localStorage only.

The Storage API is a classic name/value pair data structure. The most common methods you will use are getItem(name) and setItem(name, value). These work exactly as you might expect: getItem returns the value associated with the name or null if nothing exists, while setItem either adds the name/value pair to localStorage or replaces an existing value. There is also a removeItem(name) that, as the name suggests, removes a name/value pair from localStorage if one exists or does nothing otherwise. Finally, for iterating over all of the name/value pairs, there are a two APIs. One is the length property which gives you the total number of name/values pairs being stored. To go along with this, a key(index) method returns a name from an array of all names used in storage.

With these simple APIs, you can accomplish a lot. Some examples can be personalization or tracking user behavior. These can be important use cases for mobile Web developers, but there is a much more significant one: caching. With localStorage, you can easily cache data from your servers, on the client's local machine. This allows you to avoid waiting for potentially slow calls back to your servers and minimizes the amount of data needed from your servers. Now take a look at an example that demonstrates how to use localStorage to achieve this kind of caching.

Back to top

Example: Caching with local storage

This example builds on an example that you first started t0 develop in Part 1 of this series. That example showed how to perform a local search of Twitter by getting the location of a user with geolocation APIs. Begin with that example, simplify it, and then give it a performance boost. To start, strip down the example to just a Twitter search without geolocation. Listing 1 shows the simplified Twitter search application.

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name = "viewport" content = "width = device-width"/>
<title>Basic Twitter Search</title>
<script type="text/javascript">
    function searchTwitter(){
        var query = "http://search.twitter.com/search.json?callback
=showResults&q=";
        query += $("kwBox").value;
        var script = document.createElement("script");
        script.src = query;
        document.getElementsByTagName("head")[0].appendChild(script);
    }
    // ui code deleted for brevity
    function showResults(response){
        var tweets = response.results;
        tweets.forEach(function(tweet){
            tweet.linkUrl = "http://twitter.com/" + tweet.from_user 
+ "/status/" + tweet.id;
        });
        makeResultsTable(tweets);
    }
</script>
<!--  CSS deleted for brevity -->
</head>
<body>
    <div id="main">
        <label for="kwBox">Search Twitter:</label>
        <input type="text" id="kwBox"/>
        <input type="button" value="Go!" onclick="searchTwitter()"/>
    </div>
    <div id="results">
    </div>
</body>
</html>

In this application, you use the Twitter search API's support for JSONP. When the user submits a search, an API call is made by dynamically adding a script tag to the page and specifying the name of a callback function. This allows you to make a cross-domain call from a Web page. Once the call returns, the callback function (showResults) is invoked. You add a link URL to each tweet returned by Twitter and then create a simple table displaying the tweets. To speed this up, you can cache the results from a search query and use these cached results whenever a user submits a query. To begin with, look at how to use localStorage to store the tweets locally.

Saving locally

The basic Twitter search will provide an array of tweets from the Twitter search API. If you can save these tweets locally and associate them with the keyword search that generated them, then you have a usable cache. To save the tweets, you only need to modify the callback function that will be invoked when the call to the Twitter search API returns. Listing 2 shows the modified functions.

function searchTwitter(){
    var keyword = $("kwBox").value;
    var query = "http://search.twitter.com/search.json?callback
=processResults&q=";
    query += keyword;
    var script = document.createElement("script");
    script.src = query;
    document.getElementsByTagName("head")[0].appendChild(script);
}
function processResults(response){
    var keyword = $("kwBox").value;
    var tweets = response.results;
    tweets.forEach(function(tweet){
        saveTweet(keyword, tweet);
        tweet.linkUrl = "http://twitter.com/" + tweet.from_user + "/status/" + tweet.id;
    });
    makeResultsTable();
    addTweetsToResultsTable(tweets);
}
function saveTweet(keyword, tweet){
    // check if the browser supports localStorage
    if (!window.localStorage){
        return;
    }
    if (!localStorage.getItem("tweet" + tweet.id)){
        localStorage.setItem("tweet" + tweet.id, JSON.stringify(tweet));
    }
    var index = localStorage.getItem("index::" + keyword);
    if (index){
        index = JSON.parse(index);
    } else {
        index = [];
    }
    if (!index.contains(tweet.id)){
        index.push(tweet.id);
        localStorage.setItem("index::"+keyword, JSON.stringify(index));
    } 
}

Start with the first function, searchTwitter. This is called when a search is submitted by the user. The only thing that has changed compared to Listing 1 is the callback function. Instead of just displaying the tweets when they come back, you need to process them (save them as well as display them). Thus, you specify a new callback function, processResults. You take each tweet and call saveTweet. You also pass the keyword that was used to generate the search results. That is because you want to associate these tweets with that keyword.

In the saveTweet function, start by checking to make sure that localStorage is indeed supported by the browser. As mentioned earlier, localStorage is widely supported in both desktop and mobile browsers, but it is always a good idea to check when using a new feature like this. If it is not supported, then you simply return from the function. Nothing will be saved obviously, but no errors will be given—the application will simply not have a cache in this case. If localStorage is supported, then first check to see if the tweet is already stored. If it is not stored, then save it locally using setItem. Next, retrieve an index object that corresponds to the keyword. This is simply an array of IDs of tweets associated with the keyword. If the tweet ID is not already part of the index, then add it and update the index.

Note that when you save and load JSON in Listing 3, you have used JSON.stringify and JSON.parse. The JSON object (or more accurately, window.JSON) is part of the HTML 5 specification as a native object that is always present. It's stringify method will turn any JavaScript object into a serialized string, while its parse method does the opposite. It restores a JavaScript object from its serialized string representation. This is necessary since localStorage only stores strings. However, the native JSON object is not as widely implemented as localStorage. For example, it is not present on the latest Mobile Safari browser on the iPhone (version 3.1.3 at the time that this was written.) It is supported on the latest Android browsers. You can easily check to see if it is there, and, if not, load an extra JavaScript file. You can obtain the same JSON object that is used natively by going to the json.org Web site (see Resources). To see what these serialized strings look like locally, you can use various browser tools for inspecting what is stored in localStorage for a given site. Figure 1 shows some cached tweets stored locally and viewed using Chrome's Developer Tools.

Both Chrome and Safari have built-in developer tools that allow you to view any data being saved in localStorage. This can be very useful for debugging applications that use localStorage. It shows you the key/value pairs that are stored locally in plain text. Now that you have started saving the tweets coming from Twitter's search APIs so that they can be used as a cache, you just need to start reading them from localStorage. Look at how to do this next.

Rapid local data loading

In Listing 2, you saw some examples of reading from localStorage using its getItem method. Now when a user submits a search, you can check for a cache-hit and immediately load the cached results. Of course, you will still query against the Twitter search API, since people tweet all the time and add to the search results. However, now you are also armed with a way to make the querying even more efficient by only asking for results that you do not already have in cache. Listing 3 shows the updated search code.

function searchTwitter(){
    if ($("resultsTable")){
        $("resultsTable").innerHTML = ""; // clear results
    }
    makeResultsTable();
    var keyword = $("kwBox").value;
    var maxId = loadLocal(keyword);
    var query = "http://search.twitter.com/search.json?callback=processResults&q=";
    query += keyword;
    if (maxId){
        query += "&since_id=" + maxId;
    }
    var script = document.createElement("script");
    script.src = query;
    document.getElementsByTagName("head")[0].appendChild(script);
}
function loadLocal(keyword){
    if (!window.localStorage){
        return;
    }
    var index = localStorage.getItem("index::" + keyword);
    var tweets = [];
    var i = 0;
    var tweet = {};
    if (index){
        index = JSON.parse(index);
        for (i=0;i<index.length;i++){
            tweet = localStorage.getItem("tweet"+index[i]);
            if (tweet){
                tweet = JSON.parse(tweet);
                tweets.push(tweet);
            }
        }
    }
    if (tweets.length < 1){
        return 0;
    }
    tweets.sort(function(a,b){
        return a.id > b.id;
    });
    addTweetsToResultsTable(tweets);
    return tweets[0].id;
}

The first thing that you will notice is that when a search is submitted, you first call the new loadLocal function. This function returns an integer that is the ID of the newest tweet found in cache. The loadLocal function takes a keyword, as this is used to find the relevant tweets in the localStorage cache. If you have a maxId, then use it to modify the query to Twitter, adding the since_id parameter. You are telling the Twitter API to only return tweets that are newer than the ID given in this parameter. Potentially, this can reduce the number of results coming back from Twitter. Any time you can optimize server calls for a mobile Web application, it can really improve the user experience over slow mobile networks. Now look more closely at loadLocal.

In the loadLocal function, you make use of the data structures stored back in Listing 2. You first load the index associated with the keyword by using the getItem method. If no index is found, then there are no cached tweets, so there is nothing to show and no optimization can be made on the query (and you return a value of 0 to indicate this). If an index is found, then you get the list of IDs from it. Each of these tweets is cached locally, so you just have to use the getItem method again to load each of them from the cache. The tweets that are loaded are then sorted. Use the addTweetsToResultsTable function to display the tweets, and then return the ID of the newest tweet. In this example, the code that gets new tweets directly calls functions for updating the UI. You might be bristling at this as it creates coupling between the code for storing and retrieving tweets and the code for displaying them, all through the processResults function. Using storage events provides an alternative, less coupled approach.

Storage events

Now expand the example application to also show a Top 10 of the search terms that have the most cached results. This might represent the searches that the user submits the most. Listing 4 shows a function for calculating this top 10 and displaying it.

function displayStats(){
    if (!window.localStorage){ return; }
    var i = 0;
    var key = "";
    var index = [];
    var cachedSearches = [];
    for (i=0;i<localStorage.length;i++){
        key = localStorage.key(i);
        if (key.indexOf("index::") == 0){
            index = JSON.parse(localStorage.getItem(key));
            cachedSearches.push ({keyword: key.slice(7), numResults: index.length});
        }
    }
    cachedSearches.sort(function(a,b){
        if (a.numResults == b.numResults){
            if (a.keyword.toLowerCase() < b.keyword.toLowerCase()){
                return -1;
            } else if (a.keyword.toLowerCase() > b.keyword.toLowerCase()){
                return 1;
            }
            return 0;
        }
        return b.numResults - a.numResults;
    }).slice(0,10).forEach(function(search){
        var li = document.createElement("li");
        var txt = document.createTextNode(search.keyword + " : " + search.numResults);
        li.appendChild(txt);
        $("stats").appendChild(li);
    });
}

This function shows off more of the localStorage API. You start by getting the total number of items stored in localStorage and then iterating over them. If the item is an index, then you parse the object and create an object representing the data you are about to process: the keyword associated with the index and the number of tweets in the index. This data is stored in an array called cachedSearches. Next, sort cachedSearches, putting the searches with the most results first, and then using a case-insensitive alphabetical sort if two searches have the same number of cached results. Then take the top 10 searches, create HTML for each search, and append them to an ordered list. Let's call this function when the page first loads, as shown in Listing 5.

window.onload = function() {
    displayStats();
    document.body.setAttribute("onstorage", "handleOnStorage();");
}

The first line calls the function in Listing 4 when the page loads. The second load is where things get more interesting. Here you set an event handler for the onstorage event. This event fires whenever the localStorage.setItem function finishes executing. This allows you to re-calculate the top 10 searches. Listing 6 shows this event handler.

function handleOnStorage() {
    if (window.event && window.event.key.indexOf("index::") == 0){
        $("stats").innerHTML = "";
        displayStats();
    }
}

The onstorage event will be associated with the window. It has several useful properties: key, oldValue, and newValue. In addition to these self-explanatory properties, it also has a url (the URL of the page that changed the value) and source (the window that contains the script that changed the value). These last two are more useful if a user might have multiple windows or tabs or even iFrames to your application, none of which are particularly common in mobile applications. Going back to Listing 6, the only property you really need is the key property. You use this to see if it was an index that was modified. If it was, then you reset the top 10 list and re-draw it by calling the displayStats function again. The advantage of this technique is that no other functions need to know about the top 10 list as it is self contained.

I mentioned earlier that DOM Storage in general, which includes both localStorage and sessionStorage, is an HTML 5 feature that has seen widespread adoption. However, storage events is an exception to this—at least on desktop browsers. At the time of writing, the only desktop browsers that support storage events are Safari 4+ and Internet Explorer 8+. It is not supported in Firefox, Chrome, or Opera. However, in the mobile world things are a little better. Both the latest versions of the iPhone and Android browsers fully support storage events, and all of the code presented here runs perfectly in them.

Back to top

Summary

You, as a developer, can feel very liberated when you suddenly have a huge amount of storage space on the client. For long-time Web developers, it opens up things that they might have wanted to do for years, but did not have a good way to do them that did not rely on buggy tricks. For mobile developers, it's even more exciting as it really opens up the local caching of data. In addition to dramatic improvement of the performance of your application, local caching is key to enabling another new exciting capability of mobile Web applications: going offline. That will be the topic for the next article in this series.

Back to top

Download

Information about download methods

Resources

Learn

• Creating mobile Web applications with HTML 5, Part 1: Combine HTML 5, geolocation APIs, and Web services to create mobile mashups (Michael Galpin, developerWorks, May 2010): Find and track location coordinates to use in various Web services. Use the various aspects of the geolocation standard with HTML 5 and popular Web services to create an interesting mobile mashup in Part 1 of this series.
  
  
• Creating mobile Web applications with HTML 5, Part 3: Make mobile Web applications work offline with HTML 5 (Michael Galpin, developerWorks, June 2010): Enable your application to function with or without an Internet connection and learn to detect when your application goes from offline to online and vice versa.
  
  
• Creating mobile Web applications with HTML 5, Part 4: Use Web Workers to speed up your mobile Web applications (Michael Galpin, developerWorks, June 2010): Web Workers bring multi-threading to Web applications. Learn to work with Web Workers and which tasks are most appropriate for them.
  
  
• Creating mobile Web applications with HTML 5, Part 5: Develop new visual UI features in HTML 5: Add Canvas, CSS3, and more semantic elements to mobile Web apps (Michael Galpin, developerWorks, June 2010): Provide full 2-D graphics in the browser with Canvas, the most eye-catching of the new UI capabilities in HTML 5. Learn to use Canvas and other visual elements in your mobile Web applications.
  
  
• Create Ajax applications for the mobile Web (Michael Galpin, developerWorks, March 2010): Explore how to use Ajax, a key part of any mobile Web application.
  
  
• New elements in HTML 5 (Elliotte Rusty Harold, developerWorks, August 2007): HTML 5 is not just about JavaScript. Read about some of the new markup in HTML 5.
  
  
• Android and iPhone browser wars, Part 1: WebKit to the rescue (Frank Ableson, developerWorks, December 2009): Do you like the mobile Web application approach using HTML 5 approach, but still want your application in the iPhone App Store and Android Market? See how you can get the best of both worlds in Part 1 of this two-part article series.
  
  
• Invisible Flash: Enhance Web applications by secretly using the Flash player (Michael Galpin, developerWorks, February 2010): Many Web developers have been using Flash to get similar functionality for years. See how you can do the same in this article.
  
  
• Dive Into HTML 5: Check out this free book for a great look at HTML 5 detection techniques as well as the many features of HTML 5.
  
  
• Safari Reference Library: Keep this resource handy if you develop Web applications for the iPhone.
  
  
• W3C HTML 5 Specification (Working Draft, March 2010): Explore this definitive source on HTML 5.
  
  
• More articles by this author (Michael Galpin, developerWorks, April 2006-current): Read articles about XML, Eclipse, Apache Geronimo, Ajax, more Google APIs, and other technologies.
  
  
• My developerWorks: Personalize your developerWorks experience.
  
  
• IBM XML certification: Find out how you can become an IBM-Certified Developer in XML and related technologies.
  
  
• XML technical library: See the developerWorks XML Zone for a wide range of technical articles and tips, tutorials, standards, and IBM Redbooks.
  
  
• developerWorks technical events and webcasts: Stay current with technology in these sessions.
  
  
• developerWorks on Twitter: Join today to follow developerWorks tweets.
  
  
• developerWorks podcasts: Listen to interesting interviews and discussions for software developers.
  
  

Get products and technologies

• Modernizr project: Get a comprehensive utility for detecting HTML 5 features including like localStorage, Web Workers, applicationCache and others.
  
  
• The Android Developers Web site: Download the Android SDK, access the API reference, and get the latest news on Android.
  
  
• iPhone SDK: Get the latest iPhone SDK to develop iPad, iPhone and iPod touch applications.
  
  
• The Android Open Source Project: Get the open source code for the Android mobile platform.
  
  
• The Google App Engine SDK: Download Java™ and Python tools to build scalable Web applications using Google.
  
  
• JSON object from the json.org Web site: Get the same JSON object that is used natively.
  
  
• IBM product evaluation versions: Download or explore the online trials in the IBM SOA Sandbox and get your hands on application development tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
  
  

Discuss

• XML zone discussion forums: Participate in any of several XML-related discussions.
  
  
• developerWorks blogs: Check out these blogs and get involved.
  
  

About the author

Michael Galpin is an architect at eBay. He is a frequent contributor to developerWorks. He has spoken at various technical conferences, including JavaOne, EclipseCon, and AjaxWorld. To get a preview of what he is working on next, follow @michaelg on Twitter.

Rate this article

Comments

Back to top

Table of contents

• About this series
• Prerequisites
• Local storage 101
• Example: Caching with local storage
• Summary
• Download
• Resources
• About the author
• Comments

Next steps from IBM

Create rich Internet applications using WebSphere Portal Express Web 2.0 technologies.

---

• Try: WebSphere Portal Express
• Article: Get started using Ajax and WebSphere Portal Express
• Tutorial: Quickly build a an Ajax-enabled WebSphere Portal Express Web application
• Demo: Architecture, design, and construction using the IBM Rational Software Delivery Platform: AJAX
• Buy: WebSphere Portal Express

Dig deeper into XML on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers