Skip to main content

By clicking Submit, you agree to the developerWorks terms of use.

The first time you sign into developerWorks, a profile is created for you. Select information in your developerWorks profile is displayed to the public, but you may edit the information at any time. Your first name, last name (unless you choose to hide them), and display name will accompany the content that you post.

All information submitted is secure.

  • Close [x]

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerworks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

By clicking Submit, you agree to the developerWorks terms of use.

All information submitted is secure.

  • Close [x]

XML processing in Ajax, Part 3: JSON and avoiding proxies

Parsing and transforming XML in Ajax programs

Mark Pruett (mark.l.pruett@dom.com), System Architect, Dominion
Mark Pruett is a Systems Architect at Dominion. He's a contributing author to O'Reilly Media's book Ajax Hacks and author of the O'Reilly Shortcuts Ajax and Web Services and Yahoo! Pipes.

Summary:  Ajax-style server calls don't necessarily require XMLHttp requests. This last installment of the series uses a public Web service, JavaScript Object Notation (JSON), and dynamic script tags in a final approach to the weather badge project.

View more content in this series

Date:  18 Mar 2008
Level:  Intermediate
Also available in:   Chinese  Japanese

Activity:  14040 views
Comments:  

In the previous two installments, I presented three approaches to creating a reusable weather badge. Each approach used Asynchronous JavaScript™ + XML (Ajax) techniques, specifically the JavaScript XMLHttpRequest object, to implement a weather badge library. Each approach used some form of Web proxy to convey the National Weather Service (NWS) XML data from their server to my server, avoiding Ajax's same domain problem.

Frequently used acronyms

  • DOM: Document Object Model
  • HTML: Hypertext Markup Language
  • RSS: Rich Site Summary
  • XML: Extensible Markup Language
  • XSLT: Extensible Stylesheet Language Transformation

The same domain problem, explained in more detail in Part 1 of this series, is a security limitation that restricts XMLHttp requests to the same server that delivered the original Web page. In practice, many Ajax applications need data from realms outside that single server. This leads to design decisions that require access to Web server configurations (for example, to create Apache ProxyPass rules) or creation of special server scripts.

As it turns out, you have another alternative. It circumvents the same domain problem, freeing the programmer from the requirements described above.

Tools for Approach 4

Before detailing this final implementation of the weather badge library, I'll describe the tools that enable this final approach:

  • Yahoo! Pipes
  • JSON
  • Dynamic script tags

Yahoo! Pipes

Yahoo! Pipes is a Web-based tool for aggregating various types of Web-accessible data, such as RSS feeds. Yahoo! Pipes are created using a graphical editor (shown in Figure 1) that runs in standard Web browsers such as Windows® Internet Explorer® or Firefox.


Figure 1. The Yahoo! Pipes editor
The Yahoo! Pipes Editor

You build a pipe by dragging modules from a tool palette and arranging them on the canvas. Modules are connected to each other by attaching a pipe from the output of one module to the input of another module.

A completed Yahoo! Pipe has a unique URL. When the URL is accessed, the Pipe logic is executed on Yahoo!'s servers. Their servers handle access to any data sources, perform the data manipulations, and then output the results. By default the data is output in RSS format, but other data formats, including JSON, can be specified in the URL. The ability to output data in JSON format will become important shortly.

The NWS data used by my weather badge module can serve as a data source in Yahoo! Pipes. The four-character NWS station ID is specified as a parameter to the Yahoo! Pipe URL.

The Pipe shown in Figure 1 shows the Pipe I built for use in my last weather badge implementation. While Yahoo! Pipes can do very powerful data manipulation, here I really only use it for one purpose: to translate the NWS data from XML to JSON.

JSON

I've mentioned JSON a few times already. What is it, and why might I want to use it in the weather badge application?

JSON is JavaScript Object Notation, a data format natively understood by the JavaScript language. Unlike XML, which must be specially parsed by JavaScript programs, JSON is JavaScript code.

To compare JSON and XML, first look at an abbreviated version of the NWS XML data for Richmond, Virginia, shown in Listing 1.


Listing 1. NWS data in original XML form
<?xml version="1.0" encoding="ISO-8859-1"?> 
<current_observation version="1.0"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation=
    "http://www.weather.gov/data/current_obs/current_observation.xsd">

    <location>Richmond International Airport, VA</location>
    <station_id>KRIC</station_id>
    <observation_time>
      Last Updated on Jan 26, 1:54 pm EST
    </observation_time>
    <weather>Mostly Cloudy</weather>
    <temperature_string>41 F (5 C)</temperature_string>
    <relative_humidity>32</relative_humidity>
    <wind_string>Calm</wind_string>
    <visibility_mi>10.00</visibility_mi>
    <icon_url_base>
      http://weather.gov/weather/images/fcicons/
    </icon_url_base>
    <icon_url_name>bkn.jpg</icon_url_name>

</current_observation>

Listing 2 is the same data in JSON format.


Listing 2. NWS data translated into JSON format
{
  "count": 1,
  "value":
    {"title": "NOAA Regional Weather",
     "description": "Pipes Output",
     "link": "http:\/\/pipes.yahoo.com\/pipes\/pipe.info?_id=CI6HgSh43BGP8nmJouNLYQ",
     "callback": "",
     "items":
       [
         {
          "location": "Richmond International Airport, VA",
          "station_id": "KRIC",
          "observation_time": "Last Updated on Jan 26, 1:54 pm EST",
          "weather": "Mostly Cloudy",
          "temperature_string": "41 F (5 C)",
          "relative_humidity": "32",
          "wind_string": "Calm",
          "visibility_mi": "10.00",
          "icon_url_base": "http:\/\/weather.gov\/weather\/images\/fcicons\/",
          "icon_url_name": "bkn.jpg",
         }
       ]
    }
}

JSON, at its simplest, is merely a JavaScript data structure. The JSON in the above listing could be assigned to a JavaScript variable and then easily referenced, as shown in Listing 3.


Listing 3. Accessing a JSON data structure in JavaScript code
var weather = {
      "count": 1,
      "value":
      {
       "items":
        [
         {
          "location": "Richmond International Airport, VA",
          ...
         }
        ]
      }
    };

alert (weather.value.items[0].location);

This snippet of JavaScript code assigns an abbreviated version of the JSON data to the JavaScript variable weather. The location element is referenced as a parameter to the JavaScript alert() function. The location value pops up in an alert window, as shown in Figure 2.


Figure 2. Location displayed in an alert() through JSON
JSON location element displayed in alert()

Most JavaScript programmers find it simpler and more intuitive to reference JSON data structures than to access an equivalent XML DOM structure. That reason alone may explain the growing popularity of the data format in Ajax applications.

But JSON allows for another technique that makes it attractive to Ajax developers: through some JavaScript legerdemain, JSON lets you avoid the need to proxy your data.

Dynamic script tags

The three previous approaches to the weather badge library had one trait in common. The JavaScript program running in the browser had to request (through an XMLHttp request) data from my server. This is due to the same domain limitation of XMLHttp requests.

Two of the earlier approaches used an Apache ProxyPass rule to handle this problem, while one approach used a special-purpose server-side script to get the data from the NWS server.

But what if I don't have access to my server's configuration, or what if I can't run server-side scripts?

There is a technique available that allows a JavaScript program to access data on a server in another domain. This technique is sometimes called the script tag hack.

To see how it works, first consider the HTML script tag. It's normally embedded in a Web page as a way to include JavaScript code libraries, as shown in Listing 4.


Listing 4. An HTML script tag
<html>
  <head>
    <title>An Example</title>
    <script language="JavaScript" src="some_javascript_code.js"></script>

Listing 4 shows the first few lines from a Web page. The last line is a script tag indicating that the JavaScript source file named some_javascript_code.js should be loaded from the server. The src attribute tag specifies the URL where the script is located. In this example, the script resides on the same server, but that's not a requirement. JavaScript code can be included from any server.

This means that you can load JSON data, which is just JavaScript code, into the browser using a script tag. That doesn't seem very useful for the weather badge application, since script tags are part of the HTML that gets loaded when the page is initially loaded. For an Ajax solution, I need to be able to make a server request after the page is loaded.

But JavaScript code can manipulate the DOM of the Web page. I can add and modify data in the Web page. What happens if I dynamically create a script tag in the JavaScript code and add it to the DOM? Listing 5 is what the code looks like.


Listing 5. Dynamically creating a script tag
var head = document.getElementsByTagName("head").item(0);
var script = document.createElement ("script");
script.src = "some_javascript_code.js";
head.appendChild (script);

When this code runs, a new script tag is appended as a child to the current page's head tag. It's equivalent to the snippet of HTML in Listing 4, except that the JavaScript code in some_javascript_code.js is loaded after the initial page load.

Approach 4: JSON and dynamic script tags

You might still wonder how this helps with the weather badge library. Suppose that this is the entire contents of the some_javascript_code.js file:

alert ("Hello!");

As soon as the JavaScript file is loaded into the browser, the JavaScript interpreter runs any executable code it contains. In this case, an alert window is immediately opened.

Because the JSON data generated from Yahoo! Pipes is JavaScript code, you can load it dynamically using the technique described above. But the JSON data from Yahoo! Pipes isn't executable JavaScript code; it's just a JavaScript data structure. I can load it into a browser, but how can I tell when it's arrived?

It might be tempting to set some sort of timer before creating the dynamic script tag and just assume the JSON has arrived after, say, three seconds. That's unreliable at best and, as it turns out, unnecessary.

Each Yahoo! Pipe has a unique URL. Listing 6 shows the URL for my NWS pipe.


Listing 6. The NWS Yahoo! Pipe URL
http://pipes.yahoo.com/pipes/pipe.run?
&_id=CI6HgSh43BGP8nmJouNLYQ
&textinput1=KRIC
&_run=1
&_render=json
&_callback=myfunction

I've split the URL over several lines to more clearly show the different parameters. The _id parameter identifies this as my pipe, and textinput1 indicates the NWS station ID. The _run parameter indicates I want to execute the pipe, and _render tells the Yahoo! Pipes server to output the data in JSON format.

What does the final parameter, _callback, do? It tells the Yahoo! server to wrap up the data in a call to the function I've specified, in this case myfunction(). What Yahoo! Pipes sends back to my JavaScript program looks something like this:

myfunction ( {"count":1, ... } );

I've left out most of the JSON data for brevity's sake, but you get the idea. And just like the earlier example where I loaded a tiny JavaScript file containing a single alert() call, the JavaScript interpreter executes the code above, calling myfunction() and passing that function all the data I requested.

Using a callback function means I don't need to know when the server is finished loading the JSON data.

So what's in the callback function? In the case of my weather badge library, it handles formatting the weather data, much like the XMLHttpRequest response function I wrote for Approach 1. In that code, I extracted the data I needed from the responseXML DOM object. Here, I just need to access the JSON data structure, as in the jsonHandler method shown in Listing 7, which provides the complete code for the JSON approach to the weather badge.


Listing 7. The JSON/Yahoo! Pipes implementation of the weather badge library: weather_badge_ypipes_json.js
var yp = new Array();
var idx = 0;

function weather_badge (nws_id, div_name) {

  var callback_obj = "yp[" + idx + "]";

  var url = "http://pipes.yahoo.com/pipes/pipe.run?"
          + "&_id=CI6HgSh43BGP8nmJouNLYQ"
          + "&textinput1=" + nws_id
          + "&_run=1"
          + "&_render=json";

  yp[idx] = new YPipesWeather (url, div_name, callback_obj);
  yp[idx].requestJSON ();
  idx++;
}

// The YPipesWeather constructor

function YPipesWeather (ypipes_url, div_name, obj_name) {
  this.url = ypipes_url + "&_callback=" + obj_name + ".jsonHandler";
  this.div_name = div_name;
}

// The requestJSON method: it builds the script tag
// that launches our request to the Yahoo! server.

YPipesWeather.prototype.requestJSON = function () {

  // Dynamically create a script tag. This initiates
  // a request to the Yahoo! server.

  var head = document.getElementsByTagName("head").item(0);
  var script = document.createElement ("script");
  script.src = this.url;
  head.appendChild (script);
}

// The jsonHandler method: this is our callback function.
// It's called when the JSON is returned to our browser
// from Yahoo!.

YPipesWeather.prototype.jsonHandler = function (json) {
  var div = document.getElementById (this.div_name);

  var weather = json.value.items[0];

  var html = "";

  html += "<center>\n";

  html += "<b>" + weather.location + "</b><br>\n";

  html += weather.weather + "<br>";

  html += "<img border='0' src='"
       + weather.icon_url_base
       + weather.icon_url_name
  + "'><br>";

  html += weather.temperature_string + "<br>";

  html += "Wind: " + weather.wind_string + "<br>";
  html += "Humidity: " + weather.relative_humidity + "%<br>";
  html += "Visibility: " + weather.visibility_mi + " miles<br>";

  html += "<br><span style='font-size: 0.8em; font-weight: bold;'>"
  + weather.observation_time + "</span><br>";

  html += "</center>\n";

  div.innerHTML = html;

}

As with all the previous approaches, I implement a weather_badge() function. I pass this function a four-character NWS station ID and the name of an HTML DIV tag. The DIV tag is the area of the Web page onto which the badge is rendered.

The rest of the listing defines a JavaScript object called YPipesWeather. Its implementation consists of three functions: a constructor, a method that dynamically creates a script tag (and thereby runs the Yahoo! Pipe), and a callback function.

The result is a JavaScript library that, even though it doesn't use the XMLHttpRequest object, behaves identically to the three earlier approaches.

Figure 3 shows the data pipeline for my final weather badge approach. After the initial page load, no other access to my Web server is required.


Figure 3. JSON and dynamic script tags
Approach 4: JSON and Dynamic Script Tags

Listing 8 shows an example HTML Web page that uses the JSON version of the weather badge library.


Listing 8. Using the JSON weather badge library in a Web page
<html>
  <head>
    <title>JSON Script Tag Example</title>
    <link rel="stylesheet" type="text/css" href="weather.css" />
    <script language="JavaScript" 
            src="weather_badge_ypipes_json.js">
    </script>
    <script>
      function init () {
        weather_badge ("KRIC", "target1");
      }
    </script>
  </head>

  <body onload="init();">
    <h3>JSON Script Tag Example</h3>

    <div class="wbadge" id="target1">
      Loading...
    </div>
  </body>
</html>

Comparing the four approaches

To recap, I've presented four approaches to implementing an Ajax weather badge library that

  • Reads XML data from the NWS server
  • Parses this XML data to extract the portions that I need
  • Formats the extracted data into HTML
  • Displays the HTML on a Web page

Table 1 describes the four approaches.


Table 1. Four versions of the weather badge library
ApproachDescription
1: Walking the DOM treeA simple Web proxy on a server pulls data from the NWS server and sends it to the browser. Within the browser, the JavaScript interpreter extracts parts of the returned responseXML DOM tree, adds some HTML formatting, and inserts this into a DIV tag within the page.
2: XSLT on the serverA server-side script pulls the data from the NWS server, uses XSLT to transform that XML to HTML, then sends the HTML snippet back to the browser. The browser then just inserts the snippet into a DIV tag.
3: Client-side XSLTThis approach uses a simple Web proxy (identical to Approach 1) to send the XML data back to the browser. Unlike Approach 1, client-side XSLT is used to convert the XML to HTML and insert this into a DIV tag.
4: JSON and dynamic script tagsAn external service (Yahoo! Pipes) converts the NWS data from XML to JSON. The weather badge library exploits the special qualities of JSON and the JavaScript language to pull the converted data back to the browser—side-stepping the need for proxying.

Which approach is the best? Even for a simple program like this, the answer isn't obvious. What are the attributes of a best solution?

The last approach uses JSON to avoid the need for proxies. This makes it easier to implement in environments where modifying server configurations or writing server scripts is not an option. But the flip side of that advantage is being tied to Yahoo! Pipes, a third-party Web service. If Yahoo! changes their service, will it break my solution?

What about Approach 2, where XML is converted to HTML on the server side? Is it better to let the server do more of the work? The answer to this question depends in part on how much you trust the JavaScript language (across multiple browser versions). Approach 3 shows you can move much of the processing into the browser. But unlike the server script, which you control, the type of browser that your users run can vary. And different browsers, as shown in Part 2, handle tasks such as XSLT processing using different methods. It's harder to debug JavaScript code than a back-end server script. Until the JavaScript language is standard across all browsers, minimizing JavaScript code is a prudent tactic.

And what about XSLT in general? Does it make sense to abstract the XML-to-HTML translation using XSLT? The alternatives are Approach 1, accessing the XML DOM tree in the JavaScript code, and Approach 4, using a software service to translate XML to JSON and then access the JSON data structure. Using XSLT makes it easy to shift the translation from the server to the browser, or vice versa, without changing the XSLT code. What's the cost of this XSLT abstraction layer? I wager that Approach 1 is faster than Approach 3 because the former approach simply accesses a handful of already-parsed XML data elements. The browser-side XSLT version needs to run an XSLT processor on the same XML data—no doubt a more heavyweight endeavor.

No single answer is right, even for a simple problem such as the weather badge. You can make objective observations about the efficiency of algorithms and speed of execution, but in the real world you eventually end up with certain subjective issues. Is the abstraction afforded by XSLT worth the speed penalty? Is speed even an issue for this type of problem? What makes more sense to run on the browser? How much do you value reliability? Simplicity? Ease of maintenance?

What would I use?

So, if I wasn't presenting the weather badge as a way to teach these techniques, what approach would I use? If I knew with certainty this was a one-off solution, I'd opt for a modified version of Approach 1, where I simply access the XML DOM tree returned by XMLHttpRequest. If this was the first of many badge applications (perhaps another one for stock market figures, and another for generic new feed displays), then I'd want a more abstract approach, like either of the XSLT solutions. And, if I wanted other people to use the badge on many different servers, I'd go with the JSON/Yahoo! Pipes approach. If any of those requirements overlapped, I'd likely head back to the drawing board.



Download

DescriptionNameSizeDownload method
Sample code for this seriesx-xmlajax.zip194KB HTTP

Information about download methods


Resources

Learn

  • XML processing in Ajax, Part 1: Four approaches (Mark Pruett, developerWorks, March 2007): You can solve any programming problem in multiple right ways. Start with the Document Object Model (DOM) tree in this series that looks at four different approaches to create an Ajax weather widget.

  • XML processing in Ajax, Part 2: Two Ajax and XSLT approaches (Mark Pruett, developerWorks, March 2007): Discover two more approaches to the Ajax weather badge. Both use XSLT transformations—one on the server side and the other in the browser.

  • Generate JSON from XML to use with Ajax" (Jack D Herrington, developerWorks, September 2006): Get a deeper introduction to JSON. Convert your XML data into JSON with XSLT V2 and simplify its use with the JavaScript language.

  • Yahoo! Pipes: With this free online service, you can remix popular feed types and create data mashups using a visual editor.

  • JSON.org: Visit this good starting point for general information on JSON.

  • Ajax resource center on IBM developerWorks: Check out this resource on how to create more interactive Web apps with Ajax technologies.

  • IBM XML certification: Find out how you can become an IBM-Certified Developer in XML and related technologies.

  • XML technical library: See the developerWorks XML Zone for a wide range of technical articles and tips, tutorials, standards, and IBM Redbooks.

  • developerWorks technical events and webcasts: Stay current with technology in these sessions.

  • The technology bookstore: Browse for books on these and other technical topics.

Get products and technologies

  • IBM trial software: Build your next development project with trial software available for download directly from developerWorks.

Discuss

About the author

Mark Pruett is a Systems Architect at Dominion. He's a contributing author to O'Reilly Media's book Ajax Hacks and author of the O'Reilly Shortcuts Ajax and Web Services and Yahoo! Pipes.

Report abuse help

Report abuse

Thank you. This entry has been flagged for moderator attention.


Report abuse help

Report abuse

Report abuse submission failed. Please try again later.


developerWorks: Sign in


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Select information in your developerWorks profile is displayed to the public, but you may edit the information at any time. Your first name, last name (unless you choose to hide them), and display name will accompany the content that you post.

Choose your display name

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


Rate this article

Comments

Help: Update or add to My dW interests

What's this?

This little timesaver lets you update your My developerWorks profile with just one click! The general subject of this content (AIX and UNIX, Information Management, Lotus, Rational, Tivoli, WebSphere, Java, Linux, Open source, SOA and Web services, Web development, or XML) will be added to the interests section of your profile, if it's not there already. You only need to be logged in to My developerWorks.

And what's the point of adding your interests to your profile? That's how you find other users with the same interests as yours, and see what they're reading and contributing to the community. Your interests also help us recommend relevant developerWorks content to you.

View your My developerWorks profile

Return from help

Help: Remove from My dW interests

What's this?

Removing this interest does not alter your profile, but rather removes this piece of content from a list of all content for which you've indicated interest. In a future enhancement to My developerWorks, you'll be able to see a record of that content.

View your My developerWorks profile

Return from help

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=XML, Web development
ArticleID=295224
ArticleTitle=XML processing in Ajax, Part 3: JSON and avoiding proxies
publish-date=03182008
author1-email=mark.l.pruett@dom.com
author1-email-cc=dwxed@us.ibm.com

Tags

Help
Use the search field to find all types of content in My developerWorks with that tag.

Use the slider bar to see more or fewer tags.

For articles in technology zones (such as Java technology, Linux, Open source, XML), Popular tags shows the top tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), Popular tags shows the top tags for just that product zone.

For articles in technology zones (such as Java technology, Linux, Open source, XML), My tags shows your tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), My tags shows your tags for just that product zone.

Use the search field to find all types of content in My developerWorks with that tag. Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere). My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Special offers