A mashup is a Web application that integrates content from more than one source and delivers it for presentation in a single page. The server makes requests to each content source, parses the information it receives, and combines the results into a page to send to the browser, as Figure 1 shows.
Figure 1. Mashing up content from multiple sources
Figure 2. Interactive data display with Ajax
An Ajax mashup is a hybrid Web application. It uses Ajax techniques to present a rich UI that updates itself in place using content that it retrieves asynchronously from multiple sources. The server sends an initial page to the browser, which then makes calls to retrieve updated content. These calls can be made directly to the third-party sources from the browser or back to the originating server, which acts as a proxy for the third-party content.
When the elements comprising the current browser environments were designed, Ajax mashups were not on anybody's radar. Nothing was built into the browsers, into the Hypertext Transfer Protocol (HTTP), or into HTML or XHTML that was specifically designed to accommodate the browser's asynchronous retrieval of content from multiple sources in a secure and robust manner. Some features in the World Wide Web Consortium (W3C) HTTP specifications that might have been used for mashups, such as Document Object Model (DOM) Level 3 Load and Save Specification, were either not fully implemented or not implemented at all by a majority of browsers.
Dynamic HTML (DHTML) was not initially used in combination with dynamically retrieved content. Both the presentation and data elements of a dynamic Web page were delivered together along with scripts to manipulate them. The scripts would display, hide, move, create, and destroy document objects to create dynamic effects, but any time more data was needed from the server, the page would be replaced with a new one. Data flow was synchronous with page reload.
Consequently, developers who wished to build the kind of hybrid Web application that we now call a mashup had to take the available technology and find ways to stretch it to fit their needs. Two approaches were taken to allow the browser to retrieve content without reloading the page: embedding an external transport mechanism and using browser-native objects to perform transport duties.
An early solution was Microsoft's Remote Scripting, which used a Java™ applet that exchanged XML-formatted messages with server-side components. This approach quickly became unwieldy because of vendor squabbles as well as Java Virtual Machine (JVM) and security model differences.
Microsoft later built the
object, whose designers had the expectation that it would be used only with
Microsoft® Outlook® Web Access (OWA). The object was initially available
only to Windows® Internet Explorer® users and subsequently not widely
used until years later, when Mozilla and Safari adopted it. Originally an external
Microsoft ActiveX® object, current implementations are native objects within
the browser. Despite its name, the XHR object can transfer data in any format and
is not limited to valid XML.
Many developers use the XML communication features of Macromedia Flash to build
embeddable components to communicate with the server. The
flex.net.socket objects provide abilities
similar to XHR but with additional communication control and XML parsing
Because of the problems and dependencies associated with external transport mechanisms, the internet development community has collaborated in the discovery and development of several browser-native remote calling methods.
- Using a hidden
iframeelement to load external content: The
iframeis then accessed through the DOM to extract the content from the document it has loaded. You can specify any parameters in the URL
querystringor dynamically create a form that posts to the service with the
iframeas a target. This method is compatible across a wide range of both current and older browsers.
- Using an
imgelement to send requests for content: The server performs its task using the parameters from the URL's
querystring, and then returns encoded content in a cookie. This method is limited in the amount of data that can be easily communicated, because both
querystringand cookies are limited in size.
- Dynamically creating a script element in the DOM of the current page:
Upon loading, code that the server supplies is immediately executed.
The server uses parameters from the URL
See Resources for links to detailed information on these tools and techniques.
To have a Web page retrieve content from third-party sources, you must circumvent
the Same Origin Policy. The commonly used exception that is not restricted by the
Same Origin Policy is the
<script> tag technique,
whereby you append a
<script> element to a page's
DOM, causing it to load and run the code it finds at the URL that the element's
src attribute specifies.
<script> tag to run scripts from multiple
sites presupposes a high level of trust between all sites involved, because all
such scripts run in the same execution and security context and therefore can
possibly gain access to information and cookies from the other sites.
The workarounds currently in wide use to enable Ajax mashups each come at some cost. When stretching a browser's designed limits, you affect other aspects of the application's overall operation. Doing so typically causes an application to become either less secure or less scalable.
When restricted by the browser's Same Origin Policy, the same server that hosts the application must take on the task of fetching the third-party content and sending it to the client. The server acts as a client to the third-party service in addition to its usual server function.
Using the server as a proxy for every client transaction means that a large number of users could cause undue server load. Applications using this technique would need to be designed to be scalable on the server side, using multiple coordinated servers to handle the request load.
Use of the
<script> tag to circumvent the
Same Origin Policy allows the client to retrieve content from third parties.
This functionality eliminates the server bottleneck to scalability, because
each additional client takes on its own content-gathering role.
The scalability benefit of the
comes at the cost of sidestepping the Same Origin Policy security model,
introducing potential attack vulnerabilities:
- Cross-site cookie access becomes possible: Scripts from one site can access cookies from another site.
- There is no opportunity to inspect the retrieved code for safety issues before running it: The code runs immediately upon loading.
Clearly, the tools that browsers currently provide for mashups are insufficient to allow you to build applications that are both scalable and secure. Developers must find solutions that work both now and in the long term.
A more recently developed content-retrieval technique employs communication
between a page's script and a hidden
src URL's fragment identifier (the part
of the URL that comes after the # sign). Scripts in the parent page and
iframe can set each other's fragment
identifiers despite coming from different origins. An agreed-upon communication
periodically fire routines to check for changes in the fragment identifier.
Because the scripts must know each other's addresses and they must collaborate between themselves to agree on a protocol, trust is ensured. Because any server interaction is local to each component and separate from the inter-script communication, cookies are not exposed.
While still imperfect (for example, it relies on an anomaly that is not a designed behavior, and polling for changes is inferior to having an event fire in response to a change), this solution comes closer to providing browser-native, secure, in-page, cross-domain communication than any other.
Browser manufacturers and the development community are currently discussing several potential ways to modify elements of the browser environment to make it purpose-built for Ajax mashups. The Web Hypertext Application Technology Working Group (WHATWG) has a proposal in section 7.3 of its Web Applications 1.0 Working Draft for a mechanism called Cross Document Messaging. The Opera browser already implements this feature. It specifies a method of collaborative communication between DOM objects from different domains that allow the receiver of a message to choose which messages to respond to based on their origin.
Ian Hickson (who was at Opera and is now at Google) has proposed cross-site
extensions to the existing
His proposal consists of several modifications to the way requests are made,
including restrictions on header control and an access-control mechanism.
Several outstanding proposals are available to help address this quandary:
JSONRequestproposal: Browsers implement a new object that acts much like the existing
XMLHttpobject with several modifications:
JSONRequestwould be exempt from the Same Origin Policy.
- A minimal set of HTTP headers would be used, reducing the overall size of requests.
- No cookies would be transferred, ensuring that cross-site cookie issues are avoided.
JSONRequestwould accept only valid JSON text, which would ensure that raw executable code could not be sent for execution.
- After a communication failure, random delays are introduced before retry to frustrate certain classes of attacks.
- Each request would return a sequence identifier, allowing asynchronous responses to be associated easily with their original requests.
- Specific support for duplex connections would enable the server to asynchronously initiate communications through an open communications channel.
<module>tag proposal: A new HTML tag partitions a page into a collection of modules that are secure from each other but can communicate safely:
<module>tag would be able to access third-party resources, exempt from Same Origin Policy.
- Cooperative communication between page and module would be available only through specific interfaces. Modules would not be able to communicate with each other -- only with the page. A page can choose to facilitate communication between modules.
- Restrictions are proposed to ensure that modules and pages cannot interfere with one another's display, causing security issues.
- Content restrictions header: Gervase Markham proposes a content restrictions header specification that would allow authors to express their full intent on how their content should interact with content from other sites. A compliant implementation would submit a content restrictions header containing a policy string.
- W3C Access Control List (ACL) System: The W3C ACL System could be used as a model for an ACL-based system to govern access to HTTP-served resources in Ajax mashups.
- Cross-domain.xml: Flash objects look for a file called cross-domain.xml on the server before they attempt to access their specified URL. This file specifies which sites can host applications that access the services provided on that server. Many Web service providers already implement this file.
See Resources for links to detailed information on these proposals.
As developers, we all have a stake in the outcome of these discussions. By joining the conversation, you can help to design the most flexible yet secure improvements to the browser that will allow all to build robust and secure rich Web applications. I encourage you to seek out browser vendors and organizations advocating browser advances and join in:
- Get involved with industry associations and working groups.
- Interact with browser and tool vendors in newsgroups and forums.
- Seek out and engage key players in the industry.
You can find links to starting points in the Resources for this article.
- developerWorks XML zone: Learn all about XML at the developerWorks XML zone.
- The story of XMLHTTP: See Alex Hopmann's history about the beginnings of XMLHTTP.
- DOM Level 3 Load and Save Specification: Read the W3C spec about how programs and scripts can dynamically load the content of an XML document into a DOM document and serialize a DOM document into an XML document.
- Same Origin Policy: On the Mozilla Web site, find information about this policy to prevent documents or scripts loaded from one origin from getting or setting properties of a document from a different origin.
- Cross-site scripting (Paul Lee, developerWorks, September 2002): Learn more about these scripting issues.
- Remote scripting using a servlet (Erik Hatcher, developerWorks, February 2001) and Sending rich messages between client and server using asynchronous messaging (Erik Hatcher, developerWorks, May 2001): See the articles for more information about these techniques.
- Ajax Transport Layer Alternatives: Get Brent Ashley's presentation notes, now available on his Web site.
- Iframe/fragment identifier discovery: For more information about fragment identifiers, see James Burke's original blog.
- Dojo Toolkit's implementation: Read how the Dojo Toolkit implements the fragment identifier transport method.
- Cross-document messaging: Read more about WHATWG's proposal.
XMLHttpRequest: Read Ian Hickson's updated proposal.
<module>tag: Read Douglas Crockford's proposals about a solution to mashup security issues.
- Content restriction headers:Read Gervase Markham's proposal for how a browser might differentiate legitimate scripts from malicious ones.
- ACL System: Read more about the W3C's dynamic, file-level access control (ACL) system, including ACL storage and query mechanisms, plus the availability and use of this data on the semantic Web.
XMLSocketobject and cross-domain.xml files: Learn how the XMLSocket object and cross-domain policy files enable communication between a computer running the Flash Player and another server identified by an IP address or domain name.
- IBM XML certification: Find out how you can become an IBM-Certified Developer in XML and related technologies.
- XML technical library: See the developerWorks XML Zone for a wide range of technical articles and tips, tutorials, standards, and IBM Redbooks.
- developerWorks technical events and webcasts: Stay current with technology in these sessions.
Get products and technologies
- IBM trial software: Build your next development project with trial software available for download directly from developerWorks.
- Participate in the discussion forum.
- The OpenAjax Alliance: Help to shape the future of Ajax with this industry group focused on Ajax advocacy.
- Ajaxian: With this great resource, keep up with developments in Ajax and the front-end widgets that use it.
- XML zone discussion forums: Participate in any of several XML-centered forums.
Brent Ashley is a consultant and scripting specialist in the Toronto area. Involved in computers and technology since 1979, he has been active in developing and promoting rich Web application development techniques such as Ajax and remote scripting since 1999. Brent talks about technical subjects on his blog. You can reach Brent at email@example.com.