Implementing Web messaging: Connect Ajax clients to near-real-time data with WebSphere Application Server Community Edition

Applications that depend on near-real-time data -- like stock quotes, health-monitoring systems, sports updates, news alerts, and more -- are becoming more and more popular and essential. This article describes how you can create such an application using IBM® WebSphere® Application Server Community Edition and the Web messaging service component of the Feature Pack for Web 2.0. Step through the tasks and issues you might encounter creating a sports community application that reports live score updates, and learn about the low latency server-to-browser event push technique known as Comet. This content is part of the IBM WebSphere Developer Technical Journal.

Share:

Padmashree Kadaba Sampathkumar (padmashree@in.ibm.com), System Software Engineer, IBM

Padmashree KS is a system software engineer working on the Web 2.0 Feature Pack for WebSphere Application Server for the past two years, with a focus on the Web messaging component. She received her Bachelor’s degree in Computer Science and Engineering from the Siddaganga Institute of Technology, Tumkur. Her areas of interest include Java, J2EE, Dojo and Web2.0 technologies.



25 August 2010

Also available in Chinese

Introduction

The Web messaging service for IBM WebSphere Application Server Community Edition (hereafter referred to as Community Edition) provides support for low latency event push from server to the browser. It is a publish/subscribe implementation wherein the server sends (publishes) the data asynchronously and the subscribers receive the updates they are interested in, enabling the data updates and events to be delivered dynamically to Web browsers from Java™ Messaging Service (JMS) topics.

In the publish/subscribe mechanism shown in Figure 1, the publisher publishes the data to various topics and the subscribers receive updates of interest. For example, Subscriber B receives the updates from both Topic 1 and Topic 2, while Subscribers A and C are only interested in updates from one topic.

Figure 1. Publish/subscribe mechanism
Figure 1. Publish/subscribe mechanism

Client/server communication is achieved through the Bayeux protocol, which is a JSON (JavaScript™ Object Notation) based protocol used for sending asynchronous messages over HTTP with low latency between Web server and browser. JSON is a lightweight data-exchange format. The client side support for the Bayeux protocol is provided by the Dojo Toolkit. The Web messaging service bridges incoming Bayeux requests to JMS topics enabling Web services or JMS clients to publish events to the Bayeux-enabled Web clients. (See Resources for more information on the Bayeux protocol and Dojo Toolkit.)

Bayeux implementations support a specific HTTP method called long polling. Long polling is a technique that optimizes traditional polling to reduce latency. In a long polling request, a client sends a request to a server to retrieve available information. The server checks for any available updates and if there are no updates, the server delays completing the HTTP request until either there is information available or a timeout occurs. If the server doesn’t have any update during a timeout event, it sends back a no-data-available response indicating the client should re-send the request.

To illustrate this, this article walks through a sample application that uses the Web messaging service to connect an Ajax client to data updated in near real time. (For the purpose of this example, the data is randomly generated by the application to simulate real time updates.)

This article assumes that you have knowledge of Web application development, JavaScript, Dojo and some familiarity with Eclipse. In addition, you need to have the WebSphere Application Server Feature Pack for Web 2.0 installed in a Community Edition environment so that you have access to these JAR files for the sample application:

  • WebMsgWASCE.jar is used to provide the Web messaging service to the application. It can be located in the <web20_install>/MessagingService/lib folder.
  • JSON4J.jar is used for the JSON processing required in the application. It can be located in the <web20_install>/JSON4J/lib folder.

See Resources for information on downloading and installing Community Edition and the Web2.0 feature pack.


Understanding polling methods

In the traditional polling method, the client sends a request to the server and always either gets an immediate response if the server has any updated data, or a no-data-available response. The client in turn sends another request after a predetermined interval (poll interval) and this process continues until there is an update from the server. Disadvantages of this technique include an inefficient use of network bandwidth and slow speed.

An alternative to this technique is long polling, which can avoid both these problems. With long polling, the client requests information from the server in a manner that is similar to traditional polling. However, if the server does not have any information, it holds the request and waits for an update to be available for the client. Once the information is available, it sends the response back to the client. The client will then immediately send another request to the server. Thus, the client avoids sending multiple requests for a single response, and improves efficiency of the network bandwidth. (Figure 2)

Figure 2. Traditional polling vs. long polling
Figure 2. Traditional polling vs. long polling

However, long polling also has its limitations:

The term "Comet" was coined by Alex Russell. Comet is a Web application model that enables Web servers to send the data to the client without having to explicitly request it. Comet comes in two flavors: streaming and long polling. The Web messaging feature in the Feature Pack for Web 2.0 uses Tomcat Comet support, available in Tomcat versions 6.0 and later.

  • Two-connection limit: Current HTTP client implementation recommends permitting only two connections between a client and a server. Therefore, if the browser session has two connections already, any further connection requests have to wait until one of the two connections is finished. One way to alleviate this problem is to multiplex all events onto the same server connection. This is effectively done by Comet and Cometd, a multiplex protocol and architecture.
  • Thread per connection: In long polling, the server holds the request until there are any updates or a timeout, which means freezing the connection -- and, in turn, the thread -- until a response or a timeout occurs. Traditional I/O associates a thread with every connection, and usually Web servers have a limited number of threads. Once all threads are used, any new requests are ignored, thinking the server is busy. This poses a problem if you want to scale an application to thousands of clients.

    An alternative is to tie up the thread per request so that it can be returned to the thread pool once the request processing is done. In long polling, however, a client would have an outstanding request to the server at any point of time. Thus, the server again needs one or more threads for every client, and again there are problems with scaling.

    The Tomcat solution to this problem is Advanced IO. Tomcat uses NIO (Non-blocking IO) to keep the connection alive without wasting the waiting threads. To facilitate NIO in a servlet, it uses an event-based API (CometProcessor) that initiates the appropriate reading and writing actions on the open connections at the right time. Comet support enables the servlet to process IO asynchronously, receiving the event when data is available in the server, and writing data asynchronously.

Let’s stop for a moment and review some terms to ensure a basic understanding before moving on:

  • Server push is a mechanism of sending the data from server to client in an asynchronous fashion.
  • Long polling is a polling technique in which the server will delay completing the HTTP response until either an update is ready for the client or a timeout occurs.
  • Comet is a low latency data transfer technique that is event driven and uses the server push mechanism over the HTTP protocol.
  • CometD is a project undertaken by the Dojo Foundation to provide multiple client and server implementations that implement the Bayeux protocol.
  • NIO, also called asynchronous IO, enables the processing of other requests to continue before the current request processing is completed.
  • Bayeux protocol is a protocol for transporting asynchronous messages (primarily over HTTP) with low latency between a Web server and Web clients.

The sample application

The sample application described here (and included with this article for download) is a sports application that subscribes to the channels of two soccer teams and displays updates of the teams’ scores. The sample generates random values as scores to update the Web client to simulate near-real-time events. The Web messaging service library is used to incorporate the publish/subscribe feature in the application.

Before you begin with the application, you need to configure the NIO connector in Community Edition. See the MessagingService documentation available in the installed directory of the Web2.0 feature pack, <web20_install>/MessagingService. With that complete, you can continue with your sample application development using the Web messaging service. The steps presented in the remainder of this article describe this process.

1. Create a Web application

  1. In Eclipse, Create a New Dynamic Web Project (Figure 3) and call it SportsApp (Figure 4). Make sure that Add project to an EAR is unchecked, then click Finish.
    Figure 3. Create dynamic Web project
    Figure 3. Create dynamic Web project
    Figure 4. Name Web project
    Figure 4. Name Web project
  2. Copy the WebMsgWASCE.jar and JSON4J.jar to the lib folder of the project.
  3. Insert the Dojo Toolkit into the Web module. The Web client requires dojox.cometd libraries to connect to the Bayeux server. The Dojo Toolkit can be located at <web20_install>/AjaxClientRuntime_1.X. The result is shown in Figure 5.
    Figure 5. Layout of the application
    Figure 5. Layout of the application

2. Configure Web messaging service

  1. Create a servlet called ScoreServlet and specify the Java package com.ibm.webmsg.sample in the src folder (Figure 6). The servlet extends BayeuxServlet provided by Web messaging service. Click Next.
    Figure 6. Create servlet
    Figure 6. Create servlet
  2. Edit the URL mapping to /scoreServlet and click Finish (Figure 7).
    Figure 7. Edit URL mapping
    Figure 7. Edit URL mapping
  3. In ScoreServlet, set the necessary configuration properties and register the URI that is used for the publish/subscribe mechanism (Listing 1).
    Listing 1. ScoreServlet
    public class ScoreServlet extends BayeuxServlet {
    
    	private static final long serialVersionUID = 1L;
    
    	private static Logger logger = Logger.getLogger(ScoreServlet.class.getName());
    	
    	public void registerURL() {
    		getServletUtil().addClientManager("/scoreServlet", clientManager);
    		logger.log(Level.INFO, "Registered the URL for comet requests.");
    	}
    
    	public void setProperties() {
    		setCometTimeout(60000);
    		setClientPollInterval(0);
    		setRouterType(JMS);
    		setClientsCanPublish(false);
    		logger.log(Level.INFO,
    				"Configured properties for score board requests.");
    	}
    	}

    In the registerURL() method, you register the servlet URI (/scoreServlet). This is used by the Web messaging service for Comet requests. The setProperties() method sets the various configuration parameters of the Web messaging service:

    • setCometTimeout determines how long the request waits at the server side before timing out.
    • setClientPollInterval indicates the interval between the two Bayeux requests. This is usually set to zero in long polling.
    • setRouterType ascertains whether to use in-memory channels or JMS topics. In-memory channels are used for testing.
    • setClientsCanPublish shows whether the client can publish or not. Depending on the requirement, it can be set to true or false.

3. Set up the application to simulate data

The sample application simulates a near-real-time dataset by using a data simulator. The simulator will generate random values to emulate a server generating score updates, and will publish these "scores" to the team channels.

  1. Download and copy the data simulator files into your project. (If you wish, you can also work with with the Java files to modify the simulator logic as needed.)
  2. In the next steps, you will initialize the simulator and the team channels, and then load the servlet during startup. Begin by creating a servlet, SportsInit (Listing 3), under the package com.ibm.webmsg.sample.sports (Figure 8). In Listing 3, init() function sets the data simulator, initializing the data required for sports updates.
    Figure 8. Create SportsInit servlet
    Figure 8. Create SportsInit servlet
    Listing 3. SportsInit servlet
    private static final Logger logger = Logger.getLogger(SportsInit.class.getName());
        
     	public SportsInit() {
            	super();
        	}
        	public void init() throws ServletException {
    
    		super.init();
    		List simData = new ArrayList();
    		
    		simData.add(new SportsData("Team A", "/sports/teamA",0,0,0,0,0,0,0));
    		simData.add(new SportsData("Team B", "/sports/teamB", 0,0,0,0,0,0,0));
    		
    		try {
    			DataSimulatorAppHelper.setDataSimulator(getServletConfig()
    					.getServletContext(), simData, 3, true);
    			logger.log(Level.INFO,	"DataSimulator successfully created");
    		} catch (RuntimeException e) {
    			logger.log(Level.WARNING,"Unable to initialize DataSimulator");
    		}
    		}
  3. Load the servlet on startup by providing the required information in the web.xml file (Listing 4). To specify the order in which you want to initialize the servlets, <load-on-startup> is used. In Listing 4, SportsInit is specified to load on startup and eventually calls the init() method. Figure 9 shows the layout of the Web application after adding the data simulator Java files.
    Listing 4. web.xml illustrating the SportsInit servlet mapping
    <servlet>
    	<servlet-name>SportsInit</servlet-name>
    	<servlet-class>com.ibm.webmsg.sample.sports.SportsInit</servlet-class>
    	<load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>
    	<servlet-name>SportsInit</servlet-name>
    	<url-pattern>/SportsInit</url-pattern>
    </servlet-mapping>
    Figure 9. Web application including data simulator files
    Figure 9. Web application including data simulator files

4. Set up JMS provider

ActiveMQ is the default JMS provider in Community Edition. To provide the resource reference of the JMS provider to the application:

  1. Add the resource reference shown in Listing 5 to the application’s web.xml file.
    Listing 5. Add resource reference in web.xml
    <resource-ref>
    	<description>JMS Broker</description>
    	<res-ref-name>jms/broker</res-ref-name>
    	<res-type>javax.jms.TopicConnectionFactory</res-type>
    	<res-auth>Container</res-auth>
    </resource-ref>
  2. Add the dependency of the messaging provider to the geronimo-web.xml file, shown in bold type in Listing 6.
    Listing 6. Sample geronimo-web.xml
    <web:web-app xmlns:web="http://geronimo.apache.org/xml/ns/j2ee/web-2.0.1"
    	xmlns:dep="http://geronimo.apache.org/xml/ns/deployment-1.2"
    	xmlns:name="http://geronimo.apache.org/xml/ns/naming-1.2" >
        	<dep:environment>
            	<dep:moduleId>
            	  ...
            	</dep:moduleId>
            	<dep:dependencies>
    		<dep:dependency>
    		<dep:groupId>org.apache.geronimo.configs</dep:groupId>
    		<dep:artifactId>activemq-ra</dep:artifactId>
    		<dep:type>car</dep:type>
    		</dep:dependency> 
    		</dep:dependencies>
       	</dep:environment>
        	<name:resource-ref>
    	<name:ref-name>jms/broker</name:ref-name>
    	<name:resource-link>DefaultActiveMQConnectionFactory
    	</name:resource-link>
    	</name:resource-ref>
       </web:web-app>

5. Create the client application

Next, you will create the client side logic of the application using the Dojo Toolkit. Functionally, you will need to create a table to display the score updates, and then initialize the Cometd service and subscribe to the channels. (For the purpose of this example, a custom Dojo widget that subscribes to the topics and displaying the scores in included with the download materials.)

  1. Create a New HTML Page in the Web application, called index.html (Figure 10). Make sure that index.html is included in the <welcome-file-list> in the web.xml file (Listing 7). The index.html code is shown in Listing 8.
    Figure 10. Create new HTML page
    Figure 10. Create new HTML page

    The welcome files mechanism enables you to specify a list of files that the Web container will append to a request for a URL that is not mapped to a Web component. In Listing 7, for example, if the user requests the URL http://host:port/SportsApp, which is not mapped to any servlet, then the file index.html (actually, http://host:port/SportsApp/index.html) is returned. (Eclipse creates a possible list of welcome files in web.xml as you create the project, by default. If you prefer using a different name for your welcome file, you will have to specify it in web.xml.)

    Listing 7. web.xml welcome file list
    <welcome-file-list>
    	<welcome-file>index.html</welcome-file>
    	</welcome-file-list>
    Listing 8. index.html
    <html><head>
    <script src="dojo/dojo.js" djConfig="isDebug: false, parseOnLoad: true"></script>
    <script>
    	dojo.require("dojox.cometd");
    	dojo.require("sample.ScoreWidget");
    	dojox.cometd.init("scoreServlet");
    </script>
    <style type="text/css">
    	h1{
    		color:DarkBlue;
    	}
    	th {
    		color:darkgoldenRod;
    		padding-left:5px;
    		padding-right:5px;
    		padding-bottom:5px;
    		font-size:1.02em;
    		font-weight:bold;
    	}
    	td {
    		color:Darkred;
    		padding-left:5px;
    		padding-right:5px;
    		padding-bottom:3px;
    	}
    </style>
    </head>
    <body>
    <table><thead>
    	<tr>
    		<th>Team</th>
    		<th>Goals</th>
    		<th>Fouls</th>
    		<th>Corner Kick</th>
    		<th>Offside</th>
    		<th>Yellow Card</th>
    		<th>Red Card</th>
    	</tr>
          </thead>
          <tbody>
    	<tr dojoType="sample.ScoreWidget" teamName="Team A" team="teamA" 
    		scoreTopic="/sports/teamA"></tr>
    
    	<tr dojoType="sample.ScoreWidget" teamName="Team A" team="teamB" 
    		scoreTopic="/sports/teamB"></tr>
          </tbody>
    </table>
    </body></html>

    In Listing 8, dojo.js is imported for any Dojo operations, such as loading necessary libraries. Also, parseOnLoad: true is specified to include the dojo parser, which scans the DOM nodes with dojoType attributes and converts it into the declared classes. For example, in the above code, this line is scanned:

    <tr dojoType="sample.ScoreWidget" teamName="Team A" team="teamA" stockTopic="/sports/teamA"></tr>

    and a sample.ScoreWidget type is created, and the widget is started. The type sample.ScoreWidget is the custom dojo widget used for subscription and display purposes.

  2. Next, initialize the Cometd library. Observe that the URI is the one registered in SportsServlet. This line performs this task:

    dojox.cometd.init("scoreServlet");

    You can add some CSS properties now to improve the look and feel of the application.

  3. Create the Dojo custom widget, sample.ScoreWidget. Create a folder called sample in the application’s WebContent (Figure 11) and click Finish.
    Figure 11. Create new folder
    Figure 11. Create new folder
  4. Create a New File in the sample folder and name it ScoreWidget.js (Figure 12). ScoreWidget.js is the dojo widget used for subscribing to channels and displaying the results. The code for the widget is shown in Listing 9.
    Figure 12. Create ScoreWidget.js
    Figure 12. Create ScoreWidget.js
    Listing 9. ScoreWidget.js
    dojo.provide("sample.ScoreWidget");
    dojo.require("dojox.cometd");
    dojo.require("dijit._Widget");
    dojo.require("dijit._Templated");
    
    dojo.declare("sample.ScoreWidget", [dijit._Widget, dijit._Templated],{
    	templatePath: dojo.moduleUrl("sample", "template/ScoreWidget.html"),
    	team: "",
    	name: "",
    	goal: "--",
    	foul: "--",
    	cornerKick: "--",
    	offside: "--",
    	yCard: "--",
    	rCard: "--",
    	scoreTopic: "",
    	teamName:"",
    		
    	postCreate:function(){
    		dojox.cometd.subscribe(this.scoreTopic, this, "scoreUpdate");
    		this.name.innerHTML = this.teamName;
    	},
    	scoreUpdate:function(message){
    		this.goal.innerHTML = message.data.goal.toFixed();
    		this.foul.innerHTML = message.data.foul.toFixed();
    		this.cornerKick.innerHTML = message.data.cornerKick.toFixed();
    		this.offside.innerHTML = message.data.offside.toFixed();
    		this.yCard.innerHTML = message.data.yCard.toFixed();
    		this.rCard.innerHTML = message.data.rCard.toFixed();
    	}
    });

    In Listing 9, Cometd libraries are used to subscribe to the topics to receive score updates. The call back function is scoreUpdate, which is called when updates are available in the topic. This function performs the task of displaying the data in the table.

  5. The widget also has a template named ScoreWidget.html. It is customary to place all the template files in a template folder, so create a folder called template for this widget in the sample folder (Figure 13), and then add a New File to it, called ScoreWidget.html (Figure 14).
    Figure 13. Create template folder
    Figure 13. Create template folder
    Figure 14. Create ScoreWidget file
    Figure 14. Create ScoreWidget file

    The actual ScoreWidget.html template code (Listing 10) defines a set of columns used to display the score updates. In this code, dojoAttachPoint is used to enable the widget's JavaScript code to reference the DOM nodes in your widget's template. Dojo will automatically scan the template for these attach points and then create the contextual this variable with the same names. Figure 15 shows the structure of the completed application.

    Listing 10. ScoreWidget.html
    <tr dojoAttachPoint="team">
    	<td dojoAttachPoint="name" style="font-weight:bold;" > </td>
    	<td dojoAttachPoint="goal">--</td>
    	<td dojoAttachPoint="foul">--</td>
    	<td dojoAttachPoint="cornerKick">--</td>
    	<td dojoAttachPoint="offside">--</td>
    	<td dojoAttachPoint="yCard">--</td>
    	<td dojoAttachPoint="rCard">--</td>
    	</tr>
    Figure 15. Layout of the completed Web application
    Figure 15. Layout of the completed Web application

6. Deploy and run the application

If you have the Community Edition plug-in installed, you can run the Web application project from Eclipse. Otherwise, you can export the project using the SportsApp.war file and deploy it externally. Alternatively, you can also download the WAR file included in the download material and deploy that on the server.

To deploy the application:

  1. Open the Community Edition console (http://<host>:<port>/console) in your browser and navigate to the Deploy New portlet.
  2. Browse to the SportsApp.war file and click Install.
  3. You can run the application at http://<host>:<port>/SportsApp, where <host> and <port> are specific to your environment.

Figure 16 shows a screen capture of the Sports Update application.

Figure 16. Sports Update sample application
Figure 16. Sports Update sample application

Common issues

  • The type org.apache.catalina.CometProcessor cannot be resolved.

    This issue surfaces when you add ScoreServlet, which implements BayuexServlet. BayeuxServlet in turn implements CometProcessor of Tomcat. Hence, you are required to add Tomcat’s catalina.jar into the build path. Right-click on the project and go to the Build path. Select the Libraries tab, then Add External JARs, and then browse to the catalina.jar location and click OK. This should resolve the problem.

  • No updates in the browser.

    One of the reasons might be that the NIO connector is not configured in your server. Refer to the documentation for the Web messaging component to configure the NIO connector. Another possible cause might be the port being used. If NIO is configured as an additional listener, use the port that is configured for NIO.


Conclusion

This article explained how to add the Web messaging service component of the WebSphere Application Server Feature Pack for Web 2.0 to a Web application for WebSphere Application Server Community Edition. The publish/subscribe mechanism was discussed, along with some background on why long polling is used for Web messaging. To illustrate the general steps involved, a Java EE Web application that uses the Web messaging feature was described (and provided as a download) that demonstrated support for the low latency server push from server to the browser.

Hopefully, this information will be helpful to you in your own Web application development.


Downloads

DescriptionNameSize
Code sampleSportsApp.zip1.8 MB
Code sampleDatasimulator.zip4 KB

Resources

Learn

Get products and technologies

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere, Web development, Open source
ArticleID=512976
ArticleTitle=Implementing Web messaging: Connect Ajax clients to near-real-time data with WebSphere Application Server Community Edition
publish-date=08252010