Using the WebSphere Application Server Feature Pack for Web 2.0 and Mobile to view web application usage patterns and other analytics data

Knowing how users navigate and browse through your website can be valuable in guiding and justifying efforts to improve the site. For example, knowledge about page statistics might guide you to move the most frequently accessed pages to higher levels in the site, statistics on mouse clicks can identify rarely-used buttons, and long idle times could point to pages needing simplification. This article explains how to make use of new features in the Dojo Toolkit and IBM® WebSphere® Application Server Feature Pack for Web 2.0 and Mobile to generate analytics data, and offers advice on presenting the data in your own application. This content is part of the IBM WebSphere Developer Technical Journal.

Share:

Andy Dingsor (dingsor@us.ibm.com), Advisory Software Engineer, IBM

Author photoAndy Dingsor is an advisory software engineer at IBM. He has developed a wide variety of hardware and software products, including a video conferencing system, PCMCIA Wireless WAN adapters, a TCP/IP network load balancer, and the WebSphere Application Server. Andy holds seven U.S. patents on FM radio modulation, networking, and software techniques. He occasionally commutes to work by bicycle.



Eric Covener (ecovener@us.ibm.com), Advisory Software Engineer, IBM

Author photoEric Covener works at the IBM development lab in Raleigh, North Carolina. He currently works on IBM HTTP Server and has worked previously on WebSphere Commerce, Contexts, and Dependency Injection for Java (CDI), and WebSphere Application Server Feature Pack for Web 2.0 and Mobile. Eric is a member of the Apache Software Foundation and a PMC member on the Apache HTTP Server, Apache OpenWebBeans, and Apache Portable Runtime projects.



22 June 2011

Also available in Japanese Spanish

Introduction

Instrumenting a web application to report user navigation and browsing activity can generate valuable statistics for improving the application and guiding future development efforts. There are three major steps, represented by three message flows, that enable the gathering and presentation of this type of data, known as analytics, as shown in Figure 1:

  1. Instrument your primary application with Dojo analytics to enable Dojo to generate analytics messages.
  2. Collect analytics data with the analytics service so that this real-time information is saved in a log file for subsequent analysis.
  3. Present analytics data with the Dojo Diagrammer, a widget that displays the statistical data.

Combining all three steps brings powerful insight to interpreting how users are navigating and exploring the features of your web application.

Figure 1. Overview of message flows
Figure 1. Overview of Message Flows

This article explains how you can use new features in the Dojo Toolkit and IBM WebSphere Application Server Feature Packfor Web 2.0 and Mobile to generate analytics data, and how you can present this data in a useful way in your own application.


1. Instrument your primary application with Dojo analytics

The Dojo Toolkit is an open source cross-platform JavaScript™ library. It contains hundreds of functions and utilities, designed to simplify JavaScript programming and hide browser-specific behaviors, and contains includes as a large library of widgets.

The Dojo analytics package, dojox.analytics, is part of the Dojo Toolkit. It has been available in all versions of Dojo since version 1.1, including the version that is bundled with the IBM WebSphere Application Server Feature Pack for Web 2.0 and Mobile V1.1.

Dojo analytics provides a tracking and logging mechanism that pushes data from a browser to a web service. It can send data for various built-in sources of analytics data, and enables you to define and emit your own events. Dojo analytics sends this data to a collection service that is independent of the primary application.

To enable these features, the primary application can be augmented with a few Dojo analytics statements. JavaScript code is added to the HTML or JavaScript of the primary application in which these three items can be specified:

  • Where the messages are sent.
  • Which messages are sent.
  • How frequently the accumulated messages are sent to the service.

Listing 1 shows an example of this code, in which sendInterval specifies how often data is sent to the service (in milliseconds) and analyticsUrl specifies the destination URL to which the data is sent.

Listing 1
<script type="text/javascript" 
src="dojotoolkit/dojo/dojo.js" 
djConfig="djConfig: true, 
sendInterval: 5000, 
analyticsUrl: 'http://server/path/to/dataLogger'"> 
</script>

Dojo can generate analytics statistics for several types of events, such as page accesses and mouse clicks. These events can be enabled individually by including a dojo.require() statement for specific plug-ins. This is a valuable feature that enables the granularity of statistics to be fine-tuned. The statements in Listing 2 show how to enable a few of these events. A complete list is available on the Dojo Toolkit website (see Resources).

Listing 2
<script language="JavaScript" type="text/javascript"> 

// Access analytics. 
dojo.require("dojox.analytics"); 

// Enable mouse-over events. 
dojo.require("dojox.analytics.plugins.mouseOver"); 

// Enable mouse-click events. 
dojo.require("dojox.analytics.plugins.mouseClick"); 

</script>

In addition to pre-defined events, Dojo also enables custom log messages to be placed throughout the code. These provide a powerful way to record variables and state information in the log. For example, this Dojo console.rlog message writes a custom message into the server log.

console.rlog("console.rlog says datastore created");

The above line of code generates this output in the log file:

[2011-0502-1643-0072] {"plugin":"consoleMessages",
"data":["rlog","console.rlog says datastore created"]}

Each message in the log is prefixed with the timestamp the message was received by the server. In some cases, it is valuable to know the timestamp on the client browser. To log a custom message that includes the timestamp from the client browser, you can code:

dojox.analytics.addData("timestamp",
[new Date().getTime(), "on-load executed"]);

The code above generates this output in the log file:

[2011-0502-1643-0072] {"plugin":"timestamp",
"data":[1304368975670,"on-load executed"]}

Listing 3 shows an example that shows how to log a message whenever an accordion tab is clicked. It connects all instances of an AccordionContainer to an addData() call.

Listing 3
dijit.registry.byClass("dijit.layout.AccordionContainer")
.forEach(function(widget, index, hash) {
dojo.connect(widget, "selectChild",
function(selectedChild) {
dojox.analytics.addData("mynav", 
selectedChild.title);
});
});

Similar instrumentation can trigger a log message whenever a mobile “view” becomes active. The code in Listing 4 connects the addData() call to the initial view and to each additional view.

Listing 4
var mobileAnalytics = 
function(moveTo, dir, transition, context, method) { 
dojox.analytics.addData("mynav", moveTo); 
} 

dojo.addOnLoad(function() { 
dijit.registry.byClass("dojox.mobile.View") 
.forEach(function(widget, index, hash) { 
dojo.connect(widget, 
"onAfterTransitionIn", mobileAnalytics); 
}); 
dojox.analytics.addData("mynav", "main"); // initial view 
});

See Resources for more information on generating messages.


2. Collect analytics data with the analytics service

A web service that is independent of the primary web application collects data from browsers and writes it to a log file on the server machine. The WebSphere Application Server Feature Pack for Web 2.0 and Mobile includes an application written in Java (a JAX-RS service) specifically for this purpose. Here, you’ll see how easy gathering analytics data is using the JAX-RS service provided by the feature pack.

The analytics service, appsvcs-analytics, is included in two directories in the feature pack; one is for production purposes and the other is for demonstration. For the purpose of this article, install the demonstration EAR file:

.../web2mobilefep_1.1/samples/web20mobileshowcaseSample/showcase.ear

After installing and starting showcase.ear, browse to the application demonstration page, which simulates the primary application, at http://localhost:8080/appsvcs-analytics/ (Figure 2).

Figure 2. Browse to the appsvcs-analytics demonstration page
Figure 2. Browse to the appsvcs-analytics demonstration page

Move the mouse pointer around, and expand and collapse some elements. Moving the mouse triggers the underlying Dojo to generate analytics statistics and send them to the appsvcs-analytics application. After a minute or two, analytics statements will appear in the application log file on the server machine. In this example, the log file is analytics_default.log.

Listing 5 shows the format of the contents of a typical log file. As mentioned earlier, each line begins with a timestamp in square brackets, indicating when the message was received by the server. The remainder of the line is a string representation of the data received from dojox.analytics on the browser, formatted as a JSON object.

Three typical log messages are included in Listing 5. The first message is generated when a window launches. It contains a huge amount of information about the browser window and the page being accessed. The second message is much less verbose and is generated when the user is idle. The final message has a moderate bit of information, and is generated by a mouse-over event.

Listing 5
[2011-0502-1643-0072] {"plugin":"window", 
"data":{"outerHeight":1057, 
"offscreenBuffering":true, 
"screenTop":25, 
"name":"", 
"outerWidth":1400, 
"frameElement":null, 
"innerWidth":1668, 
"screenLeft":0, 
"defaultStatus":"", 
"console":{"memory":{"usedJSHeapSize":0, 
"jsHeapSizeLimit":0, 
"totalJSHeapSize":0}, 
"profiles":[]}, 
"PERSISTENT":1, 
"devicePixelRatio":1, 
"pageXOffset":0, 
"innerHeight":1159, 
"pageYOffset":0, 
"length":0, 
"scrollX":0, 
"scrollY":0, 
"screenX":0, 
"closed":false, 
"screenY":25, 
"TEMPORARY":0, 
"status":"", 
"defaultstatus":"", 
"location":{"pathname":"\/appsvcs-analytics\/", 
"port":"8080", 
"hostname":"localhost", 
"origin":"http:\/\/localhost:8080", 
"protocol":"http:", 
"search":"", 
"host":"localhost:8080", 
"href":"http:\/\/localhost:8080\/appsvcs-analytics\/", 
"hash":""}}} 

[2011-0502-1643-0072] {"plugin":"idle", 
"data":"isActive"} 

[2011-0502-1643-0072] {"plugin":"mouseOver", 
"data":["sample", 
{"target":{"spellcheck":true, 
"className":"claro", 
"localName":"body", 
"id":"", 
"textContent":"\n\n Welcome to the Analytics Service Application", 
"lang":""}, 
"x":"468", 
"y":"2"}]}

Tracking individual user sessions with custom initialization parameters

Analytics messages from all users are written into the same log file. By default, individual user sessions are not identified in the log file. If multiple users browse the application at the same time, the analytics messages are mixed together. To associate analytics with individual users, the appsvcs-analytics application can write the browser IP address, the JSession ID, or both into the log file with each message.

To enable this feature, define a custom initialization parameter in the web.xml file for appsvcs-analytics. (Refer to README.html included in the appsvcs-analytics.war file in the feature pack for details on using this parameter.) For example, the custom parameter in Listing 6 enables the collection of both the browser IP addresses and the HTTP session identifiers in the log.

Listing 6
<init-param> 
<param-name>
com.ibm.ws.mobile.appsvcs.analytics.
logger.LocalFileLogger.LogFormat
</param-name> 
<param-value>CLIENT_IP,CLIENT_SESSION_FORCED</param-value> 
</init-param>

After you restart the server, refresh the browser, and generate more statistics by performing actions that trigger logging, the IP and session ID appear in the log as shown in Listing 7.

Listing 7
[2011-0503-1447-5680] 
127.0.0.1 
224DA705F7233CE06CE3735662FDE2A6 
http://localhost:8080/appsvcs-analytics/ 
{"plugin":"idle","data":"isActive"}

3. Present analytics data with Dojo Diagrammer

With your analytics data available, the final step is to create a presentation application. This application will perform three tasks:

  1. Read the contents of the events log file generated by the analytics service.
  2. Parse the log file to extract information you find useful.
  3. Presents the information in an understandable way.

The application can be written as a standalone program or as a web application in an environment that has read access to the log file.

For example, you can create a Java JAX-RS web application to run as a service in the same web container as your primary application. The program can open and parse the log file, extract the data, organize it according to HTTP session ID, convert the data into a JSON hierarchical format, and return the data to the browser. This data format is expected by the IBM Dojo Diagrammer, which is available in the IBM WebSphere Application Server Feature Pack for Web 2.0 and Mobile. Listing 8 gives an example of this data format.

Listing 8. JSON data passed to Dojo Diagrammer
{ "identifier" : "Name",
  "items" : [ { "Name" : "tab1",
        "children" : [ { "_reference" : "tab2" } ]
      },
      { "Name" : "tab2",
        "children" : [ { "_reference" : "tab3" },
            { "_reference" : "tab4" }
          ]
      },
      { "Name" : "tab3",
        "children" : [ { "_reference" : "tab2" } ]
      },
      { "Name" : "tab4",
        "children" : [ { "_reference" : "tab1" } ]
      }
    ],
  "label" : "Name"
}

The browser can present this JSON data to Dojo Diagrammer as a data store and the Dojo Diagrammer renders the image. The rendering in Figure 3 presents log data that was generated by manually clicking in a random pattern through an instrumented web page.

Figure 3 shows the resulting graphical representation of the steps taken by a user manually clicking and navigating through a site. Several page artifacts, such as main, tab1, post1, and so on, are shown as boxes. The boxes are linked together with lines and arrows; the arrows indicate the direction the user progressed by clicking through the site. This rendering shows the value of a graphical representation when studying the paths taken by a user through a website.

Figure 3. Analytics data presented by Dojo Diagrammer
Figure 3. Analytics data presented by Dojo Diagrammer

Conclusion

This article described three basic steps for displaying analytics statistics for an application: instrumenting and enabling an application for analytics, saving it in a log file, and presenting it in a meaningful format. The key tasks are deciding which elements to trace, and how to convert that information into a hierarchical JSON format. After that, tying the pieces together is straightforward.

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere, Web development
ArticleID=681617
ArticleTitle=Using the WebSphere Application Server Feature Pack for Web 2.0 and Mobile to view web application usage patterns and other analytics data
publish-date=06222011