Introduction

As most of us know, logging in an application helps identify many problems that occur during a system lifetime. Application logging can involve documenting anything from system exceptions to application tracing to debugging. This article is about building an asynchronous logging application using log4j and JMS. We will build the logging application using WebSphere™Studio Application Developer and WebSphere MQ. The article will benefit application architects and developers.

Issues with logging in a distributed environment

In a distributed Web application, with the application crossing the machine barrier, centralized logging is a challenge. Architects have been forced to incorporate more sophisticated logging solutions into their design.

Logging in a distributed environment introduces many issues:

• The logging solution should be easy for a developer to use, reusable across different applications, and extensible to meet new needs.
• Logging should be centralized to a single point to allow an administrator to access it from one place.
• Logging should have little affect on application performance.
• The logging solution should filter messages, via configuration, by type to enable specific events to be triggered. These include things like persisting the message and performing alert notification, such as paging an administrator.
• The logging environment should allow for centralized and local logging by identifying different types.

Creating a simple logging framework can solve most of these issues. In our sample application, we will configure Jakarta's log4j to work with JMS and WebSphere MQ. By using these technologies, we solve the issues stated above.

Benefits of using Jakarta log4j

By using log4j, developers make use of an open source easy-to-use interface. log4j lets developers use different message formats allowing for message structure, such as XML to be used. This makes the framework extensible. log4j also allows messages to contain different target outputs based on a configuration filter. This allows for some level of control and maintenance. Please see more information about log4j .

Combining log4J with JMS and WebSphere MQ lets us expand our logging framework into the J2EE world and the enterprise. Applications can write a message to a MQ queue. Using JMS, the application can run under any compliant J2EE Container. This makes the logging framework reusable across different J2EE applications. More important, the log messages are written asynchronously, which means the application does not have to wait for a response. It can write the message and continue, thus allowing the logging solution not to hinder performance.

WebSphere MQ (formerly MQSeries) is an industry-leading message-oriented-middleware solution. WebSphere MQ is available on multiple platforms. WebSphere MQ implements the JMS standard defined in the J2EE standard. Using WebSphere MQ, messages can go to a single centralized point. Moreover, you get all the benefits of a seasoned messaging server. WebSphere MQ can interact with monitoring tools that perform many administrative functions such as paging an administrator. You can also write programs to read messages from a queue and log it into a relational database, flat file, bit-bucket, or external systems. Furthermore, you can adopt certain filtering mechanisms to ignore certain messages and alert the proper administrator based on severity.

Once the technologies are chosen, you can use it to create a logging architecture as shown in Figure 1 below.

The details of the Logging Server and the Logging Application are separated from the application. Applications now write message logs and don't care where they go.

Back to top

Sample technology

In our sample, we will configure log4j to work with JMS as shown in Figure 2 below.

The products used for the configuration described in this article are listed below. A familiarity with the basic operations of these products is assumed.

• IBM®WebSphere Studio Application Developer, Version 4.0 for Windows®
• IBM MQSeries® Version 5.2 for Windows NT® and Windows 2000
• MA88: MQSeries classes for Java™ and for Java Message Service (JMS)
• Jakarta log4j
• WebSphere Application Server Advanced Edition 4.03
• IBM HTTP Server 1.3.19.2

Back to top

Building the sample application

In this section, we will walk through the steps needed to get our sample working. First, we configure and test the example in WebSphere Studio Application Developer.

1. Importing an EAR file
2. JNDI and JMS caching classes
3. Log4j JMS Appender class
4. Simple XML Layout and Message classes
5. Log Wrapper Class
6. Log4j configuration files
7. Test servlet and JSP to write to log
8. Test servlet and JSP to read log for verification
9. Run JMSAdmin script for JNDI and JMS Objects
10. Run the WebSphere test environment
11. Run WebSphere administrative client to create external JMS and JNDI bindings
12. Test the application

Step 1. Importing an EAR file

We are going to create a simple Web application that logs to WebSphere MQ. The first step is to create a Web and an EAR project.

1. From the Application Developer Workbench, select File => Export .
2. From the Export Wizard, select EAR and click Next .
3. Navigate to the EAR file, give the EAR Project a name, and click Next .
4. Under the Manifest Class-Path page, highlight LOG4JDemoWeb.war , select log4j.jar , and click Next . See Figure 3 below.
5. Keep the default name for the Web project and click Finish .

Please see more information on utility jar packaging .

Step 2. JNDI and JMS caching classes

Next, we will use a few JNDI and JMS convenience classes for several reasons. You can download the classes below .

1. Encapsulate JNDI lookups and hide the details of the JNDI API.
2. Cache QueueConnectionFactories and Queues to avoid JNDI roundtrips
3. Cache a hidden session to activate WebSphere MQ JMS Session pooling.
   Note : As long as there is an active session, MQ JMS will use a session pool.

Since performance is a key consideration, caching JMS objects helps.

After examining the downloaded classes, we can see our caching mechanism uses Service Locator and Service Factory classes. These classes implement common J2EE design patterns (please see Core J2EE Patterns below). The classes are located under com.ibm.logdemo.servicelocator and com.ibm.logdemo.servicefactory.
Note : These classes are for demo purposes; more design should be placed on these aspects of the architecture.

ServiceLocator class is located in the com.ibm.logdemo.servicelocator package. It is used to hide JNDI complexity. Classes needing to lookup JNDI Objects can call the lookupObject method. (Note, I lazy initialized the JNDIServiceLocator class to allow the class to be serialized.) Please see the lookupObject method .

Under the com.ibm.logdemo.servicefactory package, there are a few classes that implement JNDI Object caching. All the factory classes extend the ServiceFactory class that implements common caching behavior. The JMSServiceFactory class caches JMS Connection Factories, Queues, and Topics. It also caches a hidden Session to enable WebSphere MQ Caching. Please examine the classes from the downloaded source.

Step 3. log4j JMS Appender class

Next, we create a custom log4j Appender. An Appender class allows log4j to redirect the log to different outputs. There is an existing TOPIC based Appender; however, for this demo, we will make our own JMS Queue based Appender for simplicity. The log4j Appenders exist for download on the Jakarta site; please see more information on log4j Appenders .

In order to write an Appender, we need to extend log4j's AppenderSkeleton class. By doing this, your application can intercept the Logging Event and redirect it to your desired output. The JMSQueueAppender class is located under the com.ibm.logdemo.appender . Please see the source code that illustrates the key methods in the class.

This is the heart of the log4j and JMS interaction. Here we can take the event and use our JMS service classes to redirect our output. We also set the requiresLayout() method to true because the message will have custom layout (we will see this later).

If we want to extend this example further, rather than hard code the queue factory name and queue, you can have messages routed to different queues based message types, however, we can accomplish this better using a log4j configuration file (see the Jakarta Web site ). The close method is there in your appender to open some output sources that need to be closed; for this example, we are not utilizing this.

Step 4. Simple XML Layout and Message classes

For this example, we will create a small custom XML type layout to demonstrate how log4j message formats may be customized. The intent here is not to show XML best practices, just log4j functionality.

In order to create our own layout, we need to extend the log4j Layout Class. This way, we can intercept the logging event and format it on our own. Before doing this, we will illustrate an interface and a value object class. The interface gives the Layout class a way to grab the XML message from the value object. The interface provides a convenient XML message similar to the toString() method of the Object class. The XMLMessage interface is listed below. It belongs in the com.ibm.logdemo.message package.

package com.ibm.logdemo.message;

import java.io.Serializable;

/**
 * Base interface for ValueObjects that need to be 
 * converted to XML
 * Creation date: (1/15/2002 9:13:59 AM)
 */

public interface XMLMessage extends Serializable 
{
	public String toXML();
}
			

We now can extend the XMLMessage Interface and override the toXML() method, Please examine the class and look at the overridden toXML() message .

The valueobject contains a couple of sample fields for our log message. Now we will look at our Layout class. As stated above, we override the log4J Layout class to create our own Layout. We will use the XML interface to do this. All valueobjects that use this Layout must implement the XMLMessage interface. The Layout is located in the com.ibm.logdemo.message package. The key to the class is the format method listed below.

public String format(LoggingEvent event)
	{
		StringBuffer sbuf = new StringBuffer(2000);
		sbuf.setLength(0);    

		XMLMessage message = (XMLMessage)event.getMessage();
		sbuf.append( message.toXML() );
		sbuf.append(LINE_SEP);	
    	return sbuf.toString();
	}		

Some of the methods have been deprecated, but the main one isn't. The format() method is the key here. Since the valueobject will be the logged message, we can cast to an XMLMessage interface and grab the XML format. Then we just return it as a String. Now that we are finished writing a Layout class, we will see how the Appender gets associated with the Layout class under the log4j configuration section.

Step 5. Log Wrapper Class

Although the log4j API is easy to use, we will use a wrapper class to create a standard way of using the API on a project. Basically, this entails wrapping the log4J Category class to write messages. The Category class works with the configuration file to create the proper filtering level. log4j allows log messages to be filtered based on Java packages. This allows for the ability to write messages to different or multiple outputs based on where you are in your Java-packaging scheme. Furthermore, you can have different levels of logging. log4j defines 5:

1. DEBUG
2. INFO
3. WARN
4. ERROR
5. FATAL

Each level of logging is more severe than the next. For example, if I set my logging level for a particular package to WARN, only WARN, ERROR, and FATAL will be logged. log4j allows for this configuration to be done programmatically or configurable. We will not delve into advanced log4j filtering here. For more information, visit the Jakarta Web site .

For our sample, we will just filter at the root level. It is located in the com.ibm.logdemo.appender package. The main point of the class is to create an easy log interface and convert the log to our valueobject. We could have created different value objects for different logs, however for this demo we will use one. We could have made the Log class a Singleton Object.

Step 6. log4j configuration file

Finally, we will create a log4j configuration file. As stated above, the log4j configuration file can be designed to create a powerful filtering and output location. Its main purpose is to associate an Appender with a Layout. The file can go anywhere in the CLASSPATH. In our sample, I created a very simple configuration. I put it under the /LOG4JdemoWeb/web applications/WEB-INF/classes/BasicConfig.lcf . Here is how it looks:

#Sets the root level to DEBUG, which means every log 
#message will be displayed
log4j.rootCategory=DEBUG, A1

#Next we set the Category to the Appender
log4j.appender.A1=com.ibm.logdemo.appender.JMSQueueAppender

#Next we set the Category to the Layout
log4j.appender.A1.layout=com.ibm.logdemo.message.XMLLayout

The comments explain the links. The key here is with proper architecture, one can change the behavior of your logging without changing code. To learn more about this feature, see the Jakarta Web site .

Step 7. Test servlet and JSP to write to log

Next, we will look at a test servlet and JSP used to write messages to the LOG. The servlet will only use the wrapper class. The code for the LogServlet is located under the com.ibm.logdemo.servlet package. The JSP is under the web applications directory root. Basically, the LogServlet writes some test messages to a queue and forwards to a success page JSP. Please see the goGet method that writes to the log.

Step 8. Create test servlet and JSP to read log for verification

Finally, we have also created a ReadLogServlet and LogDisplay.jsp to read the queue and display the messages. The servlet is located under com.ibm.logdemo.servlet and the JSP is under the web application folder.

The servlet simply browses a queue and displays the log on the screen. Since we are in a browser, Internet Explorer will not display the XML tags; however, when we view the source, we will see the XML syntax. The servlet and JSP code does not necessarily follow best practices, but we just want to test our results.

Creating the MQSeries Queue

We will now create a single queue for this sample. I am using MQSeries (WebSphere MQ) Version 5.2.1 and the latest MA88 package. We will use MQSeries Explorer to create the queue. Figure 4 below shows the the MQSeries Explorer Window for creating a local queue.

The Queue name is LOG4JQUEUE . My Queue Manager name is QM_ibmroly2 . Your Queue Manager name will vary.

Step 9. Run JMSAdmin Script for JNDI and JMS Objects

We need to configure JMS to the JNDI inside the Application Developer test environment. You can run the JMSAdmin located in your MQ_INSTALL_PATH\Java\bin . This can be done inside the Application Developer or at the command line. Since the Application Developer uses WebSphere Single Server, we need to use the Sun reference JNDI to register the objects and configure WebSphere Single Server point it. If this were WebSphere Advanced Full version, Sun reference implementation would not be necessary.

Before running JMSAdmin, we need to ensure we have the following entries set in our PATH.

PATH

MQ_INSTALL_PATH \bin 
MQ_INSTALL_PATH \Java\bin
MQ_INSTALL_PATH \Java\lib

Also, the following jar files that are located in the MQ_INSTALL_PATH\Java\lib need to be in the CLASSPATH

com.ibm.mq.jar
com.ibm.mqbind.jar
com.ibm.mqjms.jar
fscontext.jar
providerutil.jar
connector.jar

Next, use this code to configure the JMSAdmin.configj file to point to the correct JNDI.

Notice PROVIDER_URL, we need to create a directory on our hard drive that matches the directory in the file above.
Note : For the development environment, we will use the Sun Context to host our JNDI Bindings. The test environment in Application Developer does not support permanent bindings for JMS inside the JNDI. When we deploy to WebSphere 4.0 Advanced Edition, we will bind the JMSAdmin directly in the WebSphere 4.0 JNDI.

Next, we need to create a JMSAdminScript.scp file. The file contains the JNDI Bindings. The file is listed below

DEFINE CTX(jms)
CHANGE CTX(jms)

DEFINE QCF(LogConnectionFactory)
DEFINE Q(LogQueue) QUEUE(log4JQUEUE)

DISPLAY CTX
END

We do not need to list the Queue Manager here because we are using our default Queue Manager.

Finally, we can run the script as follows:

JMSAdmin < c:\ _\JMSAdminScript.scp

Step 10. Run the WebSphere test environment

To create a simple server project, right-click the EAR Project folder and choose Run On Server . (We can use an existing server project in our workspace). Next, we need to stop the server to alter our server configuration. Under the Server Perspective, in the Servers View, right-click your Server Instance and choose Stop .

Step 11. Run WebSphere administrative client to create external JMS and JNDI bindings

Now, let's update the server configuration and enable our administrative client.

1. From the Server Perspective, go to the Server Configuration view and double-click the Server Configuration .
2. From the editor, check Enable Administrative client and press Ctrl-S to save and close the editor.
3. From the server perspective, restart the server.
4. From the Servers View, right-click your Server Instance and click Start .
5. Right-click your Server Instance and choose Run Administrative Client to run the WebSphere Single Server Administrative Client.

We need to configure WebSphere MQ (MQSeries) as our JMS Provider as shown in Figure 5 below.

The Server ClassPath should contain the same JAR files we needed to run JMSAdmin. (The connector.jar is only necessary for Single Server). Furthermore, we need to put the same Initial Context and URL as the one we chose inside the JMSAdmin.config file. Next, we will create our QueueFactory and Queue Bindings and point them to the ones inside the Sun Context. Please see Figure 6 for the Connection Factory and Figure 7 for the Queue Binding.

Notice that for both, the JNDI name uses the forward slash and the external name uses the backslash. (This is only for the Sun implementation on the Windows platform, any UNIX-based system requires the forward slash for both.) Finally, we can save our configuration and exit the administrative client. There should be a link on the page prompting you to do so.

Step 12. Test the application

Now, we will run the LogServlet and write some messages. From any navigator view on any perspective, right-click our Web Project and choose Run On Server . Your web page should be displayed in Application Developer just like Figure 8.

We have written some messages on the queue. If we check the MQSeries Explorer as in Figure 9, we will see five messages on the queue,one for each level of logging.

Now, we can click Submit and see our log messages. The page should look like Figure 10.

To see the XML format, we can do a view source on the Web page. We should see something like this:

<mylog4j:event>
	<category>
		DEBUG
	</category>
	<class-name>
		com.ibm.logdemo.servlet.LogServlet
	</class-name>
	<date>
		Tue Jun 04 15:27:03 EDT 2002
	</date>
	<message>
		Debug Test
	</message>
</mylog4j:event>

Next, we can change the level of logging by updating the BasicConfig.lcf. We will change the logging level to ERROR as follows by setting the rootCategory=ERROR

#Sets the root level to ERROR, which means every log message 
#will be displayed
log4j.rootCategory=ERROR, A1

#Next we set the Category to the Appender
log4j.appender.A1=com.ibm.logdemo.appender.JMSQueueAppender

#Next we set the Category to the Layout
log4j.appender.A1.layout=com.ibm.logdemo.message.XMLLayout

Now, go back and rerun the LogServlet and see the result. As you can see, only two additional messages were added. The Debug, INFO, and Warn messages are ignored. If we look at the queue, we should only have 7 messages as shown in Figure 11.

Back to top

Deploying the application and creating a server group

Now, we can deploy the application to WebSphere 4 Advanced Edition and create a server group with two clones to show how-to processes that can write to a centralized location.

1. Deploy the application to WebSphere 4.0 Advanced Edition
2. Create server group and clone
3. Rerun JMSAdmin for WebSphere Advanced Edition and configure through the Admin Client
4. Test the application

Step 1. Deploy the application to WebSphere 4.0 Advanced Edition

To deploy the application on to WebSphere 4.0 Advanced Edition, we will create two application server clones on the same machine. The purpose is to demonstrate two application servers writing to a central log.

First, right-click the EAR file and select export ear to export the EAR file from WebSphere Studio. (Alternatively, you can just use the EAR you imported to Application Developer.) I exported it directly to the installable apps directory of my WebSphere 4. 0 installation directory. Please refer to the WebSphere 4. 0 Advanced Edition Handbook for how to deploy applications to WebSphere. Next, we will bring up the WebSphere Admin Console and create an application server called LogServer . Right-click Application Servers under your node and click New . Please enter LogServer as the application server name as seen in Figure 12.

To install our EAR file onto the application server, click Console => Wizards => Intall Enterprise Application . Navigate to the EAR file in the Installable Apps directory and select the EAR file we exported as seen in Figure 13.

Go through the wizard leaving all the defaults, until you get to the selecting application servers screen as seen in Figure 14. Press Select Server and choose your application server. Click Next and Finish .

Step 2. Create Server Group and clone

Next, we will create a Server Group and a clone to replicate our application server with the installed application. This will allow us to demonstrate our distributed log. First, we create the Server Group, which is a template for our application servers. We create the Server Group by using our existing application server. For more information on WebSphere 4.0 Cloning, please refer to the WebSphere Advanced Edition 4.0 Handbook or the WebSphere 4.0 Advanced Edition Scalability and Availability .

To create the Server Group, right-click Log Server and choose Create Server Group . We will call the Server Group, LogServerModel as seen in Figure 15. Leave all the other defaults and click OK .

Next, we will create a new clone from the Server Group.
Note : The original LogServer now becomes a clone of the Server Group.
Right-click the Server Group and select New => Clone .

We will call it LogServer2 and assign it to our machine. See Figure 16

Ensure that the regenerate plugin option is set to true for our demonstration purposes. This can be set to true under the custom tab of our Server Group. For clarity, we also assigned each clone a different Standard Out and Standard Error file to show both our application servers are writing to the application server (see Figure 17). We can do this by selecting each clone in under our machine (node), going to the file tab and putting in a unique name for each standard out and error.

Note : Make sure to press Apply after each change is made to a configuration on our application server.

Step 3. Rerun JMSAdmin for WebSphere Advanced Edition and configure through the Admin Client

Now, we can set up our JMS configuration. We will assume that MQSeries is properly configured and a JMS Provider and that it is installed properly on our machine and node. Please see related references for instructions on how to configure MQSeries with WebSphere 4.0. First, we need to modify our JMSAdmin configuration file to point to WebSphere 4.0 as our JNDI Server. We can do this by commenting our fscontext InitialContextFactory and uncommenting our WebSphere one. We will do the same for our PROVIDER_URL. Please see the source code .

Next, we will rerun out JMSAdmin script as we did before.

JMSAdmin < c:\=\JMSAdminScript.scp

Let's go to back to our WebSphere Admin Console and configure our Connection Factory and Queue. Although we ran our JMSAdmin Script, we need to configure our JMS Provider to point to these bindings. WebSphere treats all JMS providers as external although the bindings are really internal. Under Resources and IBM MQSeries, you can create a Connection Factory and Queue Binding by right clicking Connection Factories or Queues. Figures 18 and 19 show the wizard for creating these bindings.

Step 4. Test the application

Now, we will start up our server group and test our application. We will rerun the LogServlet multiple times and then look at the stdout and stdout2 file to see that both application servers are writing to a centralized queue. (I cleared out the queue for the new test.)

To start up both clones, start up the ServerGroup, right-click your Server Group , and click Start . Also, ensure you start your HTTP Server.

Finally, we can run our application as we did before. Click Refresh a few times and then click the ReadLog button. This will ensure that both our clones receive requests to log.

We can then check our log files to see that two different application servers are logging to a central queue. Both stdout.txt and stdout2.txt should contain some statements.

Both logs should read as follows.

[6/4/02 15:27:03:861 EDT] 45ac3ad8 WebGroup      I SRVE0091I: [Servlet LOG]: LogServlet: init
[6/4/02 15:27:03:891 EDT] 45ac3ad8 SystemOut     U Entering Log Servlet
[6/4/02 15:27:03:991 EDT] 45ac3ad8 SystemOut     U Entering append
[6/4/02 15:27:03:991 EDT] 45ac3ad8 SystemOut     U Getting Factory
[6/4/02 15:27:04:281 EDT] 45ac3ad8 SystemOut     U Getting queue connection
[6/4/02 15:27:04:281 EDT] 45ac3ad8 SystemOut     U Getting session
[6/4/02 15:27:04:281 EDT] 45ac3ad8 SystemOut     U Getting queue
[6/4/02 15:27:04:321 EDT] 45ac3ad8 SystemOut     U Getting sender
[6/4/02 15:27:04:341 EDT] 45ac3ad8 SystemOut     U Creating message
[6/4/02 15:27:04:341 EDT] 45ac3ad8 SystemOut     U Setting test
[6/4/02 15:27:04:351 EDT] 45ac3ad8 SystemOut     U Sending message
[6/4/02 15:27:04:401 EDT] 45ac3ad8 SystemOut     U Message sent
[6/4/02 15:27:04:401 EDT] 45ac3ad8 SystemOut     U Entering append

We have successfully implemented a centralized distributed log using JMS and log4j.

Back to top

Conclusion

The sample above is only a stepping stone. Once we have the interface built, we can write a server side applications that can interpret the log and react. Perhaps we could use DB2 XML Extender to build a more formalized XML Schema for the database logs.

Application logging is a very important, and often overlooked, piece of the architectural process. By designing the logging architecture early on, we can come up with an optimal solution. Open Source and IBM's WebSphere, MQSeries, and DB2 Family give us the tools to do it.

Back to top

Acknowledgements

David Salkeld, IBM Software Services for WebSphere

Back to top

Related references

For more information on Jakarta log4j, MQSeries using Java, JMS, and J2EE patterns, see the following reference list:

• Jakarta log4j
• Core J2EE Patterns: Best Practices and Design Strategies by Deepak Alur, John Crupi, Dan Malks
• IBM WebSphere Version 4.0 Advanced Edition Scalability and Availability
• IBM WebSphere V4.0 Advanced Edition Handbook

Top of page

Back to top

Download

Name	Size	Download method
Log4jdemo.zip	2KB	FTP|HTTP

Information about download methods

About the author

Roland Barcia is a senior software consultant for IBM Software Services for WebSphere in the New York/New Jersey Metro area. You can reach Roland Barcia at barcia@us.ibm.com .

Rate this article

Comments

Back to top

Table of contents

• Introduction
• Sample technology
• Building the sample application
• Deploying the application and creating a server group
• Conclusion
• Acknowledgements
• Related references
• Download
• About the author
• Comments

Next steps from IBM

• Try: WebSphere MQ
• Briefing: WebSphere flexible business processes
• Buy: WebSphere MQ

Dig deeper into WebSphere on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers