The Support Authority: A developer's guide to WebSphere Application Server logging

Log messages and trace information can be critical, time-saving elements in the initial stages of problem diagnosis, and can often alleviate the need to recreate a problem in order to troubleshoot it. This article looks at the log and trace facilities in IBM® WebSphere® Application Server, explains the difference between them, and describes how you can leverage them in your own applications. This content is part of the IBM WebSphere Developer Technical Journal.

Don Bourne (dbourne@ca.ibm.com), WebSphere Serviceability Architect, IBM

Don Bourne is a WebSphere Serviceability Architect at the IBM Toronto Lab. Don joined IBM in 1996 and has been specializing in the area of serviceability since joining the WebSphere Application Server team in 2003. Currently, Don is working with the IBM Support Assistant team architecting future solutions to help users resolve problems quickly.



Keys Botzum, Senior Technical Staff Member , EMC

Keys Botzum is a Senior Technical Staff Member with IBM Software Services for WebSphere. Mr. Botzum has over 10 years of experience in large scale distributed system design and additionally specializes in security. Mr. Botzum has worked with a variety of distributed technologies, including Sun RPC, DCE, CORBA, AFS, and DFS. Recently, he has been focusing on J2EE and related technologies. He holds a Masters degree in Computer Science from Stanford University and a B.S. in Applied Mathematics/Computer Science from Carnegie Mellon University. Mr. Botzum has published numerous papers on WebSphere and WebSphere security. Additional articles and presentations by Keys Botzum can be found at http://www.keysbotzum.com, as well as on IBM developerWorks WebSphere. He is also co-author of IBM WebSphere: Deployment and Advanced Configuration.


developerWorks Professional author
        level

Daniel Julin (dpj@us.ibm.com), Senior Technical Staff Member, EMC

Daniel Julin has 20 years of experience developing and troubleshooting complex online systems. As technical area lead for the WebSphere Serviceability Team, he currently focuses on helping the team define and implement a collection of tools and techniques to assist in problem determination for WebSphere Application Server and to maximize the efficiency of IBM support. He also occasionally assists directly in various critical customer support situations.


developerWorks Contributing author
        level

27 February 2008

Also available in Chinese Japanese

In each column, The Support Authority discusses resources, tools, and other elements of IBM Technical Support that are available for WebSphere products, plus techniques and new ideas that can further enhance your IBM support experience.

This just in...

As always, we begin with some new items of interest for the WebSphere® community at large:

  • This month, IBM announced a new Enhanced Software Support Lifecycle policy that extends the minimum time period during which technical support will remain available for older versions of many products, up to five years. This is in response to client concerns about the burden of upgrading too often to keep up with major product releases.
  • Work continues on a broad program of enhancements to the organization, and the usability of the various software support resources on ibm.com. This week, we are beginning to rollout our new design of many individual product support pages, including the one for IBM WebSphere Application Server. Check out a sample.
  • The IBM Education Assistant Web site is also getting some enhancements, most notably a new set of RSS feeds (on the front page associated with each product) to help you learn about new education modules. Incidentally, over 100 new education modules have already been released in 2008, many of them covering topics related to troubleshooting.
  • Several problem determination tools in the IBM Support Assistant have just been updated. It is getting so that there are more updates and new tools than can be conveniently enumerated each month, but we should note these in particular:
  • Several tools in alphaWorks have also been refreshed, mostly to support new Java versions and platforms. In particular:
  • The WebSphere Technical Exchange Webcasts series continues to be a great source of information, and there are particularly rich agendas over the next few months for problem determination-related topics. We won't list all the topics here, but a few that look particularly interesting include Guided troubleshooting using the IBM Guided Activity Assistant (IGAA), IBM Support Assistant Diagnostic Tools Overview, Debugging OutOfMemory errors on WebSphere AppServer z/OS, Resource starvation in Java and its diagnosis, and Automating problem identification using IBM Autonomic Computing Technology.

Continue to monitor the various support-related Web sites, as well as this column, for news about other tools as we encounter them.

And now, on to our main topic...


Good programs tell you when they're bad

One of the first things we learn as developers is how to print messages to the console. Later, we learn how valuable such print statements can be when debugging code, particularly in the early stages before finding a favorite development workbench. Having acquired an ability to follow the flow of code and spot changes in the value of variables, we pick up on the value of clear debug statements to quickly pinpoint problems in code. As each method is debugged, we strip out the print statements to make it easier to focus on the next section of code. When all is ready, we happily delete the last print statements knowing our code is ready to be used.

For any non-trivial program, diagnostic code is valuable beyond unit testing. While in some circumstances, problems can be diagnosed without the aid of diagnostic code, log and trace statements used well can greatly reduce diagnostic time. Logging code is an important part of first failure diagnostic information, and can often alleviate the need to get problems to reoccur in order to diagnose them. IBM WebSphere Application Server has valuable log and trace facilities that you can leverage in your own applications. This article explains what you need to know, as a developer, to write effective logging and tracing code into your applications that will run with WebSphere Application Server.


Logging and tracing basics

What's the difference between logging and tracing?

  • Logging is typically used for important events that you want to make note of in your log files. Logging is used to indicate important changes in state (for example, when a service starts or stops), to indicate warnings (for example, when a disk you are writing to is running out of space), or to indicate errors (for example, when your code is no longer able to continue because an expected service is not available). Logging is typically left on all the time, so logging code is necessarily fairly low volume, generally important, and something you should pay attention to. One other important distinction is that logging is useful to your application administrators. Since logged messages are intended for administrators, messages logged by the application server are translated and written to logs in the language the rest of the server process is configured for (applications can also take advantage of message localization, but this is usually not required). Care must be taken when creating log messages to ensure that they are easy to understand by anyone using the code. As such, it generally takes more time to add logging statements to code than it does to add trace statements.

  • Tracing is typically used to record any information that could be useful in debugging problems with your code. Tracing is often used to indicate which methods are called, what data is passed into a method (or what values are returned from a method), and what data is returned from calls to other methods beyond the boundaries of your code. Trace events can be high in volume, and as such, are only enabled when a problem is being diagnosed. Due to the very detailed and technical content of trace events, they are often only of value to the person who wrote the application. When tracing is turned on, it should be possible to understand any problem that could occur in your code. If trace instrumentation is not done well, you might need to send debug versions of your code (with more trace instrumentation) to users when they run into problems; this might not be practical (or desireable) and is generally best avoided with careful attention to traces before the code is made available to end users.

What is content important for logs and trace?

Having the right content in your log and trace files will help you diagnose problems more quickly. At a minimum, log and trace content should specify:

  • Precise timestamp indicating when the message was generated.
  • Class name and method name of the source of the message.
  • Thread ID of the originator of the message.
  • Severity of the event reflected by the message.
  • Name that identifies the logger responsible for emitting the message, for the purpose of setting the log/trace level that controls which messages are enabled or disabled.
  • A clear message.

What capabilities are important from a logging and tracing runtime object?

In addition to having the right content in your log and trace events, it is imperative that your logging and tracing runtime object:

  • Enables sufficient fine-grained control over which messages are enabled or disabled.
  • Enables log and trace levels to be controlled without requiring a server restart.
  • Provides a way for logs and traces to be automatically archived or truncated -- without requiring a server restart -- to keep them from growing too large.
  • Provides a way for logs to be remotely monitored.

Choosing the right log and trace API

What logging APIs are the most common?

There are a number of possible logging API options that you can use in your WebSphere Application Server application. The most common candidates are shown in Table 1.

Table 1. Logging APIs
Logging APIIntegrated with WebSphere Application Server logs?Recommended for use in WebSphere Application Server applications?
java.util.logging (JUL)Yes (Since V6.0)Yes (with V6.0 or later)
JRASYesYes (only with releases prior to V6.0)
Log4JNoNo
Jakarta Commons Logging (JCL)YesNo
System.out.println and System.err.printlnYesNo
  • java.util.logging (JUL)

    JUL has been a standard part of the Java SDK since the release of J2SE 1.4 in 2002. Architecturally, it is similar to Log4J. The JUL API has five main concepts: loggers, handlers, filters, formatters, and LogRecords. JUL's key strengths are that it is part of all runtimes based on J2SE 1.4 (or later), provides a flexible architecture that enables control of logging settings, and is easy to customize.

    JUL is the preferred logging implementation in WebSphere Application Server, and is used in WebSphere Application Server's own implementation. JUL's logging API is simple to use and sufficiently customizable to address most logging needs. JUL is well integrated with WebSphere Application Server, and any requests made to the JUL logger API, even from application code, are handled by the WebSphere Application Server logging infrastructure. Using JUL is strongly encouraged for any new application code that will operate inside WebSphere Application Server.

  • JRAS

    JRAS is a public API that has been provided with WebSphere Application Server since V3.5. The JRAS API, and the API it was based on, were predecessors to Log4J and JUL. The JRAS API was deprecated in WebSphere Application Server V6.0 with the application server's leveraging of the JUL API. As such, do not use JRAS for any new code.

  • Log4J

    Log4J is an Apache project that aims to provide a flexible and powerful logging implementation. Log4J was first publicly released in 1999 and is architecturally similar to JUL. Where JUL has loggers, handlers, formatters and LogRecords, Log4J has loggers, appenders, layouts, and LoggingEvents. Log4J comes standard with many appenders and layouts.

    Log4J is available as an open source project from Apache, but is not part of the Java language standard. Log4J's key strengths are that it provides similar capabilities to JUL, and that it has a wealth of prebuilt appender and layout implementations.

    While similar in architecture to JUL, Log4J is not integrated with the WebSphere Application Server logging infrastructure. In general, unless there is an overriding need for the advanced capabilities of Log4J's appenders or layouts, do not use Log 4J with WebSphere Application Server. You might decide to use Log4J with WebSphere Application Server and find that it works just fine, but you just won't benefit from the integration leveraged by JUL.

  • Jakarta Commons Logging (JCL)

    JCL is an Apache project that aims to provide a wrapper API to which you can write. Unlike other logging APIs, JCL delegates logging calls at run time to a concrete logging implementation appropriate to the containing environment. Thus, for example, in WebSphere Application Server, events from JCL are sent to the WebSphere Application Server log and trace facilities. In other environments, it is not uncommon to send JCL output to Log4J. The key point here is that JCL is not a standards-based API, but rather a framework that lets developers "not care" where their trace output goes. JCL predates a standard API.

    JCL's key strength is that it enables code to be instrumented with logging calls independent of any consideration of what logging facilities exist in the runtime, or what runtimes the code will ultimately run on.

    JCL is integrated and packaged with WebSphere Application Server. Applications that bundle their own copy of the JCL JAR and want to send output to destinations other than the IBM-provided defaults will need to take extreme care to avoid class loading conflicts with the version of JCL included in the application server runtime. This is typically very difficult to do properly and, therefore, not recommended.

    With the common availability of JUL in all J2SE 1.4 and later JVMs, the need for JCL as a portable logging wrapper has diminished. As such, do not use JCL in any new application code.

  • System.out.println and System.err.println

    While not really a log or trace API, many developers are hooked on using System.out.println to instrument their code. WebSphere Application Server augments data passed to System.out and System.err. Rather than just write System.out and System.err to the console, WebSphere Application Server:

    • Converts data passed to each stream into log events (an inexact process).
    • Determines the creation time of the event.
    • Determines the thread ID of the originator of the event.
    • Writes the event to a managed log file complete with the above information.

    Unfortunately, with System.out and System.err, the application server cannot determine the exact source (class or method) of the message, cannot tell how severe the message is, and cannot provide more than a basic on/off capability (server wide) to control the output of these messages. Perhaps most importantly, sending high volume trace output to System.out is expensive and will result in significant performance issues (and pressures to reduce the amount of output). These deficiencies make System.out and System.err impractical for large scale use. Therefore, do not use of System.out.println and System.err.println in application code.

Which logging API should I use with WebSphere Application Server?

As outlined above, there are a variety of logging APIs to choose from. Here is what the WebSphere Application Server serviceability team recommends for your application code:

  1. Use JUL for new code that will run on WebSphere Application Server V6.0 or later.
    With the ubiquity of JUL and its tight integration with application server log and trace facilities, JUL is the best API to use for new WebSphere Application Server application code. Other logging APIs should only be considered in cases where JUL is not available in all the runtimes where your application needs to run.

  2. Continue to use existing logging frameworks for new code if (and only if) you need to support environments that don't properly support JUL.
    Generally, if you are continuing to deploy to pre-J2SE 1.4 environments, you will already have other applications and an established logging API. Your two choices are to continue to use your existing logging API, or find another one. Now that there is a logging standard in JUL, it is less compelling to invest in a new logging infrastructure, as any investment would likely be temporary.

  3. Do not replace thousands of lines of tested logging and trace instrumentation for the sake of logging API purity.
    In cases where you are migrating to an application server that supports JUL, it can be tempting to redo existing log and trace instrumentation. Rewriting tens of thousands of lines of trace code (and having to retest) for the sake of standardizing on one logging API is usually not a good use of development resources. Where possible, consider bridging your existing logging API to write to JUL. If you are unable to consolidate your log events via JUL, you can use tools like the Log Analyzer to merge your log with the application server logs when viewing.


Instrumenting your code with JUL

  • Loggers are used in code wherever logging and tracing is required. Loggers are typically assigned names based on the Java class or package that they are used in (for example, com.acme.birdtrap.PurchaseOrder). Loggers are stored by a singleton LogManager in a hierarchy that enables loggers to inherit characteristics from their parent loggers. The name of each logger determines where in the hierarchy the logger belongs, each logger's parent having the same name as its child, minus anything starting from the last period in the child's name's (for example, com.acme is the parent logger of com.acme.birdtrap, which is the parent logger of com.acme.birdtrap.PurchaseOrder). Each Logger has a level which is a settable threshold controlling what the minimum severity an event (known as the LogRecord) must be logged with in order to not be discarded. If a logger's level is not set explicitly, the logger inherits its level from its parent. A logger can optionally have a filter, which is a class used by the logger to decide whether or not to discard the logging request based on the contents of the LogRecord.

  • Handlers are classes used by loggers to write the LogRecord to an output device. Handlers may either format the LogRecords themselves, or use a formatter to convert LogRecords into strings. Handlers also have a level setting, the level serving the same purpose as it does for a logger. Handlers might also have a filter, which serves the same purpose as a logger's filter. Handlers typically write formatted results from a LogRecord into log files, but could be designed to output to an in-memory buffer, a socket, an instant messaging destination, or any other location deemed useful. JUL comes standard with a number of handlers and formatters to take care of your basic needs.

    Figure 1. Key components in JUL log request handling
    Figure 1. Key components in JUL log request handling

Listing 1 contains sample code that illustrates good coding practices when working with JUL. These good practices will be explained after the example.

Listing 1. Sample code illustrating use of JUL
1  package com.acme;

2  import java.util.logging.Level;
3  import java.util.logging.Logger;

4  public class JULExample {

5  	// define a Logger for use by this class
6  	static String className = JULExample.class.getName();
7  	public static Logger logger = Logger.getLogger(className);

8  	public String lookupContactPhoneNumber(String contactName) {
9  		String methodName = "lookupContactPhoneNumber";
10  		logger.entering(className, methodName);

11  		// connect to contact repository
12  		ContactRepository cr = ContactRepository.connect();

13  		if (cr == null) {
14  			// LOGGING
15  			logger.logp(Level.WARNING, className, methodName,
16  						"Contact repository not available");
17  			return "";
18  		}

19  		// lookup the contact phone number
20  		String contactPhoneNumber = cr.getContactPhoneNumber(contactName);

21  		// TRACE
22  		if (logger.isLoggable(Level.FINEST)) {
23  			// use isLoggable so we don't concatenate strings
24  			// below unless trace is enabled
25  			String traceString = "name    " + contactName + "number: "
26  					+ contactPhoneNumber;

27  			logger.logp(Level.FINEST, className, methodName,       
28             	            traceString);
29		}

30		logger.exiting(className, methodName, contactPhoneNumber);
31		return contactPhoneNumber;
32	}

33	public String getSomeString() {
34		return "SomeString";
35	}

36  }

Adding logging and trace code to your applications using the JUL API will greatly improve your applications' serviceability. Here are some suggested rules of thumb to help you get great results:

  1. Store logger instances instead of calling Logger.getLogger repeatedly.
    In Listing 1, Logger.getLogger() is only called once to get a logger for the class. Loggers are thread safe, so you can safely store them in a static variable for use by any class instances. This enables you to use the logger at any point in your class without having to call Logger.getLogger() again, which improves your code's performance.

  2. Use the fully qualified class name as your logger name.
    The logger name is used to identify the logger when you need to change the logger level (for example, in the administrative console). Assigning the logger the fully qualified class name makes it easy to specify which logger's level needs adjusting to get the trace you need from your users.

  3. Ensure your log and trace entries are easy to link back to your code.
    By specifying the class name and method name in your log and trace entries, you'll be able to tell immediately where in the code the event was generated.

    You can also use the entering() method call to mark the beginning of a method, but keep in mind that entering() method calls generate Level.FINER events, and Level.FINER events are suppressed by default in WebSphere Application Server.

    Some JUL implementations will try to calculate which method and class generated the event when those parameters are not specified, but this can have undesirable performance consequences -- and may not even be correct, in cases where the JIT has removed stack frames. Within WebSphere Application Server, there is no attempt to calculate the method and class name on behalf of the caller (as this optional capability was deemed to be very inefficient). You are better off using the JUL logger methods where you can provide your own class and method names.

    Table 2. Logger methods that let you specify class and method names
    Logger methods
    entering(String sourceClass, String sourceMethod)
    entering(String sourceClass, String sourceMethod, Object param1)
    entering(String sourceClass, String sourceMethod, Object[] params)
    exiting(String sourceClass, String sourceMethod)
    exiting(String sourceClass, String sourceMethod, Object result)
    Logp(Level level, String sourceClass, String sourceMethod, String msg)
    Logp(Level level, String sourceClass, String sourceMethod, String msg, Object param1)
    logp(Level level, String sourceClass, String sourceMethod, String msg, Object[] params)
    logp(Level level, String sourceClass, String sourceMethod, String msg, Throwable thrown)
    throwing(String sourceClass, String sourceMethod, Throwable thrown)
  4. Follow the application server's convention for log and trace levels.
    The WebSphere Application Server default configuration enables JUL levels from Level.INFO and higher by default. The application server treats all of the JUL Levels from Level.CONFIG up to Level.SEVERE as log event levels, which are intended to be of use to administrators.

    Table 3. Logging levels
    Logging levels
    Level.SEVERE
    Level.WARNING
    Level.INFO
    Level.CONFIG

    The application server treats levels from Level.FINE down to Level.FINEST as trace levels (these are levels used for events intended to help the authors of the code do debugging).

    Table 4. Trace levels
    Trace levels
    Level.FINE
    Level.FINER
    Level.FINEST

    By following these same conventions in your code, the right set of events will be logged by default.

  5. Do not set logger levels in your code. Each JUL logger has an associated Level. This level controls which events the logger ignores, and which ones it logs. Logger levels are set at application server startup from data stored in the application server configuration files. Administrators can change Logger levels via the administrative console through scripts (using wsadmin) or programmatically (using JMX MBean clients). The application server is in control of logger levels.

    Changes made to a logger's level setting via direct calls to the logger setLevel() method do not alter the setting in the application server's central logging configuration, and, as such, logger levels set via setLevel() will not be reflected in the admin console, and may be ignored when the logging infrastructure applies new logging settings based on administrator changes. As such, AVOID the method: setLevel(Level newLevel). When changes are made to the server's central logger configuration, they are pushed out to all affected loggers. This means that calls to the logger getLevel() method will work as expected.

  6. Use logger entering() and exiting() methods for non-trival methods only.
    JUL provides Logger.entering() and Logger.exiting() methods that are recommended for use at the start and end of methods. Adding these trace points to every method in your code would be excessive and would not add to the serviceability of your application. In general, avoid use of entering() and exiting() methods where the content of a particular method is trivial (as in the case of a simple getter or setter method).

  7. Favor writing application logs and traces to the application server logs.
    While JUL provides the ability for you to add new log files (handlers) with minimal code or configuration, it is usually not advantageous to do so. Having your logs and trace interwoven with the application server logs and trace is beneficial when debugging In cases where you cannot write logs to the same log files, or where there are multiple systems involved, tools such as the Log Analyzer (available as a plug-in to the IBM Support Assistant) can help you merge your log files.

  8. Use Logger.isLoggable() where appropriate.
    Because WebSphere Application Server runs with trace disabled by default, it is important that disabled trace statements run as efficiently as possible. To avoid paying a performance cost for the computation of trace method parameters, JUL provides the Logger.isLoggable() method, which checks to see if the logger is set at a level at or above the level you specify. Keep in mind that the logger will make the same check, so it is only necessary to wrap your log or trace statements with the Logger.isLoggable() check when computation of the log and trace parameters would incur some performance cost.


JUL integration with WebSphere Application Server

All versions of WebSphere Application Server since Version V6.0 take advantage of JUL. Here are some additional benefits you will get by using JUL to instrument your code:

  • Built-in log and trace files
    WebSphere Application Server will format your log and trace events and write them into the appropriate application server log files (on Windows®, Linux®, HP/UX, Solaris™, AIX®, and i5/OS operating systems, these are the SystemOut.log, trace.log, and activity.log files; on System z™, this is the platform logging facility). The application server manages these log files for you and provides easy-to-configure archive policies. You can use tools that know how to work with application server log files (such as IBM Rational® Application Developer or the Log Analyzer from IBM Support Assistant) or various IBM Tivoli® tools to work with your log content.

  • Built-in logger level administration for your applications
    WebSphere Application Server provides a way to set the level for any named JUL loggers, including the loggers you create in your own applications. JUL logger levels can be set using the adminsitrative console, wsadmin, or JMX configuration.

    Figure 2. The WebSphere Application Server Change Log Detail Levels panel displays your JUL loggers
    Figure 2. The WebSphere Application Server Change Log Detail Levels panel displays your JUL loggers
  • Inherent correlation between your events and application server events
    The application server log and trace events will be interleaved with your own log and trace events, thus eliminating the need to correlate these separate sets of events. This saves time when diagnosing problems with logs and trace. In cases where this interleaving is undesirable, you can opt to create your own log files (more on this in a future article).

  • WebSphere Application Server extensions to JUL
    WebSphere Application Server provides (via a Logger.properties file) a means to associate a number of useful attributes to a logger. With these attributes, you can add your loggers to a group or improve performance using your loggers with resource bundles for localization.

  • Enterprise level configuration
    WebSphere Application Server initializes the logging environment based on server configuration settings rather than using the JUL logging.properties settings in the JRE/lib directory (which is the JUL default). The benefit of this is that log settings are associated with your server. That means that when you add your server to a cluster, the logging behavior will be consistent across all servers, and is only dependent on settings associated with your J2EE™ environment.


Conclusion

Logging and tracing are important elements of your application code. By following a few best practices you can be sure your serviceability code integrates well with the runtimes in which it is used, provides necessary information for debugging your code in the field, and doesn't significantly degrade your code's performance.

Future installments of this column will cover some more advanced log and trace topics such as:

  • What to do if you have code that uses Log4J or JCL in the application server.
  • How to add your own log files when using JUL.
  • How to use resource bundles for localization with JUL.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=291189
ArticleTitle=The Support Authority: A developer's guide to WebSphere Application Server logging
publish-date=02272008