Demystifying class loading problems, Part 1

An introduction to class loading and debugging tools

Learn how class loading works and how your JVM can help you sort out class loading problems

Comments

Content series:

This content is part # of # in the series: Demystifying class loading problems, Part 1

Stay tuned for additional content in this series.

This content is part of the series:Demystifying class loading problems, Part 1

Stay tuned for additional content in this series.

Class loaders are responsible for loading classes into the Java Virtual Machine (JVM). Simple applications can use the Java platform's built-in class loading facility to load their classes; more complex applications tend to define their own custom class loaders. No matter what kind of class loader you're using, however, there are many problems that can occur in the class loading process. If you want to avoid such problems, you need to understand the fundamental mechanics of class loading. When problems do occur, an appreciation of the available diagnostic features and debugging techniques should help you resolve them.

In this series of articles, we'll provide an in-depth look at class loading problems and use comprehensive examples to illustrate them. The first section of this introductory article describes the fundamentals of class loading; the second introduces some JVM debugging features. The next three articles in this series will focus on resolving class loading exceptions and illustrate some of the trickier class loading problems that you may encounter.

Fundamentals of class loading

This section describes the core concepts of class loading to provide a knowledge base for the rest of this series.

Class loader delegation

The class loader delegation model is the graph of class loaders that pass loading requests to each other. The bootstrap class loader is at the root of this graph. Class loaders are created with a single delegation parent and looks for a class in the following places:

  • Cache
  • Parent
  • Self

A class loader first determines if it has been asked to load this same class in the past. If so, it returns the same class it returned last time (that is, the class stored in the cache). If not, it gives its parent a chance to load the class. These two steps repeat recursively and depth first. If the parent returns null (or throws a ClassNotFoundException), then the class loader searches its own path for the source of the class.

Because the parent class loader is always given the opportunity to load a class first, the class is loaded by the class loader nearest the root. This means that all core bootstrap classes are loaded by the bootstrap loader, which makes sure that the correct versions of classes such as java.lang.Object are loaded. This also has the effect of only allowing a class loader to see classes loaded by itself or its parent or ancestors; it cannot see classes loaded by its children.

Figure 1 shows the three standard class loaders:

Figure 1. The class loader delegation model
The class loader delegation model
The class loader delegation model

Unlike all other class loaders , the bootstrap class loader (also known as the primordial class loader) cannot be instantiated by Java code. (Often, this is because it is implemented natively as part of the VM itself.) This class loader loads the core system classes from the boot classpath, which is normally the JAR files located in the jre/lib directory. However, you can modify this classpath using the -Xbootclasspath command-line options (which we describe later).

The extension class loader (also known as the standard extensions class loader) is a child of the bootstrap class loader. Its primary responsibility is to load classes from the extension directories, normally located the jre/lib/ext directory. This provides the ability to simply drop in new extensions, such as various security extensions, without requiring modification to the user's classpath.

The system class loader (also known as the application class loader) is the class loader responsible for loading code from the path specified by the CLASSPATH environment variable. By default, this class loader is the parent of any class loader created by the user. This is also the class loader returned by the ClassLoader.getSystemClassLoader() method.

Classpath options

Table 1 summarizes the command-line options for setting the classpaths of the three standard class loaders:

Table 1. Classpath options
Command-line optionExplanationClass loader involved
-Xbootclasspath:<directories and zip/JAR files separated by ; or :>Sets the search path for bootstrap classes and resources.Bootstrap
-Xbootclasspath/a:<directories and zip/JAR files separated by ; or :>Appends the path to end of the boot classpath.Bootstrap
-Xbootclasspath/p:<directories and zip/JAR files separated by ; or :>Prepends the path to front of the boot classpath.Bootstrap
-Dibm.jvm.bootclasspath=<directories and zip/JAR files separated by ; or :>The value of this property is used as an additional search path, which is inserted between any value that is defined by -Xbootclasspath/p: and the boot classpath. The boot classpath is either the default, or the path that is defined by the -Xbootclasspath: option.Bootstrap
-Djava.ext.dirs=<directories and zip/JAR files separated by ; or :>Specifies the search path for the extension classes and resources.Extension
-cp or -classpath <directories and zip/JAR files separated by ; or :>Sets the search path for application classes and resources.System
-Djava.class.path=<directories and zip/JAR files separated by ; or :>Sets the search path for application classes and resources.System

The phases of class loading

The loading of a class can essentially be broken down into three phases: loading, linking, and initializing.

Most, if not all, problems relating to class loading can be tracked down to a problem occurring in one of these phases. Therefore, a thorough understanding of each phase helps in the diagnosing of class loading problems. The phases are illustrated in Figure 2:

Figure 2. The phases of class loading
The phases of class loading
The phases of class loading

The loading phase consists of locating the required class file (by searching though the respective classpaths) and loading in the bytecode. Within the JVM, the loading process gives a very basic memory structure to the class object. Methods, fields, and other referenced classes are not dealt with at this stage. As a result, the class is not usable.

Linking is the most complicated of the three phases. It can be broken down into three main stages:

  • Bytecode verification. The class loader does a number of checks on the bytecodes of the class to ensure that it is well formed and well behaved.
  • Class preparation. This stage prepares the necessary data structures that represent fields, methods, and implemented interfaces that are defined within each class.
  • Resolving. In this stage, the class loader loads all the other classes referenced by a particular class. The classes can be referenced in a number of ways:
    • Superclasses
    • Interfaces
    • Fields
    • Method signatures
    • Local variables used in methods

During the initializing phase, any static initializers contained within a class are executed. At the end of this phase, static fields are initialized to their default values.

At the end of these three phases, a class is fully loaded and is ready for use. Note that class loading can be performed in a lazy manner and therefore some parts of the class loading process may be done on first use of the class rather than at load time.

Explicit vs. implicit loading

There are two ways in which classes can be loaded -- explicitly or implicitly -- with subtle variations between the two.

Explicit class loading occurs when a class is loaded using one of the following method calls:

  • cl.loadClass() (where cl is an instance of java.lang.ClassLoader)
  • Class.forName() (the starting class loader is the defining class loader of the current class)

When one of these methods is invoked, the class whose name is specified as an argument is loaded by the class loader. If the class is already loaded, then a reference is simply returned; otherwise, the loader goes through the delegation model to load the class.

Implicit class loading occurs when a class is loaded as result of a reference, instantiation, or inheritance (not via an explicit method call). In each of these cases, the loading is initiated under the covers and the JVM resolves the necessary references and loads the class. As with explicit class loading, if the class is already loaded, then a reference is simply returned; otherwise, the loader goes through the delegation model to load the class.

Classes are often loaded through a combination of explicit and implicit class loading. For example, a class loader could first load a class explicitly and then load all of its referenced classes implicitly.

Debugging features of the JVM

The previous section introduced the fundamental principles of class loading. This section covers the variety of features built into the IBM JVM to assist debugging. Other JVMs may have similar debugging features available; refer to the appropriate documentation for details.

Verbose output

You can turn on the IBM JVM's verbose output by using the -verbose command-line option. Verbose output displays information on the console when certain events occur -- when a class has been loaded, for instance. For additional class loading information, you can use verbose class output. This is activated using the -verbose:class option.

Interpreting verbose output
Verbose output lists all the JAR files that have been opened and includes the full path to those JARs. Here's an example:

...
[Opened D:\jre\lib\core.jar in 10 ms]
[Opened D:\jre\lib\graphics.jar in 10 ms]
...

All the classes that are loaded are listed, along with the JAR file or directory from which they were loaded. For example:

...
[Loaded java.lang.NoClassDefFoundError from D:\jre\lib\core.jar]
[Loaded java.lang.Class from D:\jre\lib\core.jar]
[Loaded java.lang.Object from D:\jre\lib\core.jar]
...

Verbose class output shows additional information, such as when superclasses are being loaded, and when static initializers are being run. Some example output follows:

...
[Loaded HelloWorld from file:/C:/myclasses/]
[Loading superclass and interfaces of HelloWorld]
[Loaded HelloWorldInterface from file:/C:/myclasses/]
[Loading superclass and interfaces of HelloWorldInterface]
[Preparing HelloWorldInterface]
[Preparing HelloWorld]
[Initializing HelloWorld]
[Running static initializer for HelloWorld]
...

Verbose output also shows some internally thrown exceptions (if they occur), including the stack trace.

Resolving problems using -verbose
Verbose output can help to solve classpath problems, such as JAR files not being opened (and therefore not on the classpath) and classes being loaded from the wrong place.

IBM Verbose Class Loading

It is often useful to know where class loaders look for classes and which class loader loads a particular class. You can obtain this information using the IBM Verbose Class Loading command-line option: -Dibm.cl.verbose=<class name>. You can use regular expressions to declare the name of the class; for instance, Hello* traces any classes with names starting with Hello.

This option also works on user-defined class loaders, as long as they directly or indirectly extend java.net.URLClassLoader.

Interpreting IBM Verbose Class Loading output
IBM Verbose Class Loading output shows the class loaders that attempt to load the specified class and the locations in which they look. For example, imagine we used the following command line:

java -Dibm.cl.verbose=ClassToTrace MainClass

Here, MainClass references ClassToTrace in its main method. This would produce output similar to the output here.

MainClass references ClassToTrace in its main method
ExtClass loader attempting to find ClassToTrace
ExtClass loader using classpath D:\jre\lib\ext\gskikm.jar;D:\jre\lib\ext\ibmjceprovider.jar;
 D:\jre\lib\ext\indicim.jar;
 D:\jre\lib\ext\jaccess.jar;D:\jre\lib\ext\ldapsec.jar;
 D:\jre\lib\ext\oldcertpath.jar
ExtClass loader could not find ClassToTrace.class in D:\jre\lib\ext\gskikm.jar
ExtClass loader could not find ClassToTrace.class in D:\jre\lib\ext\ibmjceprovider.jar
ExtClass loader could not find ClassToTrace.class in D:\jre\lib\ext\indicim.jar
ExtClass loader could not find ClassToTrace.class in D:\jre\lib\ext\jaccess.jar
ExtClass loader could not find ClassToTrace.class in D:\jre\lib\ext\ldapsec.jar
ExtClass loader could not find ClassToTrace.class in D:\jre\lib\ext\oldcertpath.jar
ExtClass loader could not find ClassToTrace
AppClass loader attempting to find ClassToTrace
AppClass loader using classpath C:\tests;D:\lib\tools.jar
AppClass loader could not find ClassToTrace.class in C:\tests
AppClass loader could not find ClassToTrace.class in D:\lib\tools.jar
AppClass loader could not find ClassToTrace
Exception in thread "main" java.lang.NoClassDefFoundError: ClassToTrace at MainClass(MainClass.java:6)

The class loaders are listed with parents before children because of the way that the standard delegation model works: Parents go first.

Notice that there is no output for the bootstrap class loader. Output is only produced for class loaders that extend java.net.URLClassLoader. Note also that class loaders are listed under their class name; if there are two instances of a class loader, it may not be possible to distinguish between them.

Resolving problems using IBM Verbose Class Loading
The IBM Verbose Class Loading option is a great way to check what the classpaths for all class loaders have been set to. It also indicates which class loader loads a given class and where it loads it from. This makes it easy to see if the correct version of a class is being loaded.

Javadump

A Javadump (also know as a Javacore) is another IBM diagnosis tool that you may find useful; to learn more about it, see the IBM Diagnostics Guide (see Related topics for a link). A Javadump is generated by the JVM when one of the following events occurs:

  • A fatal native exception occurs
  • The JVM runs out of heap space
  • A signal is sent to the JVM (for example, if Control-Break is pressed on Windows, or Control-\ on Linux)
  • The com.ibm.jvm.Dump.JavaDump() method is called

The moment that Javadump is triggered, detailed information is recorded in a date-stamped text file saved in the current working directory. This information includes data about threads, locks, stacks, and so on, as well as a rich set of information about the class loaders in the system.

Interpreting the class loading section of a Javadump
The exact information that is provided in a Javadump file depends on the platform on which the JVM is running. The class loader section includes:

  • The defined class loaders and the relationship between them
  • Lists of classes loaded by each class loader

The following is a snapshot of the class loader information taken from a Javadump:

CL subcomponent dump routine
============================
 Classpath Z(D:\jre\lib\core.jar),...
 Oldjava mode false
 Bootstrapping false 
 Verbose class dependencies false
 Class verification VERIFY_REMOTE
 Namespace to classloader 0x00000000
 Start of cache entry pool 0x44D85430
 Start of free cache entries 0x44D86204
 Location of method table 0x44C23AA0 
 Global namespace anchor 0x00266894
 System classloader shadow 0x00376068
 Classloader shadows 0x44D7BA60
 Extension loader 0x00ADB830
 System classloader 0x00ADB7B0 
 Classloader summaries
         12345678: 1=primordial,2=extension,3=shareable,4=middleware,
                   5=system,6=trusted,7=application,8=delegating
         -----ta- Loader sun/misc/Launcher$AppClassLoader(0x44D7BA60),
                  Shadow 0x00ADB7B0,
                  Parent sun/misc/Launcher$ExtClassLoader(0x00ADB830)
                 Number of loaded classes 1
                 Number of cached classes 260
                 Allocation used for loaded classes 1
                 Package owner 0x00ADB7B0 
         -xh-st-- Loader sun/misc/Launcher$ExtClassLoader(0x44D71288),
                  Shadow 0x00ADB830,
                  Parent *none*(0x00000000)
                 Number of loaded classes 0
                 Number of cached classes 0
                 Allocation used for loaded classes 3
                 Package owner 0x00ADB830 
         p-h-st-- Loader *System*(0x00376068), Shadow 0x00000000  
                 Number of loaded classes 304 
                 Number of cached classes 304         
                 Allocation used for loaded classes 3     
                 Package owner 0x00000000           
 ClassLoader loaded classes                  
         Loader sun/misc/Launcher$AppClassLoader(0x44D7BA60)  
                 HelloWorld(0x00ACF0E0)
         Loader sun/misc/Launcher$ExtClassLoader(0x44D71288) 
         Loader *System*(0x00376068)            
                 java/io/WinNTFileSystem(0x002CD118)        
                 java/lang/Throwable(0x002C03A8)            
                 java/lang/IndexOutOfBoundsException(0x44D45208) 
                 java/lang/UnsatisfiedLinkError(0x44D42D38) 
....................classes left out to save space........................
                 [Ljava/lang/Class;(0x002CA9E8)  
                 java/io/InputStream(0x002C9818)  
                 java/lang/Integer$1(0x002C83E8)          
                 java/util/Dictionary(0x002C4298)

In this example, there are only the three standard class loaders:

  • The system class loader (sun/misc/Launcher$AppClass loader)
  • The extension class loader (sun/misc/Launcher$ExtClass loader)
  • The bootstrap class loader (*System*)

The Classloader summaries section provides details about each class loader in the system, including the type of the class loader. In this series of articles, the types of interest are the primordial, extension, system, application, and delegation (used in reflection). The other types (shareable, middleware, and trusted) are used in the Persistent Reusable JVM, which is beyond the scope of these articles (see the Persistent Reusable JVM User Guide for more information; there's a link in the Related topics section below). The summaries section also shows the parent class loader: The parent of the system class loader is sun/misc/Launcher$ExtClass loader(0x00ADB830). This parent address corresponds to the native data structure of the parent class loader (called the shadow).

The ClassLoader loaded classes section lists the classes loaded by each class loader. In this example, the system class loader has only loaded one class, HelloWorld (at address 0x00ACF0E0).

Resolving problems using Javadumps
Using the information provided in the Javadump, it is possible to ascertain which class loaders exist within the system. This includes any user-defined class loaders. From the lists of loaded classes, it is possible to find out which class loader loaded a particular class. If the class cannot be found, that means that it was not loaded by any of the class loaders present in the system (which would usually result in a ClassNotFoundException).

Other types of problems that could be diagnosed using a Javadump include:

  • Class loader namespace problems. A class loader namespace is a combination of a class loader and all the classes that it has loaded. For example, if a particular class is present but is loaded by the wrong class loader (sometimes resulting in a NoClassDefFoundError), then the namespace is incorrect -- that is, the class is on the wrong classpath. To rectify such problems, try putting the class in a different location -- in the normal Java classpath, for instance -- and make sure that it gets loaded by the system class loader.
  • Class loader constraint problems. We'll discuss an example of this kind of problem in the final article in this series.

Java method tracing

The IBM JVM has a built-in method tracing facility. This allows methods in any Java code (including the core system classes) to be traced without modification to the code. Because this facility can provide a large amount of data, you can control the level of trace in order to zero in on the information that you want.

The option for enabling trace varies depending on the release of the JVM. For details of these options, refer to the IBM Diagnostics Guides (see Related topics for a link).

Here are some example command lines:

To trace all java.lang.ClassLoader methods when running HelloWorld in IBM Java 1.4.2:

java -Dibm.dg.trc.methods=java/lang/ClassLoader.*() -Dibm.dg.trc.print=mt HelloWorld

To trace the loadClass() method in ClassLoader and the methods in HelloWorld, also in IBM Java 1.4.2:

java -Dibm.dg.trc.methods=java/lang/ClassLoader.loadClass(),HelloWorld.*() 
  -Dibm.dg.trc.print=mt HelloWorld

Interpreting method trace output
Here is a sample of method trace output (using the second command line from the previous section).

Sample of method trace output
...
 > java/lang/ClassLoader.loadClass Bytecode method, This = 0x00D2B7B0, Signature: (Ljava/lang/String;)Ljava/lang/Class;
  > java/lang/ClassLoader.loadClass Bytecode method, This = 0x00D2B7B0, Signature: (Ljava/lang/String;Z)Ljava/lang/Class;
   > java/lang/ClassLoader.loadClass Bytecode method, This = 0x00D2B830, Signature: (Ljava/lang/String;Z)Ljava/lang/Class;
   < * java/lang/ClassLoader.loadClass Bytecode method, looking for matching catch block for java.lang.ClassNotFoundException
  < java/lang/ClassLoader.loadClass Bytecode method
 < java/lang/ClassLoader.loadClass Bytecode method
 > HelloWorld.main Bytecode static method, Signature: ([Ljava/lang/String;)V
 < HelloWorld.main Bytecode static method

Each line of trace provides more information than shown above. Let's look at one of the above lines in full:

12:57:41.277 0x002A23C0 04000D   > java/lang/ClassLoader.loadClass Bytecode method, 
       This = 0x00D2B830, Signature: (Ljava/lang/String;Z)Ljava
/lang/Class;

This tracing includes:

  • 12:57:41.277: The timestamp of method entry or exit.
  • 0x002A23C0: The thread ID.
  • 04000D: An internal JVM trace point used by some advanced diagnostics.
  • The remaining information shows whether a method is being entered (>) or exited (<), followed by details of the method.

Resolving problems with method trace
Method tracing can be used to resolve different types of problems, including:

  • Performance hotspots: Using timestamps, it is possible to find methods that take a significant amount of time to execute.
  • Hangs: The last method entry is usual a good indication of where the application has hung.
  • Incorrect objects: Using the address, it is possible to check that methods are being invoked on the desired object by matching to the address on the constructor call for that object.
  • Unexpected code paths: By following the entry and exit points, it is possible to see any unexpected code paths taken by the program.
  • Other faults: The last method entry is usual a good indication of where the fault has occurred.

What's next

In this article, you learned the fundamentals of class loading in JVMs and the debugging features available in the IBM JVM. In the next article in this series, you'll learn how to apply this knowledge to understand and resolve various class loading problems typically encountered when running Java applications.


Downloadable resources


Related topics


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Java development
ArticleID=99058
ArticleTitle=Demystifying class loading problems, Part 1: An introduction to class loading and debugging tools
publish-date=11292005