Demystifying class loading problems, Part 3: Tackling more unusual class loading problems

Understand class loading and quash subtle exceptions

This four-part article series examines Java™ class loading, in an effort to help application developers understand and debug problems they may encounter. In Part 3, authors Lakshmi Shankar and Simon Burns from the IBM Hursley Labs build on the first two parts of the series and detail different kinds of class loading problems, including those related to classpaths, class visibility, and garbage collection.

Lakshmi Shankar, Java Technology Center Development Team, IBM Hursley Labs

Lakshmi ShankarLakshmi Shankar is a Software Engineer in IBM Hursley Labs, UK. He has worked for IBM for more than three years and has a broad range of experience, having worked in Java performance, test, and development within Hursley Labs. Until recently he was the Class Loading component owner for IBM's Java technology. He is now a developer working as part of the Information Management team.



Simon Burns (simon_burns@uk.ibm.com), Java Technology Center Development Team, IBM Hursley Labs

Simon BurnsSimon Burns was the component owner and team lead for the Persistant Reusable JVM in the Java Technology Centre in IBM Hursley Labs. He worked in JVM development for over three years, specializing in the Persistant Reusable JVM technology and the z/OS platform. He has also worked closely with CICS, helping them to exploit this technology. Simon worked on the OSGi framework as part of the open-source Eclipse Equinox project, which has now been integrated into Eclipse 3.1. He is now working on componentization.



13 December 2005

Also available in

This article, the third in a series of four, examines some of the more complex and unusual class loading problems that you may encounter in the course of your Java development. The causes of these problems are not always obvious from the symptoms; as a result, they can be difficult and time-consuming to resolve. As with the previous articles in this series, we provide examples to illustrate the problems and then discuss the various resolution techniques.

Before you begin this article, you should be familiar with the class loader delegation model, along with the phases and stages of class linking. We recommend that you start by reading the first article in this series.

Problems related to classpaths

There is a very simple problem that often catches users setting classpaths. The example in Listings 1 and 2 illustrates this problem.

The test case creates two class loaders, each with what appears to be the same classpath. However, there is one minor difference that is actually significant: One of the classpaths has a trailing /, while the other does not. There is a class, Z (shown in Listing 2), in a subdirectory called cp, that is on both classpaths. Both class loaders will attempt to load Z:

Listing 1. ClasspathTest.java
import java.net.URL;
import java.net.URLClassLoader;

public class ClasspathTest {
	
    String userDir;
    URL withSlash;
    URL withoutSlash;

    ClasspathTest() {
        try {
            userDir = System.getProperty("user.dir");
            withSlash = new URL("file://C:/CL_Article/ClasspathIssues/cp/");
            withoutSlash = new URL("file://C:/CL_Article/ClasspathIssues/cp");
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    void run() {
        try {
            System.out.println(withSlash);
            URLClassLoader cl1 = new URLClassLoader(new URL[] { withSlash });
            Class c1 = cl1.loadClass("Z");
            System.out.println("Class Z loaded.");
            System.out.println(withoutSlash);
            URLClassLoader cl2 = new URLClassLoader(new URL[] { withoutSlash });
            Class c2 = cl2.loadClass("Z");
            System.out.println("Class Z loaded.");
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) {
        new ClasspathTest().run();
    }
}
Listing 2. Z.java
public class Z {

}

This test case produces the following output:

file://C:/CL_Article/ClasspathIssues/cp/
Class Z loaded.
file://C:/CL_Article/ClasspathIssues/cp
java.lang.ClassNotFoundException: Z
    at java.net.URLClassLoader.findClass(URLClassLoader.java:376)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:572)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
    at ClasspathTest.run(ClasspathTest.java:28)
    at ClasspathTest.main(ClasspathTest.java:36)

As you can see, each URLClassloader is passed a slightly different parameter. The first class loader, cl1, is given a classpath that has a trailing /. The classpath for the second class loader, cl2, does not have a trailing /. This is significant because the class loader assumes that any path that does not end with a / refers to a JAR file. Only those paths that end with a / are assumed to refer to directories.

Because the classpath of cl1 is regarded as a directory, that class loader is able to find class Z in that location and is able to load it. The classpath of cl2 is assumed to be a JAR file; that class loader cannot find class Z because there is no such file. As a result, cl2.loadClass() throws a ClassNotFoundException.

Obviously, the way to fix this problem is to make sure that there is a trailing / at the end of any path that refers to a directory.


Problems related to visibility of classes

There may be classes in a system that cannot be seen by a class loader. This is because a class loader can only see classes that it loads itself or that are loaded by other class loaders to which it has a reference (either directly or indirectly). In the standard class loading delegation model, the classes that are visible to a class loader are limited to those that it loads itself or those loaded by its parent and ancestor class loaders -- in other words, a class loader cannot see down.

Figure 1 illustrates an example of this kind of problem:

Figure 1. Visibility example
Visibility example

Class A is on the classpath of the system class loader, and A's superclass, B, is on the classpath of a user-defined class loader, which is a child of the system class loader. When the system class loader attempts to load class A, the load fails because it cannot see class B. This is because B is not in the classpath of the system class loader or its parent or ancestor class loaders.

The test case in Listings 3 through 5 implements this scenario:

Listing 3. VisibilityTest.java
import java.net.*;

public class VisibilityTest {
    public static void main(String[] args) {
        try {
            URLClassLoader mycl = new URLClassLoader(new URL[] { new URL(
                "file://C:/CL_Article/VisibilityTest/cp/") });

            Class c2 = mycl.loadClass("A");
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}
Listing 4. A.java
public class A extends B {
    public static void method1() {
        System.out.println("HELLO!");
    }
}
Listing 5. B.java
public class B {
}

This test case produces the following output:

Exception in thread "main" java.lang.NoClassDefFoundError: B
    at java.lang.ClassLoader.defineClass0(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:810)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:147)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:475)
    at java.net.URLClassLoader.access$500(URLClassLoader.java:109)
    at java.net.URLClassLoader$ClassFinder.run(URLClassLoader.java:848)
    at java.security.AccessController.doPrivileged1(Native Method)
    at java.security.AccessController.doPrivileged(AccessController.java:389)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:371)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:572)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:442)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:563)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
    at VisibilityTest.main(VisibilityTest.java:9)
    at VisibilityTest.main(VisibilityTest.java:10)

The only way to resolve class visibility problems is to ensure that all the application classes are visible. How exactly you place classes to ensure their visibility will vary depending on the class loading model you're using. However, if you're using the standard class loading delegation model, class visibility is a simple issue: all that is required is to ensure that there are no references pointing to a lower class space. For instance, classes in the system class loader class space should not refer to classes in the class space of a child or a descendant class loader.


Problems when overriding loadClass()

If class loaders only use the standard delegation model, then there is no need to override the loadClass() method. However, if a different model is required, then loadClass() must be overridden, in which case there are special considerations that must be taken into account.

Delegation

Listing 6 is a simple implementation of loadClass():

Listing 6. A simple loadClass() implementation
public Class loadClass(String name) throws ClassNotFoundException {
    return findClass(name);
}

Although this looks reasonable, a call to this method results in the following exception:

Exception in thread "main" java.lang.NoClassDefFoundError: java/lang/Object

This exception is thrown because the overridden loadClass() method never delegates to its parent. This implementation assumes that all classes required are on the classpath of this class loader. This implementation is fundamentally flawed because all classes (implicitly) extend java.lang.Object, which must be the version loaded by the bootstrap class loader.

This problem can be fixed by modifying the implementation of loadClass() as shown in Listing 7:

Listing 7. An improved loadClass() implementation
public Class loadClass(String name) throws ClassNotFoundException {
    Class c = null;
    try {
        c = getParent().loadClass(name);
    } catch (ClassNotFoundException e) {
    }
    if(c == null)
        c = findClass(name);
    return c;
}

The method now delegates to its parent class loader before attempting to find the class itself. This means that it now finds java.lang.Object loaded by the bootstrap class loader.

Cache

While the delegating implementation of loadClass() given in Listing 7 fixes that problem, the implementation is still is incomplete. Another error can occur when using this version of loadClass(). Here's what the exception looks like on an IBM JVM:

Exception in thread "main" java.lang.LinkageError: 
    JVMCL048:redefine of class A (&name=44CA3B08). old_cb=ACEE80,
     new_cb=ACED50, (&old_name=44CA3B08) old_name=A

And here it is on a Sun JVM:

Exception in thread "main" java.lang.LinkageError: duplicate class definition: A

This exception occurs because the application is asking the class loader to load the same class twice, and loadClass() tries to reload it from scratch. This causes a conflict between the two versions. This can be dealt with in loadClass() by first checking the cache of the class loader. If the class is found in the cache, then this version should be returned. This logic has been added to the version of the loadClass() method in Listing 8:

Listing 8. loadClass(), further refined
public Class loadClass(String name) throws ClassNotFoundException {
    Class c = findLoadedClass(name);
    if(c == null) {
        try {
            c = getParent().loadClass(name);
        } catch (ClassNotFoundException e) {
        }
        if(c == null)
            c = findClass(name);
    }
    return c;
}

This method will now work fine; however, it now follows the standard class loading delegation (cache, parent, disk). Of course, if the standard delegation model is required, then there is no need to override loadClass() in the first place. It is possible to write useful loadClass() methods that do not follow the standard delegation model; there are problems that can potentially arise as a result, but they are beyond the scope of this series.


Problems related to garbage collection and serialization

The garbage collector interacts closely with the class loader. Among other things, the collector examines the class loader data structures to determine which classes are live -- that is, are not garbage collectable. This can often lead to some unexpected problems.

Figure 2 illustrates a situation where serialization affects the garbage collection (GC) of classes and a class loader in an unexpected way:

Figure 2. Serialization example
Serialization example

In this example, SerializationTest instantiates a URLClassLoader, called loader. After loading SerializationClass, the class loader is dereferenced. The expectation is that this will allow the classes loaded by it to be garbage collected. The code for these classes is illustrated in Listings 9 and 10:

Listing 9. SerializationTest.java
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;

public class SerializationTest extends ClassLoader {

   public static void main(String args[]) {
      try {
         URLClassLoader loader = new URLClassLoader(new URL[] { new URL(
               "file://C:/CL_Article/Serialization/dir1/") });
         System.out.println("Loading SerializationClass");
         Class c = loader.loadClass("SerializationClass");
         System.out.println("Creating an instance of SerializationClass");
         c.newInstance();
         System.out.println("Dereferencing the class loader");
         c = null;
         loader = null;
         
         System.out.println("Running GC...");
         System.gc();
         System.out.println("Triggering a Javadump");
         com.ibm.jvm.Dump.JavaDump();
         
      } catch (MalformedURLException e) {
         e.printStackTrace();
      } catch (InstantiationException e) {
         e.printStackTrace();
      } catch (IllegalAccessException e) {
         e.printStackTrace();
      } catch (ClassNotFoundException e) {
         e.printStackTrace();
      }
   }
}
Listing 10. SerializationClass.java
import java.io.File;
import java.io.FileOutputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;

public class SerializationClass implements Serializable {

    private static final long serialVersionUID = 5024741671582526226L;

    public SerializationClass() {
        try {
            File file = new File("C:/CL_Article/Serialization/test.txt");
            FileOutputStream fos = new FileOutputStream(file);
            ObjectOutputStream oos = new ObjectOutputStream(fos);
            oos.writeObject(this);
            oos.reset();
            oos.close();
            fos.close();
            oos = null;
            fos = null;
            file = null;
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Using a Javadump, it is possible to discover whether the class loader has been garbage collected. (See the first article in this series for more on using Javadump.) If the following section appears in the list of class loaders, then it has not been collected:

------a- Loader java/net/URLClassLoader(0x44DC6DE0), Shadow 0x00ADB6D8,
        Parent sun/misc/Launcher$AppClassLoader(0x00ADB7B0) 
        Number of loaded classes 1 
        Number of cached classes 11      
        Allocation used for loaded classes 1      
        Package owner 0x00ADB6D8

Though dereferencing a user-defined class loader seems like a way to ensure that the classes are garbage collected, this is not actually the case. In the previous example, the problem stems from the use of java.io.ObjectOutputStream.writeObject(Object obj) and its implications on GC.

When writeObject() is invoked (to serialize SerializationClass), a reference to this class object is passed internally to ObjectStreamClass and stored in a lookup table (that is, in an internal cache). This reference is kept to speed up future serialization of the same class.

When the class loader is dereferenced, the classes that it loaded are not garbage collectable. This is because there is a live reference to the SerializationClass class from the ObjectStreamClass lookup table. ObjectStreamClass is a primordial class and therefore is never garbage collected. The lookup table is referenced from a static field in ObjectStreamClass and is kept in the class itself rather than in an instance of it. As a result, the reference to SerializationClass exists for the lifetime of the JVM, and the class thus cannot be garbage collected. Importantly, the SerializationClass class has a reference to its defining class loader, and so it cannot be completely dereferenced either.

To avoid this problem, any classes that are to be serialized should be loaded by a class loader that does not need to be garbage collected -- by the system class loader, for example.


What's next

In this article, you learned about some of the more intricate problems that can occur in class loading. In the final article in this series, we'll look at two of the most complex problems that can occur: deadlocks and constraint violations.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Java technology on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Java technology
ArticleID=100160
ArticleTitle=Demystifying class loading problems, Part 3: Tackling more unusual class loading problems
publish-date=12132005