Simplify your application delivery with One-JAR

It has been noted that history repeats itself, first as tragedy and then as farce. I recently experienced this first hand when I had to deliver a runnable Java application to a client. I've done this many times and it's always fraught with complications. There is plenty of room for error when gathering all of an application's JAR files, writing the launch scripts for DOS and Unix (and Cygwin), and making sure that the client's environment variables all point to the right place. If everything goes perfectly the application will arrive and run as it should. When things go wrong, as they usually do, many hours of client-side support are the result.

After recently talking a bemused client through a host of ClassNotFound exceptions, I decided I'd had enough. I was going to find a way to pack my application into a single JAR file and give my clients a simple mechanism (such as java -jar ) to run it.

The result is One-JAR, a very simple software packaging solution that utilizes a Java custom classloader to dynamically load all your application classes from inside a single archive, while preserving the structure of supporting JAR files. In this article, I'll walk you through my process of developing One-JAR, and then tell you how to use it to deliver your runnable applications in a self-contained file.

Overview of One-JAR

Before I describe the details of One-JAR, let me first discuss the goals I had in building it. I decided that a One-JAR archive should:

• Be executable using the java -jar mechanism.
• Be able to contain all of the files needed by an application -- that is, both classes and resources in their original (unexpanded) form.
• Have a simple internal structure that could be assembled using only a jar tool.
• Be invisible to the original application -- that is, the original application should be able to be packaged inside a One-JAR archive without modification.

Back to top

Problems and solutions

The biggest hurdle I tackled in the process of developing One-JAR was how to load JAR files that are contained inside another JAR file. The Java classloader sun.misc.Launcher$AppClassLoader, which takes over at the start of java -jar, only knows how to do two things:

• Load classes/resources that appear at the root of a JAR file.
• Load classes/resources that are in codebases pointed to by the META-INF/MANIFEST.MF Class-Path attribute.

Moreover, it deliberately ignores any environment variable settings for CLASSPATH or the command-line argument -cp that you supply. And it does not know how to load classes or resources from a JAR file that is contained inside another JAR file.

Clearly, I would need to get around this to meet my goals for One-JAR.

Solution 1: Expand supporting JAR files

My first attempt at creating a single executable JAR file was to do the obvious and expand the supporting JAR files inside the deliverable JAR file, which we'll call main.jar. Given an application class called com.main.Main, and assuming it depends on two classes -- com.a.A (inside a.jar) and com.b.B (inside b.jar) -- the One-JAR file would look like this:

    main.jar
    |  com/main/Main.class
    |  com/a/A.class
    |  com/b/B.class

The fact that A.class originally came from a.jar has been lost, as has the original location of B.class. While this may appear to be a minor point, it can cause real problems, as I'll explain in a moment.

Expanding the supporting JAR files to the filesystem to create a flat structure can be quite time-consuming. It also requires working with build tools like Ant to expand and re-archive the supporting classes.

Aside from this minor annoyance, I quickly encountered two serious problems with expanding the supporting JAR files:

• If a.jar and b.jar contain a resource with the same pathname (say, log4j.properties ) which one do you pick?
• What do you do if the license for b.jar expressly requires you to redistribute it in an unmodified form. You can't expand it like this without violating the terms of that license.

I felt that these limitations warranted another approach.

Solution 2: The MANIFEST Class-Path

I decided to investigate a mechanism in the java -jar loader that will load classes which are specified inside a special file in the archive named META-INF/MANIFEST.MF. By specifying a property called Class-Path I hoped to be able to add other archives to the bootstrap classloader. Here is what such a One-JAR file would look like:

    main.jar
    |  META-INF/MANIFEST.MF
    |  +  Class-Path: lib/a.jar lib/b.jar
    |  com/main/Main.class
    |  lib/a.jar
    |  lib/b.jar

Did this work? Well it appeared to until I moved the main.jar file somewhere else and tried to run it. In order to assemble main.jar I had created a subdirectory named lib and pushed a.jar and b.jar down into it. Unfortunately, the application classloader was simply picking up the supporting JAR files from the file-system. It wasn't loading classes from the embedded JAR files.

To get around this, I tried using Class-Path with several variations on the rather arcane jar:!/ syntax (see "A note and a clue"), but I couldn't get anything to work. What I could do was deliver a.jar and b.jar separately and scatter them into the filesystem alongside main.jar; but that was exactly the kind of thing I wanted to avoid.

Back to top

Enter the JarClassLoader

At this point I was getting frustrated. How could I make an application load its classes from a lib directory inside its own JAR file? I decided that I would have to create a custom classloader to do the heavy lifting. Writing custom classloaders isn't a task to be undertaken lightly. While they're not really that complex, a classloader has such a profound impact on the application it controls that it becomes difficult to diagnose and interpret failures when they occur. While a complete treatment of classloading is beyond the scope of this article (see Resources), I will go over some basic concepts to ensure that you get the most from the following discussion.

Loading a class

When a JVM comes across an object whose class isn't known, it invokes a classloader. The job of the classloader is to locate the bytecodes for the class (based on its name) and then hand those bytes over to the JVM, which links them into the rest of the system and makes the new class available to the running code. The crucial class in the JDK is java.lang.Classloader, and the loadClass method which is outlined here:

    public abstract class ClassLoader {
        ...
        protected synchronized Class loadClass(String name, boolean resolve)
            throws ClassNotFoundException {...}
    }

The main entry point to the ClassLoader class is the loadClass() method. You will note that ClassLoader is an abstract class but it doesn't declare any abstract methods, which leaves you without a clue that loadClass() is the method to focus on. In fact, it isn't the main method to focus on: back in the good old days of JDK 1.1 classloaders, loadClass() was the only place you could effectively extend a classloader, but since JDK 1.2 it is best left alone to do what it already does, which is the following:

• Checks to see if the class is already loaded.
• Checks to see if a parent classloader can load it.
• Calls findClass(String name) to let a derived classloader load the class.

The implementation of ClassLoader.findClass() is to throw a new ClassNotFoundException, and is the first method to focus on when implementing a custom classloader.

When is a JAR file not a JAR file?

In order to be able to load the classes inside a JAR file inside a JAR file (the crucial issue, as you'll recall), I first had to be able to open and read the top-level JAR file (main.jar above). It turned out that because I was using the java -jar mechanism, the first (and only) element on the java.class.path system property was the full path name of the One-JAR file! You can get to it as follows:

    jarName = System.getProperty("java.class.path");

My next step was to iterate over all my application's JAR file entries and load them into memory, as shown in Listing 1:

Listing 1. Iterating to find embedded JAR files

    JarFile jarFile = new JarFile(jarName);
    Enumeration enum = jarFile.entries();
    while (enum.hasMoreElements()) {
        JarEntry entry = (JarEntry)enum.nextElement();
        if (entry.isDirectory()) continue;
        String jar = entry.getName();
        if (jar.startsWith(LIB_PREFIX) || jar.startsWith(MAIN_PREFIX)) {
            // Load it! 
            InputStream is = jarFile.getInputStream(entry);
            if (is == null) 
                throw new IOException("Unable to load resource /" + jar + " using " + this);
            loadByteCode(is, jar);	
            ...

Note that LIB_PREFIX evaluates to the string lib/ and MAIN_PREFIX evaluates to the string main/. I wanted to load the bytecodes for anything starting with either lib/ or main/ into memory for use by the classloader and ignore any other JAR file entries in the loop.

The main directory

I've talked about the role of the lib/ subdirectory, but what's this main/ directory for? In brief, the delegation mode for classloaders required that I put the main class com.main.Main into its own JAR file so that it would be able to locate the library classes (on which it depends). The new JAR file looked like so:

	one-jar.jar
	|  META-INF/MANIFEST.MF
	|  main/main.jar
	|  lib/a.jar
	|  lib/b.jar

In Listing 1 above, the loadByteCode() method takes the stream from the JAR file entry and an entry name, loads the bytes for the entry into memory, and assigns it up to two names depending on whether the entry represents a class or a resource. The best way to illustrate this is by example. Suppose a.jar contains a class A.class , and a resource A.resource. The One-JAR classloader builds the following Map structure named JarClassLoader.byteCode with a single key-value pair for classes, and two keys for a resource.

Figure 1. The in-memory structures of One-JAR

If you stare long enough at Figure 1 you can see that class entries are keyed based on their classnames, and resources are keyed on a pair of names: a global name and a local name. This mechanism is used to resolve resource-name conflicts: if two library JAR files define a resource with the same global name, the local names will be used based on the stack-frame of the caller. See Resources for further details.

Finding the classes

Recall that I left off in my overview of classloading at the findClass() method. Method findClass() takes the name of a class as a String and must locate and define the bytecodes that that name represents. Since loadByteCode kindly builds a Map between a class name and its bytecode, implementing this is now very straightforward: simply look up the bytecodes based on class name, and call defineClass(), as shown in Listing 2:

Listing 2. An outline for findClass()

    protected Class findClass(String name) throws ClassNotFoundException {
        ByteCode bytecode = (ByteCode)JarClassLoader.byteCode.get(name);
        if (bytecode != null) {
            ...
            byte bytes[] = bytecode.bytes;
            return defineClass(name, bytes, pd);
        }
        throw new ClassNotFoundException(name);
    }

Back to top

Loading resources

During the development of One-JAR, findClass was the first thing I got working as a proof of concept. But when I started to deploy more complex applications I found I had to deal with loading resources as well as classes. This is where things got slippery. Casting about for a suitable method in ClassLoader to override in order to lookup resources, I picked the one with which I was most familiar, shown in Listing 3:

Listing 3. The getResourceAsStream() method

    public InputStream getResourceAsStream(String name) {
        URL url = getResource(name);
        try {
            return url != null ? url.openStream() : null;
        } catch (IOException e) {
            return null;
        }
    }

Alarm bells should have been sounding at this point: I simply couldn't understand why URLs were being used to locate the resources. So I ignored this implementation and inserted my own, shown in Listing 4:

Listing 4. One-JAR's implementation of getResourceAsStream()

    public InputStream getResourceAsStream(String resource) {
        byte bytes[] = null;
        ByteCode bytecode = (ByteCode)byteCode.get(resource);
        if (bytecode != null) {
            bytes = bytecode.bytes; 
        }
        ...
        if (bytes != null) {
            return new ByteArrayInputStream(bytes);
        }
        ...
        return null;
    }

One last hurdle

My new implementation of the getResourceAsStream() method seemed to do the trick, until I tried to One-JAR an application that loaded a resource using the URL url = object.getClass().getClassLoader().getResource() pattern; at which point things fell apart. Why? Because the URLs returned by the default implementation of ClassLoader are null, which broke the callers code.

At this point things started to get really confusing. I had to figure out what URL should be used to refer to a resource inside a JAR file in the lib/ directory. Would it be something like jar:file:main.jar!lib/a.jar!com.a.A.resource?

I tried all the possible combinations I could think of, none of which worked. The jar: syntax simply doesn't support nested JAR files, which left me facing an apparent dead-end to the whole One-JAR approach. While most applications don't seem to use ClassLoader.getResource some definitely do, and I wasn't happy with an exclusion that said "If your application uses ClassLoader.getResource() you can't use One-JAR."

And finally, a solution ...!

While I was trying to figure out the jar: syntax, I stumbled onto the mechanism by which the Java Runtime Environment maps URL prefixes to handlers. This was the clue I needed to fix the findResource problem: I would simply invent my own protocol prefix called onejar:. I could then map the new prefix to a protocol handler, which would return the byte stream for the resource, as shown in Listing 5. Note that Listing 5 represents code in two files, the JarClassLoader and a new file called com/simontuffs/onejar/Handler.java.

Listing 5. findResource and the onejar: protocol

com/simontuffs/onejar/JarClassLoader.java

    protected URL findResource(String $resource) {
        try {
            // resolve($resource) returns the name of a resource in the
            // byteCode Map if it is known to this classloader.
            String resource = resolve($resource);
            if (resource != null) {
                // We know how to handle it.
                return new URL(Handler.PROTOCOL + ":" + resource); 
            }
            return null;
        } catch (MalformedURLException mux) {
            WARNING("unable to locate " + $resource + " due to " + mux);
        }
        return null;
    }

com/simontuffs/onejar/Handler.java

    package com.simontuffs.onejar;
    ...
    public class Handler extends URLStreamHandler {
        /**
         * This protocol name must match the name of the package in which this class
         * lives.
         */
        public static String PROTOCOL = "onejar";
        protected int len = PROTOCOL.length()+1;
        
        protected URLConnection openConnection(URL u) throws IOException {
            final String resource = u.toString().substring(len);
            return new URLConnection(u) {
                public void connect() {
                }
                public InputStream getInputStream() {
                    // Use the Boot classloader to get the resource.  There
                    // is only one per one-jar.
                    JarClassLoader cl = Boot.getClassLoader();
                    return cl.getByteStream(resource);
                }
            };
        }
    }

Back to top

Bootstrapping the JarClassLoader

You probably just have one remaining question at this point: How did I insert the JarClassLoader into the launch sequence so that it could start loading classes from the One-JAR file in the first place? The precise details are beyond the scope of this article; but ,basically, instead of using the main class com.main.Main as the META-INF/MANIFEST.MF/Main-Class attribute, I created a new bootstrap main class, com.simontuffs.onejar.Boot , which is specified as the Main-Class attribute. The new class does the following:

• Creates a new JarClassLoader.
• Uses the new loader to load com.main.Main from main/main.jar (based on the META-INF/MANIFEST.MF Main-Class entry in main.jar).
• Calls com.main.Main.main(String[]) (or whatever the name of the Main-Class is in the main.jar/MANIFEST.MF file) by loading the class and using reflection to invoke main(). Arguments passed on the One-JAR command line are passed through to the application main method without modification.

Back to top

In conclusion

If all this has left your head spinning, don't worry: Using One-JAR is much simpler than trying to understand how it works. With the advent of the FatJar Eclipse Plugin (see FJEP in Resources), Eclipse users can now create a One-JAR application by selecting a checkbox in a Wizard. The dependent libraries are placed into a lib/ directory, the main program and classes are placed into main/main.jar, and the META-INF/MANIFEST.MF file is written automatically. If you use JarPlug (again, see Resources) you can look inside the JAR file you built and launch it from within the IDE.

Overall, One-JAR is a simple yet powerful solution to the problem of packaging applications for delivery. It doesn't provide for every possible application scenario, however. For example, if your application uses an older-style JDK 1.1 classloader that doesn't delegate to its parent, then the classloader will fail to locate classes from within a nested JAR file. You could overcome this by building and deploying a "wrapping" classloader to modify the recalcitrant classloader, although this would entail using bytecode-manipulation techniques with tools such as Javassist or Byte Code Engineering Library (BCEL).

You might also encounter problems with some particular types of classloader used by embedded application and Web servers. Specifically, you could have problems with classloaders that don't first delegate to a parent classloader, or those that look for codebases in the filesystem. It should help that One-JAR includes a mechanism by which it can expand JAR file entries into the filesystem. The mechanism is controlled by a One-JAR-Expand attribute in the META-INF/MANIFEST.MF file. Alternatively, you could try using bytecode manipulation to modify the classloaders on the fly, without violating the integrity of the supporting JAR files. If you go this route, each individual situation will probably require a customized wrapping classloader.

See Resources to download the FatJar Eclipse Plugin and JarPlug, and to learn more about One-JAR.

Resources

• You'll find more documentation, downloads, and examples for One-JAR on Sourceforge.net.
• One-JAR has recently been integrated with the Fat JAR Eclipse Plugin (FJEP), a tool that helps you build flattened out JAR files for deployment. Available from release 0.0.12.
• The Java Archive Eclipse Plugin tool lets you view and launch JAR files within Eclipse.
• Check out the following bytecode manipulation tools (my personal preference is Javassist):
  • Javassist
  • Byte Code Engineering Library (BCEL)
• "Java programming dynamics, Part 1: Classes and class loading" (developerWorks, April 2003) gives a high-level introduction to the subject of classloaders, and has more resources for you to follow. Author Dennis Sosnoski also discusses Javassist and BCEL later in this same series; a link to those articles is included in this first part.
• "J2EE class loading demystified" (developerWorks, August 2002) is a beginner's look at classloading.
• David Gallardo's "Getting started with the Eclipse platform" (developerWorks, April 2003) introduces the features and architecture behind Eclipse.
• You'll find articles about every aspect of Java programming in the developerWorks Java technology zone.
• Browse for books on these and other technical topics.

Comments

Dig deeper into Java technology on developerWorks

• Overview
• New to Java programming
• Technical library (tutorials and more)
• Forums
• Blogs
• Communities
• Downloads and products
• Open source projects
• Standards
• Events

• 

  Bluemix Developers Community

  Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

• 

  developerWorks Weekly Newsletter

  Keep up with the best and latest technical info to help you tackle your development challenges.

• 

  DevOps Services

  Software development in the cloud. Register today to create a project.

• 

  IBM evaluation software

  Evaluate IBM software and solutions, and transform challenges into opportunities.

Back to top