Jar hell: how to use a classloader to replace one jar library version with another at runtime

Question

I'm still relatively new to Java, so please bear with me.

My issue is that my Java application depends on two libraries. Let's call them Library 1 and Library 2. Both of these libraries share a mutual dependency on Library 3. However:

• Library 1 requires exactly version 1 of Library 3.
• Library 2 requires exactly version 2 of Library 3.

This is exactly the definition of JAR hell (or at least one its variations). As stated in the link, I can't load both versions of the third library in the same classloader. Thus, I've been trying to figure out if I could create a new classloader within the application to solve this problem. I've been looking into URLClassLoader, but I've not been able to figure it out.

Here's an example application structure that demonstrates the problem. The Main class (Main.java) of the application tries to instantiate both Library1 and Library2 and run some method defined in those libraries:

Main.java (original version, before any attempt at a solution):

public class Main {
    public static void main(String[] args) {
        Library1 lib1 = new Library1();
        lib1.foo();

        Library2 lib2 = new Library2();
        lib2.bar();
    }
}

Library1 and Library2 both share a mutual dependency on Library3, but Library1 requires exactly version 1, and Library2 requires exactly version 2. In the example, both of these libraries just print the version of Library3 that they see:

Library1.java:

public class Library1 {
  public void foo() {
    Library3 lib3 = new Library3();
    lib3.printVersion();    // Should print "This is version 1."
  }
}

Library2.java:

public class Library2 {
  public void foo() {
    Library3 lib3 = new Library3();
    lib3.printVersion();    // Should print "This is version 2." if the correct version of Library3 is loaded.
  }
}

And then, of course, there are multiple versions of Library3. All they do is print their version numbers:

Version 1 of Library3 (required by Library1):

public class Library3 {
  public void printVersion() {
    System.out.println("This is version 1.");
  }
}

Version 2 of Library3 (required by Library2):

public class Library3 {
  public void printVersion() {
    System.out.println("This is version 2.");
  }
}

When I launch the application, the classpath contains Library1 (lib1.jar), Library2 (lib2.jar), and version 1 of Library 3 (lib3-v1/lib3.jar). This works out fine for Library1, but it won't work for Library2.

What I somehow need to do is replace the version of Library3 that appears on the classpath before instantiating Library2. I was under the impression that URLClassLoader could be used for this, so here is what I tried:

Main.java (new version, including my attempt at a solution):

import java.net.*;
import java.io.*;

public class Main {
  public static void main(String[] args)
    throws MalformedURLException, ClassNotFoundException,
          IllegalAccessException, InstantiationException,
          FileNotFoundException
  {
    Library1 lib1 = new Library1();
    lib1.foo();     // This causes "This is version 1." to print.

    // Original code:
    // Library2 lib2 = new Library2();
    // lib2.bar();

    // However, we need to replace Library 3 version 1, which is
    // on the classpath, with Library 3 version 2 before attempting
    // to instantiate Library2.

    // Create a new classloader that has the version 2 jar
    // of Library 3 in its list of jars.
    URL lib2_url = new URL("file:lib2/lib2.jar");        verifyValidPath(lib2_url);
    URL lib3_v2_url = new URL("file:lib3-v2/lib3.jar");  verifyValidPath(lib3_v2_url);
    URL[] urls = new URL[] {lib2_url, lib3_v2_url};
    URLClassLoader c = new URLClassLoader(urls);

    // Try to instantiate Library2 with the new classloader    
    Class<?> cls = Class.forName("Library2", true, c);
    Library2 lib2 = (Library2) cls.newInstance();

    // If it worked, this should print "This is version 2."
    // However, it still prints that it's version 1. Why?
    lib2.bar();
  }

  public static void verifyValidPath(URL url) throws FileNotFoundException {
    File filePath = new File(url.getFile());
    if (!filePath.exists()) {
      throw new FileNotFoundException(filePath.getPath());
    }
  }
}

When I run this, lib1.foo() causes "This is version 1." to be printed. Since that's the version of Library3 that's on the classpath when the application starts, this is expected.

However, I was expecting lib2.bar() to print "This is version 2.", reflecting that the new version of Library3 got loaded, but it still prints "This is version 1."

Why is it that using the new classloader with the right jar version loaded still results in the old jar version being used? Am I doing something wrong? Or am I not understanding the concept behind classloaders? How can I switch jar versions of Library3 correctly at runtime?

I would appreciate any help on this problem.

Answer 1

I can't believe that for more than 4 years no one has answered this question correctly.

https://docs.oracle.com/javase/8/docs/api/java/lang/ClassLoader.html

The ClassLoader class uses a delegation model to search for classes and resources. Each instance of ClassLoader has an associated parent class loader. When requested to find a class or resource, a ClassLoader instance will delegate the search for the class or resource to its parent class loader before attempting to find the class or resource itself. The virtual machine's built-in class loader, called the "bootstrap class loader", does not itself have a parent but may serve as the parent of a ClassLoader instance.

Sergei, the problem with your example was that Library 1,2 & 3 were on the default class path, so the Application classloader which was the parent of your URLClassloder was able to load the classes from Library 1,2 & 3.

If youremove the libraries from the classpath, the Application classloader won't be able to resolve classes from them so it will delegate resolvation to its child - the URLClassLoader. So that is what you need to do.

Answer 2

You need to load both Library1 and Library2 in separate URLClassloaders. (In your current code, Library2 is loaded in a URLClassloader whose parent is the main classloader - which has already loaded Library1.)

Change your example to something like this:

URL lib1_url = new URL("file:lib1/lib1.jar");        verifyValidPath(lib1_url);
URL lib3_v1_url = new URL("file:lib3-v1/lib3.jar");  verifyValidPath(lib3_v1_url);
URL[] urls1 = new URL[] {lib1_url, lib3_v21_url};
URLClassLoader c1 = new URLClassLoader(urls1);

Class<?> cls1 = Class.forName("Library1", true, c);
Library1 lib1 = (Library1) cls1.newInstance();    


URL lib2_url = new URL("file:lib2/lib2.jar");        verifyValidPath(lib2_url);
URL lib3_v2_url = new URL("file:lib3-v2/lib3.jar");  verifyValidPath(lib3_v2_url);
URL[] urls2 = new URL[] {lib2_url, lib3_v2_url};
URLClassLoader c2 = new URLClassLoader(url2s);


Class<?> cls2 = Class.forName("Library2", true, c);
Library2 lib2 = (Library2) cls2.newInstance();

Answer 3

Trying to get rid of classpath lib2 and invoke the bar() method by reflection:

try {
    cls.getMethod("bar").invoke(cls.newInstance());
} catch (Exception e) {
    e.printStackTrace();
}

gives following output:

Exception in thread "main" java.lang.ClassNotFoundException: Library2
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at Main.main(Main.java:36)

This means you're in fact loading Library2 from classpath using default classloader, not your custom URLClassLoader.

Answer 4

classloader are something simple in concept, but actually quite complex

I recommend you not to use a custom solution

you have some partial open source solutions, such as DCEVM

but there are also very good commercial product, such as JRebel

Answer 5

Use jar class loader which can be used to load classes from jar files on runtime.

Answer 6

You can use a ParentLastClassloader to solve Jar Hell. Please check this blog post out

Answer 7

I'd suggest a solution using JBoss-Modules.

You only need to create a module for Library1:

    final ModuleIdentifier module1Id = ModuleIdentifier.fromString("library1");
    ModuleSpec.Builder moduleBuilder = ModuleSpec.build(module1Id);
    JarFile jarFile = new JarFile("lib/lib3-v1/lib3.jar", true);
    ResourceLoader rl1 = ResourceLoaders.createJarResourceLoader("lib3-v1", jarFile);
    moduleBuilder.addResourceRoot(ResourceLoaderSpec.createResourceLoaderSpec(
            rl1
            ));
    moduleBuilder.addResourceRoot(ResourceLoaderSpec.createResourceLoaderSpec(
            TestResourceLoader.build()
            .addClass(Library1.class)
            .create()
            ));
    moduleBuilder.addDependency(DependencySpec.createLocalDependencySpec());
    moduleLoader.addModuleSpec(moduleBuilder.create());

In a similar way you can create a module for Library2.

And then you can create a module for Main depending in these two:

    //Building main module
    final ModuleIdentifier moduleMainId = ModuleIdentifier.fromString("main");
    moduleBuilder = ModuleSpec.build(moduleMainId);
    moduleBuilder.addResourceRoot(ResourceLoaderSpec.createResourceLoaderSpec(
            TestResourceLoader.build()
            .addClass(Main.class)
            .create()
            ));
    //note the dependencies
    moduleBuilder.addDependency(DependencySpec.createModuleDependencySpec(module1Id, true, false));
    moduleBuilder.addDependency(DependencySpec.createModuleDependencySpec(module2Id, true, false));
    moduleBuilder.addDependency(DependencySpec.createLocalDependencySpec());
    moduleLoader.addModuleSpec(moduleBuilder.create());

Finally you can load the Main class and run it through reflection:

    Module moduleMain = moduleLoader.loadModule(moduleMainId);
    Class<?> m = moduleMain.getClassLoader().loadClass("tmp.Main");
    Method method = m.getMethod("main", String[].class);
    method.invoke(null, (Object) new String[0]);

You can download the full working example here