java class loader

Problem

In a project, We use a stub jar file generated by Axis2. It always throw: java.lang.NoSuchFieldError: localClientNameTracker

 

While in another project, everything is ok using same jar file. No exception is thrown.

 

CLUE

1. Guess it's because environment, but has no idea about the cause. 
2. Clean the project and re-add all the lib in Eclipse, the issue disappear 
3. If we put axis1 before axis2 on classpath, the issue disappear but the project broken
4. If we put axis2 before axis1 on classpath, the project ok but the issue occur

 

Suspect

 

It seem because of Axis1 and axis2 jar file conflict,  we need dig into detail to find out which jar cause this conflict.

 

 

diagnosis 

1. Search the internet, try to find the Axis1 and Axis2 jar conflict, do not any any hit
2. Search the internet, try to understanding class loader

•  
  • prints a trace of the class loading process on the console in Eclipse . From an article on IBM java programming dynamics series, part1 Java classes and class loadin g. -verbose:class
  • In Eclipse, we append Eclipse console to the File , Run Configurations -> Common -> Standard Input and Output
  • Use Beyond compare to compare the two trace file, one is success trace, the other is fail trace

Cause

 

1. localClientNameTracker is a field in a parent class called ServerReuqest (x.jar)
2. There is happened another jar file(y.jar) which has same class with exactly same class name and package name, while they are different version.
3. When add y.jar file before x.jar into classpath, then JVM search y.jar first and load ServerRequest from y.jar file. Then NoSuchFieldError occur 
4. When add x.jar file before y.jar into classpath, then JVM search x.jar first and load ServerRequest from x.jar file, everything is OK.

Solution1

According to problem solving policy : Try to find another ways which achieve same target but do not this problem. So our first solution is remove two version of same classes. Only keep a latest version of ServerRequest

 

 

Solution2

How about keep two version of same jar file.  According to Managing Component Dependencies Using ClassLoaders

We can let each jar file have one classloader, see the diagram 

 

The policy of our class loader

1. Each Jar file has own class loader, JarClassLoader, which extended from URLClassLoader and load class from a single jar file. If can not load from jar, delegate to AppJarClassLoader.
2. Add a Customer class loader (AppJarClassLoader ) to boot class path as System class loader, which iterate each JarClassLoader. If can not load from jars, delegate to the parent.  -Xbootclasspath/a:loader.jar -Djava.system.class.loader=com.loader.AppJarClassLoader  
3.  If no parent delegation, NoClassDefError may be occur.

package com.loader; import java.io.File; import java.net.MalformedURLException; import java.net.URL; import java.net.URLClassLoader; public class JarClassLoader extends URLClassLoader { private AppJarClassLoader mcl; public JarClassLoader(File file, AppJarClassLoader parent) throws MalformedURLException { super(new URL[] { file.toURI().toURL() }, null); this.mcl = parent; } @Override protected synchronized Class<?> loadClass(String name, boolean res) throws ClassNotFoundException { try { return loadClassFromJar(name, resolve); } catch (ClassNotFoundException ex) { } return mcl.loadClass(name, resolve); } public synchronized Class<?> loadClassFromJar(String name, boolean resolve) throws ClassNotFoundException { Class c = findLoadedClass(name); // Even if findLoadedClass returns a real class, we might simply // be its initiating ClassLoader. Only return it if we're actually // its defining ClassLoader (as determined by Class.getClassLoader). if (c == null || c.getClassLoader() != this) { c = findClass(name); if (resolve) { resolveClass(c); } } return c; } }  

 

package com.loader; import java.io.File; import java.io.IOException; import java.net.MalformedURLException; import java.util.ArrayList; import java.util.Iterator; import java.util.List; import java.util.StringTokenizer; public class AppJarClassLoader extends ClassLoader { private List<JarClassLoader> classLoaders; public AppJarClassLoader(ClassLoader parent) throws MalformedURLException, IOException { super(parent); initClassLoaders(); } private void initClassLoaders() throws MalformedURLException { classLoaders = new ArrayList<JarClassLoader>(); String classpath = System.getProperty("java.class.path"); StringTokenizer tok = new StringTokenizer(classpath, File.pathSeparator); while (tok.hasMoreTokens()) { String nextToken = tok.nextToken(); File file = new File(nextToken); classLoaders.add(new JarClassLoader(file, this)); } } public Class loadClass(String name, boolean resolve) throws ClassNotFoundException { for (Iterator<JarClassLoader> i = classLoaders.iterator(); i.hasNext();) { JarClassLoader cl = (JarClassLoader) i.next(); try { return cl.loadClassFromJar(name, resolve); } catch (ClassNotFoundException e) { } } return super.loadClass(name, resolve); } }  

postmortem

ClassLoader

 

• Startup (Main class is loaded by bootstrap class loader, In Sun JVM, Default loaded by AppClassLoader)

The Java virtual machine starts up by creating an initial class, which is specified in an implementation-dependent manner, using the bootstrap class loader (§5.3.1) . The Java virtual machine then links the initial class, initializes it, and invokes its public  class method void  main(String[]) . The invocation of this method drives all further execution. Execution of the Java virtual machine instructions constituting themain  method may cause linking (and consequently creation) of additional classes and interfaces, as well as invocation of additional methods.

• Namespaces(runtime package )

At run time, a class or interface is determined not by its name alone, but by a pair: its fully qualified name and its defining class loader. Each such class or interface belongs to a single runtime package . The runtime package of a class or interface is determined by the package name and defining class loader of the class or interface.

 

• Same Loader (L load c, if c reference d, then L will load d)

Creation of a class or interface C denoted by the name N consists of the construction in the method area of the Java virtual machine (§3.5.4)  of an 

implementation-specific internal representation of C. Class or interface creation is triggered by another class or interface D, which references C through its runtime constant pool. Class or interface creation may also be triggered by D invoking methods in certain Java class libraries (§3.12)  such as reflection.

 

The Java virtual machine uses one of three procedures to create class or interface C denoted by N:

If N denotes a nonarray class or an interface, one of the two following methods is used to load and thereby create C :

If D was defined by the bootstrap class loader, then the bootstrap class loader initiates loading of C (§5.3.1).

If D was defined by a user-defined class loader, then that same user-defined class loader initiates loading of C (§5.3.2).

Otherwise N denotes an array class. An array class is created directly by the Java virtual machine (§5.3.3), not by a class loader. However, the defining class loader of D is used in the process of creating array class C.

 

ClassNotFoundException NoClassDefFoundError

Basic Class Loading problems

 

ClassNotFoundException  but no definition for the class with the specified name could be found.

NoClassDefFoundError  is another common exception thrown by the class loader during the loading phase. The JVM specification defines NoClassDefFoundError  as follows

Essentially, this means that a NoClassDefFoundError  is thrown as a result of a unsuccessful implicit class load.

Java

-verbose:class

Display information about each class loaded.

-verbose:gc

Report on each garbage collection event.

-verbose:jni

Report information about use of native methods and other Java Native Interface activity.

-Xprof

Profiles the running program, and sends profiling data to standard output. This option is provided as a utility that is useful in program development and is not intended to be used in production systems.

-Xbootclasspath: bootclasspath

Specify a semicolon-separated list of directories, JAR archives, and ZIP archives to search for boot class files. These are used in place of the boot class files included in the Java 2 SDK.  Note: Applications that use this option for the purpose of overriding a class in rt.jar should not be deployed as doing so would contravene the Java 2 Runtime Environment binary code license.

-Xbootclasspath/a: path

Specify a semicolon-separated path of directires, JAR archives, and ZIP archives to append to the default bootstrap class path.

-Xbootclasspath/p: path

Specify a semicolon-separated path of directires, JAR archives, and ZIP archives to prepend in front of the default bootstrap class path.  Note: Applications that use this option for the purpose of overriding a class in rt.jar should not be deployed as doing so would contravene the Java 2 Runtime Environment binary code license.

 

Tomcat Classloader

 

Finding a way out of ClassLoader , the author said we have at least 3 choices:

the  system   (also referred to as the  application ) classloader, the  current   classloader, and the current thread  context   classloader. The question above refers to the latter. Which classloader is the right one?

If you choose the system classloader

  As soon as you move your code into an Enterprise JavaBean, a Web application, or a Java Web Start application, things are guaranteed to break.

So let's digg into Tomcat to see how a web concainter using customized class loader.

    
      Bootstrap
          |
       System
          |
       Common
      /        /
Catalina   Shared
              /       /
        Webapp1  Webapp2 ... 
 

So each webapp has own class loader. Tomcat implements it by controlling the App entries: Filter, Listener, Servlet


Listener

Each StandardContext 
represent an App



 //org.apache.catalina.core.StandardContext.listenerStart()     
 ClassLoader loader = getLoader().getClassLoader();
        String listeners[] = findApplicationListeners();
        Object results[] = new Object[listeners.length];
        boolean ok = true;
        for (int i = 0; i < results.length; i++) {
            if (getLogger().isDebugEnabled())
                getLogger().debug(" Configuring event listener class '" +
                    listeners[i] + "'");
            try {
                Class clazz = loader.loadClass(listeners[i]);
                results[i] = clazz.newInstance();
                // Annotation processing
              ...
            } catch (Throwable t) {
                getLogger().error
                    (sm.getString("standardContext.applicationListener",
                                  listeners[i]), t);
                ok = false;
            }
        }




 Filter

Each StandardWrapper is an individual servlet definition. StandardWrapperValue is a pipeline implements the default basic behavior for the StandardWrapper

// StandardWrapperValve.invoke // -> ApplicationFilterFactory.createFilterChain(request, wrapper, servlet); // -> ApplicationFilterConfig.getFilter() // Identify the class loader we will be using String filterClass = filterDef.getFilterClass(); ClassLoader classLoader = null; if (filterClass.startsWith("org.apache.catalina.")) classLoader = this.getClass().getClassLoader(); else classLoader = context.getLoader().getClassLoader(); ClassLoader oldCtxClassLoader = Thread.currentThread().getContextClassLoader(); // Instantiate a new instance of this filter and return it Class clazz = classLoader.loadClass(filterClass);

 

Servlet

// StandardWrapperValve.invoke // -> StandardWrapper.allocate // ->StandardWrapper.loadServlet ClassLoader classLoader = loader.getClassLoader(); // Special case class loader for a container provided servlet // if (isContainerProvidedServlet(actualClass) && ! ((Context)getParent()).getPrivileged() ) { // If it is a priviledged context - using its own // class loader will work, since it's a child of the container // loader classLoader = this.getClass().getClassLoader(); } // Load the specified servlet class from the appropriate class loader Class classClass = null; try { if (SecurityUtil.isPackageProtectionEnabled()){ final ClassLoader fclassLoader = classLoader; final String factualClass = actualClass; try{ classClass = (Class)AccessController.doPrivileged( new PrivilegedExceptionAction(){ public Object run() throws Exception{ if (fclassLoader != null) { return fclassLoader.loadClass(factualClass); } else { return Class.forName(factualClass); } } }); } catch(PrivilegedActionException pax){ Exception ex = pax.getException(); if (ex instanceof ClassNotFoundException){ throw (ClassNotFoundException)ex; } else { getServletContext().log( "Error loading " + fclassLoader + " " + factualClass, ex ); } } } else { if (classLoader != null) { classClass = classLoader.loadClass(actualClass); } else { classClass = Class.forName(actualClass); } } } catch (ClassNotFoundException e) { unavailable(null); getServletContext().log( "Error loading " + classLoader + " " + actualClass, e ); throw new ServletException (sm.getString("standardWrapper.missingClass", actualClass), e); }

 

WebAppClassLoader

It's very important in web class loader, it verfiy the jar file, if it's servlet jar, discard this jar file. So all the interface javax.servlet is loaded from parent class loader while all the implementation is loaded from App class loader

protected boolean validateJarFile(File jarfile) throws IOException { if (triggers == null) return (true); JarFile jarFile = new JarFile(jarfile); for (int i = 0; i < triggers.length; i++) { Class clazz = null; try { if (parent != null) { clazz = parent.loadClass(triggers[i]); } else { clazz = Class.forName(triggers[i]); } } catch (Throwable t) { clazz = null; } if (clazz == null) continue; String name = triggers[i].replace('.', '/') + ".class"; if (log.isDebugEnabled()) log.debug(" Checking for " + name); JarEntry jarEntry = jarFile.getJarEntry(name); if (jarEntry != null) { log.info("validateJarFile(" + jarfile + ") - jar not loaded. See Servlet Spec 2.3, " + "section 9.7.2. Offending class: " + name); jarFile.close(); return (false); } } jarFile.close(); return (true); }

 

OSGI

We do not need define own class loader, Other choice is OSGI

http://www.slideshare.net/stephanj/intro-to-osgi