类加载器泄露学习（二）找出并解决不想要的引用 —— Classloader leaks II – Find and work around unwanted references...

This time we will discuss different reasons for leaks, look at an example of a leak in a third party library, and see how we can fix that leak by a workaround.

理解：探讨不同的泄露原因并寻找解决方案

Different reasons for ClassLoader leaks

In order to know what you should be looking for in your heapdump analysis, we could categorize ClassLoader leaks into three different types. In the end, they are all just variants of the first one.

1. References from outside your webapp – that is from the application server or the JDK classes – to either the ClassLoader itself or one of the classes it has loaded (which in turn has a reference to the ClassLoader).
2. Threads running inside your webapp. If you spawn new threads from within your web application that may not terminate, they are likely to prevent your ClassLoader from being garbage collected. This can happen even if the thread does not use any of the classes loaded by your webapps ClassLoader. This is because threads have a context classloader, to which there is a reference (contextClassLoader) in thejava.lang.Thread class. More about this in the next post.
3. ThreadLocals with values whose class is loaded in your webapp. If you use ThreadLocals in your webapp, you need to explicitly clear all ThreadLocals whenever the webapp closes down. This is because a) the application server uses a thread pool, which means that the thread will outlive your webapp instance and b)ThreadLocal values are actually stored in the java.lang.Thread object. Therefore, this is just a variation of 
4. (Note: This may be the case most likely created by yourself, but also exists in third party libraries)

理解：通过对堆存储文件的分析，我们可以将类加载器泄露分为三种不同类型（其实他们都是第一个原因的变形）：（1）来自应用服务器或者JDK类的引用指向了我们web应用中类加载器，或者是指向了被类加载器锁加载的类（这个类有一个指向类加载器的引用）（2）（3）已加载类中的ThreadLocal变量。如果你在web应用中使用了ThreadLocal变量，在应用结束时，你必须清除掉它们，因为：1.应用服务用了线程池，它的生命周期比web应用长 2.ThreadLocal变量存储在线程对象中，而不是webapp对象中，也就是原因1的变形

Example of reference from outside your application

When trying to hunt down a ClassLoader leak in our web application, I created a little JSP page in which I looped through all the third party JARs of our application. I tried to load every single class that was found in a custom ClassLoader, added a ZombieMarker to the ClassLoader (see previous post) and then disposed the ClassLoader. I ran the JSP page over and over again until I got a java.lang.OutOfMemoryError: PermGen space. That is, I was able to trigger ClassLoader leaks just by loading classes from our third party libraries… It actually turned out to be more than one of them, that triggered this behaviour.

理解：为了重现我们应用中存在的类加载器泄露问题，我建立了一个JSP页面来循环加载所有第三方的jar包。我试图加载每一个被用户类加载器所加载的类，同时为每一个类加载器添加一个ZombieMarker。我运行这个JSP页面一次又一次直到出现永久带内存溢出。

Here a MAT trace for one of them:

(In this picture, it’s not obvious where our ClassLoader is. The custom ClassLoader was an anonymous inner class in my JSP, so it’s the second entry with the strange class name ending with $1.)

At first glance, it may seem like this is type 2 above, with a running thread. This is not the case however, since the thread itself is not the GC root (not at the bottom level). In fact, there is a Thread involved, but it is not running.

Rather we can see that what keeps our ClassLoader from being garbage collected is a reference from outside the webapp (java.lang.*) to an instance of com.sun.media.jai.codec.TempFileCleanupThread, which in turn is loaded by our ClassLoader. From the names of the referenced and referencing (java.lang.ApplicationShutdownHook) classes, I suspected that a JVM shutdown hook was added by some Java Advanced Imaging (JAI) class when it was loaded.

理解：我们的类加载器不能被GC的真正原因：一个来自java.lang.ApplicationShutdownHooks中的引用指向了一个TempFileCleanupThread实例，而这个实例所属的类是被我们的类加载器所加载的。猜测可能是JAI中的某个类在加载时添加了一个shutdown钩子。

The com.sun.media.jai.codec.TempFileCleanupThread class is in the Codec part of JAI; version 1.1.2_01 in our case. The sources can be found in the official SVN repo (1.1.2_01 tag). As you can see, TempFileCleanupThread.java class is not in that list. That is because someone thought is was a great idea to put it as a package protected class inFileCacheSeekableStream.java.

There we can also find the source of the leak.

理解：原因在TempFileCleanupThread这个类，它在FileCacheSeekableStream类中，以获取保护，泄露的来源如下：

// Create the cleanup thread. Use reflection to preserve compile-time
// compatibility with JDK 1.2.
static {
    try {
        Method shutdownMethod =
            Runtime.class.getDeclaredMethod("addShutdownHook",
                                            new Class[] {Thread.class});

        cleanupThread = new TempFileCleanupThread();    //这个cleanupThread对象是在类加载时初始化的，直到JVM死掉时才会释放，web应用死掉后仍然存在

        shutdownMethod.invoke(Runtime.getRuntime(),
                              new Object[] {cleanupThread});
    } catch(Exception e) {
        // Reset the Thread to null if Method.invoke failed.
        cleanupThread = null;
    }
}

As suspected, there is a static block that (via reflection) adds a JVM shutdown hook, as soon as the com.sun.media.jai.codec.FileCacheSeekableStream class is loaded. Not very practical in a web application environment, since the JVM will will not shutdown until the application server is shut down.

The JAI TempFileCleanupThread is supposed to delete temporary files when the JVM shuts down. In a web application, what we want is probably to remove those temporary files as soon as the web application is redeployed. If this was our own code, we should have changed this. In this case it’s a third party library, and judging from the SVN trunk, this still has not been fixed, so upgrading doesn’t help. (This has been reported here.)

理解：我们在FileCacheSeekableStream类加载的时候用静态块添加了一个虚拟机（JVM）死掉后的钩子函数（shutdown hook）。这种方式在web应用中不是太实用，因为虚拟机只有在你的应用程序死掉后才会结束。JAI的TempFileCleanupThread类是用来在虚拟机死掉后删除临时文件，但是在web应用中，我们是想在应用程序重新部署时删除掉所有的临时文件。第三方库还没解决这个问题。

 

Cleaning up leaking references at redeploy

In order to clean up references as part of web application shutdown, to prevent ClassLoader leaks, there are two approaches. You can either put the code in the destroy()method of a Servlet that is load-on-startup

<servlet servlet-name='cleanup' servlet-class='my.CleanupServlet'>
  <load-on-startup>1</load-on-startup>
</servlet>

or (probably slightly more correct) you can create a javax.servlet.ServletContextListener and add the cleanup to the contextDestroyed() method.

<listener>
  <listener-class>my.CleanupListener</listener-class>
</listener>

 理解：两种在应用重启时清理引用泄露的方法：（1）servlet中在“启动时加载”标签中哦个添加cleanup方法 （2）更正确的方法：创建一个ServeletContextListener并添加contextDestroyed函数

 

The workaround

Fortunately, FileCacheSeekableStream keeps a reference to the shutdown hook in our case.

public final class FileCacheSeekableStream extends SeekableStream {

    /** A thread to clean up all temporary files on VM exit (VM 1.3+) */
    private static TempFileCleanupThread cleanupThread = null;

So let’s grab that reference and remove the shutdown hook. But we probably don’t just want to throw away the hook, since in theory that may leave us with temporary files that should have been deleted at JVM shutdown. Instead get the hook, remove it, and then run it immediately.

We may actually turn this into a generic method, to be reused for other third party shutdown hooks we want to remove. (System.out is used for logging, since logging frameworks usually needs to be cleaned up too, and I suggest you do that before calling this method.)

private static void removeShutdownHook(Class clazz, String field) {
  // Note that loading the class may add the hook if not yet present... 
  try {
    // Get the hook
    final Field cleanupThreadField = clazz.getDeclaredField(field);
    cleanupThreadField.setAccessible(true);
    Thread cleanupThread = (Thread) cleanupThreadField.get(null);

    if(cleanupThread != null) {
      // Remove hook to avoid PermGen leak
      System.out.println("  Removing " + cleanupThreadField + " shutdown hook");
      Runtime.getRuntime().removeShutdownHook(cleanupThread);  //清除掉shutdown钩子函数
      
      // Run cleanup immediately
      System.out.println("  Running " + cleanupThreadField + " shutdown hook");
      cleanupThread.start();    //运行清理线程
      cleanupThread.join(60 * 1000); // Wait up to 1 minute for thread to run
      if(cleanupThread.isAlive())
        System.out.println("STILL RUNNING!!!");
      else
        System.out.println("Done");
    }
    else
      System.out.println("  No " + cleanupThreadField + " shutdown hook");
    
  }
  catch (NoSuchFieldException ex) {
    System.err.println("*** " + clazz.getName() + '.' + field + 
      " not found; has JAR been updated??? ***");
    ex.printStackTrace();
  }
  catch(Exception ex) {
    System.err.println("Unable to unregister " + clazz.getName() + '.' + field);
    ex.printStackTrace();
  }    
}

 Now we just call that method in our application shutdown (CleanupServlet.destroy() /CleanupListener.contextDestroyed()) like so:

理解：FileCacheSeekableStream类中仍然持有一个cleanupThread的引用，我们可以利用这个引用同时清除掉shutdown钩子。由于在虚拟机结束时有一些临时文件需要清理，我们不能直接丢弃这个钩子，相反，我们可以首先获取这个钩子，清除掉钩子函数，并立即运行cleanupThread线程。我们可以写一个通用的函数，用这个函数来清除第三方的shutdown钩子（例如系统输出）

In a worst case scenario, if there is no reference kept to the shutdown hook, we may use reflection into the JVM classes. It would look like this:

 That’s all for this post. Next time we’ll look at threads running within your ClassLoader.

 

final Field field = 
  Class.forName("java.lang.ApplicationShutdownHooks").getDeclaredField("hooks");
field.setAccessible(true);
Map<Thread, Thread> shutdownHooks = (Map<Thread, Thread>) field.get(null);
// Iterate copy to avoid ConcurrentModificationException
for(Thread t : new ArrayList<Thread>(shutdownHooks.keySet())) {
  if(t.getClass().getName().equals("class.name.of.ShutdownHook")) { // TODO: Set name
    // Make sure it's from this web app instance
    if(t.getClass().getClassLoader().equals(this.getClass().getClassLoader())) {
      Runtime.getRuntime().removeShutdownHook(t); // 清除
      t.start(); // 运行
      t.join(60 * 1000); // Wait up to 1 minute for thread to run
    }
  }
}

 

 

转载于:https://www.cnblogs.com/Guoyutian/p/5060316.html