背景
- 如果需要对web server中的app做字节码注入或修改,就必须知道如何定位该app中的class
- class都是由ClassLoader来加载的,web server中的每个app都有自己独立的ClassLoader,如何查找这些ClassLoader,将是完成一切的基础
本文内容
- 介绍 web server 中 classLoader 的层次体系
- 如何查找web server中各个app的ClassLoader(以Jetty和Tomcat为例)
- 对于如何在实践中运用本篇内容,可以参考:
使用Instrumentation和Javassist修改web应用字节码
Web Server ClassLoader 层次体系
- 如果是在终端启动web server,那么加载它的ClassLoader一般就是 sun.misc.Launcher$AppClassLoader
- web server 一般会有一个common lib目录,存放自己所有用到的jar包,在启动和初始化阶段,会new一个URLClassLoader来加载这些jar包。而各个app共享的jar包,以及用户自定义要让web server使用的jar包(通过类似–lib等参数或者是配置文件指定),也一般会使用这个ClassLoader来加载,Launcher$AppClassLoader会是它的parent或者祖先。
- web server中有一到多个war包,每个war包对应一个app,为了支持多个app的同时运行而又不相互影响,web server在加载每个war包的时候,会单独使用一个ClassLoader,一般称之为WebAppClassLoader,而2中提到的URLClassLoader会是它的parent或者是祖先。
- 有些web server还允许用户在WebAppClassLoader之上定义自己专有的ClassLoader,以实现更强大的功能
- WebAppClassLoader加载class的时候,不一定都是优先走委派parent加载路线,而更可能的却是先从本地war包中加载,失败后才委派parent来加载。当然,这种顺序一般可以通过配置进行修改。
- 这种层次结构,需要对web server和app用到的公共jar包进行很好的管理,否则就会在运行时出现LinkageError,具体的可以参考:探讨ClassLoader引发的 java.lang.LinkageError
下图省略了 Bootstrap和Ext ClassLoader
查找 JettyRunner 中各个 App 的 ClassLoader
方法1
随便写个class打进war包,放到jetty中运行:
package test.jetty;
public class TestObject {
public void test() {
System.out.println("class loader: " + getClass().getClassLoader());
}
}
在终端启动JettyRunner,访问一下,能看到以下输出:
2019-07-31 00:11:11.580:INFO::main: Logging initialized @511ms to org.eclipse.jetty.util.log.StdErrLog
2019-07-31 00:11:11.652:INFO:oejr.Runner:main: Runner
2019-07-31 00:11:12.248:INFO:oejs.Server:main: jetty-9.4.12.v20180830; built: 2018-08-30T13:59:14.071Z; git: 27208684755d94a92186989f695db2d7b21ebc51; jvm 1.8.0_172-b11
...
2019-07-31 00:11:13.548:INFO:oejs.Server:main: Started @2482ms
class loader: WebAppClassLoader=Test Server@6615435c
由此可知JettyRunner加载war包用的ClassLoader叫:WebAppClassLoader
写个小程序启动 JettyRunner 进行debug
PS: 如果想知道启动类是哪个,可以解压 jetty-runner.jar包,查看里面的META-INF/MANIFEST.MF文件
package com.test;
import org.eclipse.jetty.runner.Runner;
public class TestJetty {
public static void main(String[] ss) {
String dir = "/home/helowken/test_jetty";
String[] args = new String[]{
"--stop-port", "9100",
"--stop-key", "stop_test_jetty",
"--config", dir + "/config/jetty-ssl.xml",
"--config", dir + "/config/jetty.xml",
dir + "/config/test.war.xml",
dir + "/config/test2.war.xml"
};
Runner.main(args);
}
}
在JettyRunner的源码中找到WebAppClassLoader,给它的构造函数打上断点,等debug到它的时候,从调用栈中查看它和调用者之间的引用关系。
由于这种debug需要一些技巧,篇幅太大,这里就省略掉,直接出结果:
WebAppClassLoader
->WebAppContext (field: "_classLoader")
->ContextHandlerCollection (field: "_beans")
->HandlerCollection (field: "_handlers")
->Server (field: "_beans")
->Runner(field: "_server")
根据上面这个引用关系,修改一下小程序,加入查找ClassLoader的逻辑
package test.jetty;
import agent.base.utils.ReflectionUtils;
import org.eclipse.jetty.runner.Runner;
import org.eclipse.jetty.server.handler.ContextHandlerCollection;
import org.eclipse.jetty.server.handler.HandlerCollection;
import org.eclipse.jetty.webapp.WebAppContext;
import org.junit.Test;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import static org.junit.Assert.*;
public class TestJetty {
@Test
public void test() throws Exception {
Runner runner = new Runner();
new Thread(() -> {
try {
Thread.sleep(5000);
Object server = ReflectionUtils.getFieldValue("_server", runner);
assertNotNull(server);
List objects = ReflectionUtils.getFieldValue("_beans", server);
assertNotNull(objects);
Object handlerCollection = findBean(objects, HandlerCollection.class);
Object[] objArray = ReflectionUtils.getFieldValue("_handlers", handlerCollection);
assertNotNull(objArray);
Object contextHandlerCollection = find(Arrays.asList(objArray), ContextHandlerCollection.class);
objects = ReflectionUtils.getFieldValue("_beans", contextHandlerCollection);
assertNotNull(objects);
for (Object object : objects) {
Object context = ReflectionUtils.getFieldValue("_bean", object);
assertEquals(context.getClass(), WebAppContext.class);
Object classLoader = ReflectionUtils.getFieldValue("_classLoader", context);
assertNotNull(classLoader);
String contextPath = ReflectionUtils.invoke("getContextPath", context);
System.out.println(contextPath + ": " + classLoader);
}
} catch (Exception e) {
e.printStackTrace();
}
}).start();
String dir = "/home/helowken/test_jetty";
String[] args = new String[]{
"--stop-port", "9100",
"--stop-key", "stop_test_jetty",
"--config", dir + "/config/jetty-ssl.xml",
"--config", dir + "/config/jetty.xml",
dir + "/config/test.war.xml",
dir + "/config/test2.war.xml"
};
runner.configure(args);
runner.run();
}
private Object findBean(List objects, Class<?> clazz) throws Exception {
List beans = new ArrayList();
for (Object obj : objects) {
beans.add(ReflectionUtils.getFieldValue("_bean", obj));
}
return find(beans, clazz);
}
private Object find(List objects, Class<?> clazz) {
for (Object obj : objects) {
if (obj.getClass().equals(clazz))
return obj;
}
fail();
return null;
}
}
输出结果:
...
2019-07-31 23:34:38.966:INFO:oejs.Server:main: Started @3442ms
/test: WebAppClassLoader=Test Server@5d76b067
/test2: WebAppClassLoader=Test Server@add0edd
可以看到JettyRunner底下有2个app,/test 和 /test2,每个app对应一个不同的WebAppClassLoader实例。
如果打印一下 WebAppClassLoader 的层次结构,会看到:
org.eclipse.jetty.webapp.WebAppClassLoader
sun.misc.Launcher$AppClassLoader
sun.misc.Launcher$ExtClassLoader
没有看到 URLClassLoader,那是因为没有添加 --lib 参数指定额外需要加载的 jar包目录,因此也就没必要使用 URLClassLoader。
加上 –lib /home/helowken/lib 参数后,重新打印,可以看到:
org.eclipse.jetty.webapp.WebAppClassLoader
java.net.URLClassLoader (这是由于加入了 --lib 参数)
sun.misc.Launcher$AppClassLoader
sun.misc.Launcher$ExtClassLoader
方法2
在启动 JettyRunner 后,对它做个 heap dump。
如何做 heap dump 可以参考:Java heap dump及分析
用 MAT 打开 heap dump文件,
进入 “dominator_tree”,
搜索 “WebAppClassLoader”,
选择一个WebAppClassLoader实例,
右键菜单选择 “Path to GC Roots” -> “exclude weak references”
可以看到 WebAppClassLoader 的引用关系图
从上图可以知道,除了通过之前debug出来的路径,还有一条更短的路径可以直接获取到 WebAppClassLoader:
WebAppClassLoader
->WebAppContext (method: "getClassLoader")
->ContextHandlerCollection (method: "getHandlers")
->Runner(field: "_contexts")
把之前的小程序改成使用短路径
new Thread(() -> {
try {
Thread.sleep(5000);
Object contexts = ReflectionUtils.getFieldValue("_contexts", runner);
Object[] handlers = ReflectionUtils.invoke("getHandlers", contexts);
for (Object handler : handlers) {
String contextPath = ReflectionUtils.invoke("getContextPath", handler);
ClassLoader loader = ReflectionUtils.invoke("getClassLoader", handler);
System.out.println(contextPath + ": " + loader);
}
} catch (Exception e) {
e.printStackTrace();
}
}).start();
输出结果:
...
2019-08-01 00:08:23.068:INFO:oejs.Server:main: Started @2013ms
/test: WebAppClassLoader=Test Server@67205a84
/test2: WebAppClassLoader=Test Server@1198b989
与方法1的结果一致,而且过程更简单,不需要使用复杂的dubg技巧。
方法3
使用 JNI 和 JVMTI 在heap上来查找。
要了解如何使用 JNI 和 JVMTI 请参考:在heap上查找class的对象实例
System.load("/home/helowken/test_jni/jni_jvmti/libagent_jvmti_JvmtiUtils.so");
List loaders = JvmtiUtils.getInstance().findObjectsByClass(WebAppClassLoader.class, Integer.MAX_VALUE);
for (Object loader : loaders) {
System.out.println(loader);
}
输出结果:
Find object count: 2.
WebAppClassLoader=Test Server@1f554b06
WebAppClassLoader=Test Server@31206beb
这个方法最简单,但是因为调用了native方法,所以在不同的OS之间不具备可移植性。
查找 Tomcat 中 各个App 的 ClassLoader
方法与JettyRunner一样,这里直接给出测试程序
package test.tomcat;
import org.apache.catalina.startup.Bootstrap;
import org.junit.Test;
import java.util.Map;
import static agent.base.utils.ReflectionUtils.getFieldValue;
import static agent.base.utils.ReflectionUtils.setStaticFieldValue;
import static org.junit.Assert.assertNotNull;
public class TomcatTest {
@Test
public void test() throws Exception {
Bootstrap bootstrap = new Bootstrap();
new Thread(() -> {
try {
Thread.sleep(10000);
Object catalina = getFieldValue("catalinaDaemon", bootstrap);
assertNotNull(catalina);
Object server = getFieldValue("server", catalina);
assertNotNull(server);
Object[] services = getFieldValue("services", server);
assertNotNull(services);
Object engine = getEngine(services);
assertNotNull(engine);
Map childrenMap = getFieldValue("children", engine);
assertNotNull(childrenMap);
Object host = getHost(childrenMap);
assertNotNull(host);
childrenMap = getFieldValue("children", host);
assertNotNull(childrenMap);
Object webappLoader = getWebappLoader(childrenMap);
assertNotNull(webappLoader);
Object webappClassLoader = getFieldValue("classLoader", webappLoader);
assertNotNull(webappClassLoader);
System.out.println(webappClassLoader);
} catch (Exception e) {
e.printStackTrace();
}
}).start();
setStaticFieldValue(Bootstrap.class, "daemon", bootstrap);
bootstrap.init();
Bootstrap.main(new String[]{"start"});
}
private Object getWebappLoader(Map childrenMap) throws Exception {
final String contextClassName = "org.apache.catalina.core.StandardContext";
for (Object child : childrenMap.values()) {
if (child.getClass().getName().equals(contextClassName)) {
return getFieldValue(
contextClassName,
"loader",
child
);
}
}
return null;
}
private Object getHost(Map childrenMap) {
final String hostClassName = "org.apache.catalina.core.StandardHost";
for (Object child : childrenMap.values()) {
if (child.getClass().getName().equals(hostClassName)) {
return child;
}
}
return null;
}
private Object getEngine(Object[] services) throws Exception {
final String standardServiceClassName = "org.apache.catalina.core.StandardService";
for (Object service : services) {
if (service.getClass().getName().equals(standardServiceClassName)) {
return getFieldValue(
standardServiceClassName,
"container",
service
);
}
}
return null;
}
}
输出结果:
...
INFO: Server startup in 3446 ms
WebappClassLoader
context: /test
delegate: false
repositories:
/WEB-INF/classes/
----------> Parent Classloader:
java.net.URLClassLoader@4fccd51b
...
最后补充需要用到的反射代码
package agent.base.utils;
import java.lang.reflect.AccessibleObject;
import java.lang.reflect.Constructor;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
@SuppressWarnings("unchecked")
public class ReflectionUtils {
private static final String[] javaPackages = {"java.", "javax.", "sun."};
public static boolean isJavaNativePackage(String namePath) {
for (String javaPackage : javaPackages) {
if (namePath.startsWith(javaPackage))
return true;
}
return false;
}
public static <T> T newInstance(Object classOrClassName) throws Exception {
return newInstance(classOrClassName, new Class[0]);
}
public static <T> T newInstance(Object classOrClassName, Object[] argClassOrClassNames, Object... args) throws Exception {
Class[] argTypes = convertArray(argClassOrClassNames);
Constructor<T> constructor = (Constructor<T>) convert(classOrClassName).getDeclaredConstructor(argTypes);
return exec(constructor, () -> constructor.newInstance(args));
}
public static void setStaticFieldValue(Object classOrClassName, String fieldName, Object value) throws Exception {
setFieldValue(classOrClassName, fieldName, null, value);
}
public static void setFieldValue(String fieldName, Object target, Object value) throws Exception {
assertTarget(target);
setFieldValue(target.getClass(), fieldName, target, value);
}
public static void setFieldValue(Object classOrClassName, String fieldName, Object target, Object value) throws Exception {
Field field = getField(classOrClassName, fieldName);
exec(field, () -> {
field.set(target, value);
return null;
});
}
public static <T> T getStaticFieldValue(Object classOrClassName, String fieldName) throws Exception {
return getFieldValue(classOrClassName, fieldName, null);
}
public static <T> T getFieldValue(String fieldName, Object target) throws Exception {
assertTarget(target);
return getFieldValue(target.getClass(), fieldName, target);
}
public static <T> T getFieldValue(Object classOrClassName, String fieldName, Object target) throws Exception {
Field field = getField(classOrClassName, fieldName);
return exec(field, () -> (T) field.get(target));
}
public static void useDeclaredFields(Object classOrClassName, AccessibleObjectConsumer<Field> consumer) throws Exception {
Field[] fields = convert(classOrClassName).getDeclaredFields();
if (fields != null) {
for (Field field : fields) {
exec(field, () -> {
consumer.consume(field);
return null;
});
}
}
}
public static <T> T useField(Object classOrClassName, String fieldName, AccessibleObjectValueFunc<Field, T> func) throws Exception {
Field field = getField(classOrClassName, fieldName);
return exec(field, () -> func.exec(field));
}
private static Field getField(Object classOrClassName, String fieldName) throws Exception {
Class<?> clazz = convert(classOrClassName);
return findFromClassCascade(clazz,
tmpClass -> tmpClass.getDeclaredField(fieldName)
);
}
private static Class[] convertToTypes(Object... args) {
Class<?>[] argTypes = args == null ?
new Class<?>[0] :
new Class<?>[args.length];
if (argTypes.length > 0) {
for (int i = 0; i < argTypes.length; ++i) {
argTypes[i] = args[i].getClass();
}
}
return argTypes;
}
public static <T> T invokeStatic(Object classOrClassName, String methodName, Object... args) throws Exception {
return invokeStatic(classOrClassName, methodName, convertToTypes(args), args);
}
public static <T> T invokeStatic(Object classOrClassName, String methodName, Object[] argClassOrClassNames, Object... args) throws Exception {
return invoke(classOrClassName, methodName, argClassOrClassNames, null, args);
}
public static <T> T invoke(String methodName, Object target, Object... args) throws Exception {
return invoke(methodName, convertToTypes(args), target, args);
}
public static <T> T invoke(String methodName, Object target) throws Exception {
return invoke(methodName, new Class[0], target);
}
public static <T> T invoke(String methodName, Object[] argClassOrClassNames, Object target, Object... args) throws Exception {
assertTarget(target);
return invoke(target.getClass(), methodName, argClassOrClassNames, target, args);
}
public static <T> T invoke(Object classOrClassName, String methodName, Object[] argClassOrClassNames, Object target, Object... args) throws Exception {
Method method = getMethod(classOrClassName, methodName, argClassOrClassNames);
return exec(method, () -> (T) method.invoke(target, args));
}
private static <T extends AccessibleObject, V> V exec(T ao, AccessibleObjectValueSupplier<V> supplier) throws Exception {
boolean old = ao.isAccessible();
ao.setAccessible(true);
try {
return supplier.supply();
} finally {
ao.setAccessible(old);
}
}
public static <T> T invokeMethod(Object classOrClassName, String methodName, Object[] argClassOrClassNames,
AccessibleObjectValueFunc<Method, T> func) throws Exception {
Method method = getMethod(classOrClassName, methodName, argClassOrClassNames);
return exec(method, () -> func.exec(method));
}
private static Method getMethod(Object classOrClassName, String methodName, Object... argClassOrClassNames) throws Exception {
Class<?> clazz = convert(classOrClassName);
Class[] argTypes = convertArray(argClassOrClassNames);
return findFromClassCascade(clazz,
tmpClass -> tmpClass.getDeclaredMethod(methodName, argTypes)
);
}
private static <T> T findFromClassCascade(Class<?> clazz, FindFunc<T> func) throws Exception {
Class<?> tmpClass = clazz;
Exception e = null;
while (tmpClass != null) {
try {
return func.find(tmpClass);
} catch (Exception e2) {
if (e == null)
e = e2;
}
tmpClass = tmpClass.getSuperclass();
}
if (e != null)
throw e;
throw new Exception("Unknown exception");
}
private static Class[] convertArray(Object... classOrClassNames) throws Exception {
if (classOrClassNames == null)
return new Class[0];
Class[] classes = new Class[classOrClassNames.length];
for (int i = 0; i < classOrClassNames.length; ++i) {
classes[i] = convert(classOrClassNames[i]);
}
return classes;
}
private static Class<?> convert(Object classOrClassName) throws Exception {
if (classOrClassName instanceof Class)
return (Class<?>) classOrClassName;
else if (classOrClassName instanceof String)
return findClass((String) classOrClassName);
throw new IllegalArgumentException("Argument must be a class or classname.");
}
public static <T> Class<T> findClass(String className) throws Exception {
ClassLoader loader = Thread.currentThread().getContextClassLoader();
Class<?> clazz = loader == null ?
Class.forName(className) :
loader.loadClass(className);
return (Class<T>) clazz;
}
private static void assertTarget(Object target) {
if (target == null)
throw new IllegalArgumentException("Target is null!");
}
private interface FindFunc<V> {
V find(Class<?> clazz) throws Exception;
}
private interface AccessibleObjectValueSupplier<V> {
V supply() throws Exception;
}
public interface AccessibleObjectValueFunc<T extends AccessibleObject, V> {
V exec(T ao) throws Exception;
}
public interface AccessibleObjectConsumer<T extends AccessibleObject> {
void consume(T ao) throws Exception;
}
}