使用sun misc Unsafe及反射对内存进行内省 introspection

               

    对于一个有经验的JAVA程序员来说，了解一个或者其它的JAVA对象占用了多少内存，这将会非常有用。你可能已经听说过我们所生活的世界，存储容量将不再是一个问题，这个对于你的文本编辑器来说可能是对的（不过，打开一个包含大量的图片以及图表的文档，看看你的编辑器会消耗多少内存），对于一个专用服务器软件来说也可能是对的（至少在你的企业成长到足够大或者是在同一台服务器运行其它的软件之前），对于基于云的软件来说也可能是对的，如果你足够的富有可以花足够的钱可以买顶级的服务器硬件。

    然而，现实是你的软件如果是受到了内存限制，需要做的是花钱优化它而不是尝试获取更好的硬件（原文：in the real world your software will once reach a point where it makes sense to spend money in its optimization rather than trying to obtain an even better hardware）（目前你可以获取到的最好的商业服务器是64G内存），此时你不得不分析你的应用程序来找出是哪个数据结构消耗了大部份的内存。对于这种分析任务，最好的工具就是一个好的性能分析工具，但是你可以在刚开始的时候，使用分析你代码中的对象这种用廉价的方式。这篇文章描述了使用基于Oracle JDK的ClassIntrospector类，来分析你的应用程序内存消耗。

    我曾经在文章字符串包装第1部分：将字符转换为字节中提到了JAVA对象内存结构，例如我曾经写过，在JAVA1.7.0_06以前，一个具有28个字符的字符串会占用104个字节，事实上，我在写这篇文章的时候，通过自己的性能分析器证实了我的计算结果。现在我们使用Oracle JDK中特殊类sun.misc.Unsafe，通过纯JAVA来实现一个JAVA对象内省器（introspector）。

    我们使用sun.misc.Unsafe的以下方法：

//获取字节对象中非静态方法的偏移量(get offset of a non-static field in the object in bytespublic native long objectFieldOffset(java.lang.reflect.Field field);//获取数组中第一个元素的偏移量(get offset of a first element in the array)public native int arrayBaseOffset(java.lang.Class aClass);//获取数组中一个元素的大小(get size of an element in the array)public native int arrayIndexScale(java.lang.Class aClass);//获取JVM中的地址值(get address size for your JVM)public native int addressSize();

    在sun.misc.Unsafe中有两个额外的内省方法：staticFieldBase及staticFieldOffset，但是在这篇文章中不会使用到。这两个方法对于非安全的读写静态方法会有用。

    我们应如何找到一个对象的内存布局？

    1、循环的在分析类及父类上调用Class.getDeclaredFields，获取所有对象的字段，包括其父类中的字段；

    2、针对非静态字段（通过Field.getModifiers() & Modifiers.STATIC判断静态字段），通过使用Unsafe.objectFieldOffset在其父类中获取一个字段的偏移量以及该字段的shallow（注：shallow指的是当前对象本身的大小）大小：基础类型的默认值及4个或8个字节的对象引用（更多看下面）；

    3、对数组来说，调用Unsafe.arrayBaseOffset及Unsafe.arrayIndexScale，数组的整个shallow大小将会是 当前数组的偏移量+每个数组的大小*数组的长度（原文是：offset + scale * Array.getLength(array)），当然了也包括对数组本身引用的大小（看前面提到的）；

    4、别忘了对象图的循环引用，因而就需要对前面已经分析过的对象进行跟踪记录（针对这些情况，推荐使用IdentityHashMap）

    Java对象引用大小是一个非常不确定的值（原文：Java Object reference size is quite a virtual value），它可能是4个字节或者是8个字节，这个取决于你的JVM设置以及给了多少内存给JVM，针对32G以上的堆，它就总是8个字节，但是针对小一点的堆就是4个字节除非你在JVM设置里关掉设置-XX:-UseCompressedOops（我不确定这个功能是在JVM的哪个版本加进来的，或者是默认是打开的）。结果就是，安全的方式获取对像引用的大小就是找到Object[]数组中一个元素的大小：unsafe.arrayIndexScale( Object[].class )，针对这种情况，Unsafe.addressSize倒不实用了。   

    针对32G以下堆内存中例用4字节引用的一点小小注意。一个正常的4个字节的指针可以定位到4G地址空间任何地址。如果我们假设所有已分配的对象将通过8字节边界对齐，在我们的32位指针中我们将不再需要最低3位（这些位将总是等于零）。这意味着我们可以存储35位地址在32位中。（这一节附上原文如下：

    A small implementation note on 4 byte references on under 32G heaps. A normal 4 byte pointer could addressany byte in 4G address space. If we will assume that all allocated objects will be aligned by 8 bytes boundary, we won’t need 3 lowest bits in our 32 bit pointers anymore (these bits will always be equal to zeroes). This means that we can store 35 bit addresses in 32 bit value:）   

32_bit_reference = ( int ) ( actual_64_bit_pointer >> 3 )

    35位允许寻址 32位*8=4G*8=32G地址空间。

     写这个工具时发现的其它的一些有趣的事情

    1、要打印数组的内容，必须使用Arrays.toString（包括基本类型及对象数组）；

    2、你必须要小心 - 内省方法（introspection method）只接受对象作为字段值，因此你最终可能处在无限循环中：整型打包成整数，以便传递到内省的方法。里面你会发现一个Integer.value字段，并尝试再次内省了 - 瞧，你又回到了开始的地方！

    3、要内省(introspect)对象数组中所有非空的值 - 这仅仅是间接的对象图中的外部level（原文：this is just an extra level of indirection in the object graph）

    如何使用ClassIntrospector类？仅需要实例化它并且在你的任意的对象中调用它的实例内省（introspect）方法，它会返回一个ObjectInfo对象，这个对象与你的“根‘对象有关，这个对象将指向它的所有子项，我想这可能是足够的打印其toString方法的结果和/或调用ObjectInfo.getDeepSize方法（原文：I think it may be sufficient to print itstoString method result and/or to call ObjectInfo.getDeepSize method），它将通过你的”根“对象引用，返回你的所有对象的总内存消耗。

    ClassIntrospector不是线程安全的，但是你可以在同一个线程中任意多次调用内省（introspect）方法。

    总结：

    1、你可以使用sun.misc.Unsafe的这些方法获取Java对象的布局信息：objectFieldOffset,arrayBaseOffset and arrayIndexScale；

    2、Java对象引用的大小取决于你当前的环境，根据不同JVM的设置以及分配给JVM的内存大小，它可能是4个或者8个字节。在大于32G的堆中，对像引用的大小总会是8个字节，但是在一个比较小的堆中它就会是4个字节，除非关闭JVM设置：-XX:-UseCompressedOops。

    源码

    ClassIntrospector：

import sun.misc.Unsafe; import java.lang.reflect.Array;import java.lang.reflect.Field;import java.lang.reflect.Modifier;import java.math.BigDecimal;import java.util.*; /** * This class could be used for any object contents/memory layout printing. */public class ClassIntrospector{    public static void main(String[] args) throws IllegalAccessException {        final ClassIntrospector ci = new ClassIntrospector();        final Map<String, BigDecimal> map = new HashMap<String, BigDecimal>( 10);        map.put( "one", BigDecimal.ONE );        map.put( "zero", BigDecimal.ZERO );        map.put( "ten", BigDecimal.TEN );        final ObjectInfo res;        res = ci.introspect( "0123456789012345678901234567" );        //res = ci.introspect( new TestObjChild() );        //res = ci.introspect(map);        //res = ci.introspect( new String[] { "str1", "str2" } );        //res = ci.introspect(ObjectInfo.class);        //res = ci.introspect( new TestObj() );         System.out.println( res.getDeepSize() );        System.out.println( res );    }     /** First test object - testing various arrays and complex objects */    private static class TestObj    {        protected final String[] strings = { "str1", "str2" };        protected final int[] ints = { 14, 16 };        private final Integer i = 28;        protected final BigDecimal bigDecimal = BigDecimal.ONE;         @Override        public String toString() {            return "TestObj{" +                    "strings=" + (strings == null ? null : Arrays.asList(strings)) +                    ", ints=" + Arrays.toString( ints ) +                    ", i=" + i +                    ", bigDecimal=" + bigDecimal +                    '}';        }    }     /** Test class 2 - testing inheritance */    private static class TestObjChild extends TestObj    {        private final boolean[] flags = { true, true, false };        private final boolean flag = false;         @Override        public String toString() {            return "TestObjChild{" +                    "flags=" + Arrays.toString( flags ) +                    ", flag=" + flag +                    '}';        }    }     private static final Unsafe unsafe;    /** Size of any Object reference */    private static final int objectRefSize;    static    {        try        {            Field field = Unsafe.class.getDeclaredField("theUnsafe");            field.setAccessible(true);            unsafe = (Unsafe)field.get(null);             objectRefSize = unsafe.arrayIndexScale( Object[].class );        }        catch (Exception e)        {            throw new RuntimeException(e);        }    }     /** Sizes of all primitive values */    private static final Map<Class, Integer> primitiveSizes;     static    {        primitiveSizes = new HashMap<Class, Integer>( 10 );        primitiveSizes.put( byte.class, 1 );        primitiveSizes.put( char.class, 2 );        primitiveSizes.put( int.class, 4 );        primitiveSizes.put( long.class, 8 );        primitiveSizes.put( float.class, 4 );        primitiveSizes.put( double.class, 8 );        primitiveSizes.put( boolean.class, 1 );    }     /**     * Get object information for any Java object. Do not pass primitives to this method because they     * will boxed and the information you will get will be related to a boxed version of your value.     * @param obj Object to introspect     * @return Object info     * @throws IllegalAccessException     */    public ObjectInfo introspect( final Object obj ) throws IllegalAccessException    {        try        {            return introspect( obj, null );        }        finally { //clean visited cache before returning in order to make this object reusable            m_visited.clear();        }    }     //we need to keep track of already visited objects in order to support cycles in the object graphs    private IdentityHashMap<Object, Boolean> m_visited = new IdentityHashMap<Object, Boolean>( 100 );     private ObjectInfo introspect( final Object obj, final Field fld ) throws IllegalAccessException    {        //use Field type only if the field contains null. In this case we will at least know what's expected to be        //stored in this field. Otherwise, if a field has interface type, we won't see what's really stored in it.        //Besides, we should be careful about primitives, because they are passed as boxed values in this method        //(first arg is object) - for them we should still rely on the field type.        boolean isPrimitive = fld != null && fld.getType().isPrimitive();        boolean isRecursive = false; //will be set to true if we have already seen this object        if ( !isPrimitive )        {            if ( m_visited.containsKey( obj ) )                isRecursive = true;            m_visited.put( obj, true );        }         final Class type = ( fld == null || ( obj != null && !isPrimitive) ) ?                obj.getClass() : fld.getType();        int arraySize = 0;        int baseOffset = 0;        int indexScale = 0;        if ( type.isArray() && obj != null )        {            baseOffset = unsafe.arrayBaseOffset( type );            indexScale = unsafe.arrayIndexScale( type );            arraySize = baseOffset + indexScale * Array.getLength( obj );        }         final ObjectInfo root;        if ( fld == null )        {            root = new ObjectInfo( "", type.getCanonicalName(), getContents( obj, type ), 0, getShallowSize( type ),                    arraySize, baseOffset, indexScale );        }        else        {            final int offset = ( int ) unsafe.objectFieldOffset( fld );            root = new ObjectInfo( fld.getName(), type.getCanonicalName(), getContents( obj, type ), offset,                    getShallowSize( type ), arraySize, baseOffset, indexScale );        }         if ( !isRecursive && obj != null )        {            if ( isObjectArray( type ) )            {                //introspect object arrays                final Object[] ar = ( Object[] ) obj;                for ( final Object item : ar )                    if ( item != null )                        root.addChild( introspect( item, null ) );            }            else            {                for ( final Field field : getAllFields( type ) )                {                    if ( ( field.getModifiers() & Modifier.STATIC ) != 0 )                    {                        continue;                    }                    field.setAccessible( true );                    root.addChild( introspect( field.get( obj ), field ) );                }            }        }         root.sort(); //sort by offset        return root;    }     //get all fields for this class, including all superclasses fields    private static List<Field> getAllFields( final Class type )    {        if ( type.isPrimitive() )            return Collections.emptyList();        Class cur = type;        final List<Field> res = new ArrayList<Field>( 10 );        while ( true )        {            Collections.addAll( res, cur.getDeclaredFields() );            if ( cur == Object.class )                break;            cur = cur.getSuperclass();        }        return res;    }     //check if it is an array of objects. I suspect there must be a more API-friendly way to make this check.    private static boolean isObjectArray( final Class type )    {        if ( !type.isArray() )            return false;        if ( type == byte[].class || type == boolean[].class || type == char[].class || type == short[].class ||            type == int[].class || type == long[].class || type == float[].class || type == double[].class )            return false;        return true;    }     //advanced toString logic    private static String getContents( final Object val, final Class type )    {        if ( val == null )            return "null";        if ( type.isArray() )        {            if ( type == byte[].class )                return Arrays.toString( ( byte[] ) val );            else if ( type == boolean[].class )                return Arrays.toString( ( boolean[] ) val );            else if ( type == char[].class )                return Arrays.toString( ( char[] ) val );            else if ( type == short[].class )                return Arrays.toString( ( short[] ) val );            else if ( type == int[].class )                return Arrays.toString( ( int[] ) val );            else if ( type == long[].class )                return Arrays.toString( ( long[] ) val );            else if ( type == float[].class )                return Arrays.toString( ( float[] ) val );            else if ( type == double[].class )                return Arrays.toString( ( double[] ) val );            else                return Arrays.toString( ( Object[] ) val );        }        return val.toString();    }     //obtain a shallow size of a field of given class (primitive or object reference size)    private static int getShallowSize( final Class type )    {        if ( type.isPrimitive() )        {            final Integer res = primitiveSizes.get( type );            return res != null ? res : 0;        }        else            return objectRefSize;    }}

    ObjectInfo：

import java.util.ArrayList;import java.util.Collections;import java.util.Comparator;import java.util.List; /** * This class contains object info generated by ClassIntrospector tool */public class ObjectInfo {    /** Field name */    public final String name;    /** Field type name */    public final String type;    /** Field data formatted as string */    public final String contents;    /** Field offset from the start of parent object */    public final int offset;    /** Memory occupied by this field */    public final int length;    /** Offset of the first cell in the array */    public final int arrayBase;    /** Size of a cell in the array */    public final int arrayElementSize;    /** Memory occupied by underlying array (shallow), if this is array type */    public final int arraySize;    /** This object fields */    public final List<ObjectInfo> children;     public ObjectInfo(String name, String type, String contents, int offset, int length, int arraySize,    int arrayBase, int arrayElementSize)    {        this.name = name;        this.type = type;        this.contents = contents;        this.offset = offset;        this.length = length;        this.arraySize = arraySize;        this.arrayBase = arrayBase;        this.arrayElementSize = arrayElementSize;        children = new ArrayList<ObjectInfo>( 1 );    }     public void addChild( final ObjectInfo info )    {        if ( info != null )            children.add( info );    }     /**    * Get the full amount of memory occupied by a given object. This value may be slightly less than    * an actual value because we don't worry about memory alignment - possible padding after the last object field.    *    * The result is equal to the last field offset + last field length + all array sizes + all child objects deep sizes    * @return Deep object size    */    public long getDeepSize()    {        return length + arraySize + getUnderlyingSize( arraySize != 0 );    }     private long getUnderlyingSize( final boolean isArray )    {        long size = 0;        for ( final ObjectInfo child : children )            size += child.arraySize + child.getUnderlyingSize( child.arraySize != 0 );        if ( !isArray && !children.isEmpty() )            size += children.get( children.size() - 1 ).offset + children.get( children.size() - 1 ).length;        return size;    }     private static final class OffsetComparator implements Comparator<ObjectInfo>    {        @Override        public int compare( final ObjectInfo o1, final ObjectInfo o2 )        {            return o1.offset - o2.offset; //safe because offsets are small non-negative numbers        }    }     //sort all children by their offset    public void sort()    {        Collections.sort( children, new OffsetComparator() );    }     @Override    public String toString() {        final StringBuilder sb = new StringBuilder();        toStringHelper( sb, 0 );        return sb.toString();    }     private void toStringHelper( final StringBuilder sb, final int depth )    {        depth( sb, depth ).append("name=").append( name ).append(", type=").append( type )            .append( ", contents=").append( contents ).append(", offset=").append( offset )            .append(", length=").append( length );        if ( arraySize > 0 )        {            sb.append(", arrayBase=").append( arrayBase );            sb.append(", arrayElemSize=").append( arrayElementSize );            sb.append( ", arraySize=").append( arraySize );        }        for ( final ObjectInfo child : children )        {            sb.append( '\n' );            child.toStringHelper(sb, depth + 1);        }    }     private StringBuilder depth( final StringBuilder sb, final int depth )    {        for ( int i = 0; i < depth; ++i )            sb.append( '\t' );        return sb;    }}

    原文地址：http://java-performance.info/memory-introspection-using-sun-misc-unsafe-and-reflection/

           

再分享一下我老师大神的人工智能教程吧。零基础！通俗易懂！风趣幽默！还带黄段子！希望你也加入到我们人工智能的队伍中来！https://blog.csdn.net/jiangjunshow