Again about determining size of Java object

Sometimes it is necessary to estimate the size a Java object takes in memory. The paper describes one of the approaches that uses Java Instrumentation API.

When it can be necessary
Java object size estimation can be helpful in the following cases:

• Cache implementations ¢ Caches are normally used to speed up the performance of accessing more frequently used data. They normally can not keep all the data from database (or other storage) because of Java process memory size limitations. This means that cache must somehow estimate the size of data it keeps and throw away some old records to keep size not exceeding some defined for the cache limit.

• Memory leak detection ¢ In some cases you can notice a memory leak and measure the heap size before and after leaking operations. If you suspect some objects you may need to measure their exact sizes and compare with leaking amount of memory. There are special big tools for such purposes however they are normally quite heavyweight, have performance impact and if you had a very basic size estimation method you cold solve the problem faster in some cases.

• Other memory size estimations ¢ for instance you can analytically estimate the settings for JVM maximal heap size if you know how many objects you are going to create in your application.

• Just for fun :)
  

Overview of different approaches
There are several different approaches of estimating Java object size. They mostly where known before JDK 5.0 has appeared.

• http://jroller.com/page/mipsJava?entry=sizeof_java_objects - uses System.gc() and Runtime.freeMemory(), Runtime.totalMemory() methods for measuring size of Java object. This approach normally requires much resources for precise determining of the object size. It is necessary to create many instances (preferable several thousands) of estimated object, perform heap size measurements before and after. This makes this approach almost useless for production systems like cache implementations. The advantage of this approach is that it should gives quite precise size estimations on each Java implementation and OS where object sizes may differ.
  

• Another more successful approach is described here: http://www.javaspecialists.co.za/archive/Issue078.html ¢ It is more tricky. It uses experimentally determined table of all primitive type sizes for determining full object size. Reflection API is used for iterating through object's member variable hierarchy and counting all primitive variable sizes. This approach does not require so much resources like previous one and can be used for cache implementations. The drawback is that primitive type size table is different for different JVM implementations and should be reevaluated for each one.

Here are also several other articles describing similar approaches:
http://www.javaworld.com/javaworld/javatips/jw-javatip130.html
http://www.javaworld.com/javaworld/javaqa/2003-12/02-qa-1226-sizeof.html
http://www.javapractices.com/Topic83.cjp
http://forum.java.sun.com/thread.jspa?threadID=565721&messageID=2790847

Using Instrumentation API for determining object size
Starting from 5.0 version JDK includes Instrumentation API that finally provides the getObjectSize method. However there are two issues with this method:

1. It is impossible to use this method directly, It is necessary to implement instrumentation agent that must be packaged into a JAR file

2. It returns only one object size without counting its child variable sizes which are derived from Object

The issues are easily solvable. Java agent can be implemented by declaring premain method in any class:

1: public class SizeOfAgent {
2: 
3:         static Instrumentation inst;
4:         
5:         /** initializes agent */
6:         public static void premain(String agentArgs, Instrumentation instP) {
7:                 inst = instP;           
8:         }
9: }

The premain method is called by JVM on startup and instance of Instrumentation is passed into. The SizeOfAgent just memorizes reference to Instrumentation in the static variable. To tell JVM about the instrumentation agent the class existence it must be packaged into a JAR file with special attributes in the manifest.mf file. In our case we need the following attributes:
    Premain-Class: sizeof.agent.SizeOfAgent
    Boot-Class-Path:
    Can-Redefine-Classes: false

Additionally java application must be launched with -javaagent parameter pointing the jar file. In our case it looks like this:

    java -javaagent:sizeofag.jar <Your main class>

After we got reference to Instrumentation instance it is very simple to implement a basic sizeOf method.

1: public class SizeOfAgent {
2: 
3:         static Instrumentation inst;
4:         
5:         // ...
6:                 
7:         public static long sizeOf(Object o) {
8:                 return inst.getObjectSize(o);
9:         }
10: }

The SizeOgAgent.sizeOf() can be simply called from your application then. As it was mentioned the method returns only one object size without taking into account the sizes of its member variables. Full object size estimation can be done through reflection. We simply can recursively go through all child variables of estimated object and sum their sizes. Not everybody knows that it is possible to access private and protected variables through reflection. You just have to call Field.setAccessible(true) method before obtaining value of private field. Here is the full source code of SizeOfAgentwith fullSizeOf method implementation:

001: package sizeof.agent;
002: 
003: import java.lang.instrument.Instrumentation;
004: import java.lang.reflect.Array;
005: import java.lang.reflect.Field;
006: import java.lang.reflect.Modifier;
007: import java.util.IdentityHashMap;
008: import java.util.Map;
009: import java.util.Stack;
010: 
011: /** Instrumentation agent used */
012: public class SizeOfAgent {
013: 
014:         static Instrumentation inst;
015:         
016:         /** initializes agent */
017:         public static void premain(String agentArgs, Instrumentation instP) {
018:                 inst = instP;           
019:         }
020:         
021:         /**
022:          * Returns object size without member sub-objects.
023:          * @param o object to get size of
024:          * @return object size
025:          */
026:         public static long sizeOf(Object o) {
027:                 if(inst == null) {
028:                         throw new IllegalStateException("Can not access instrumentation environment.\n" +
029:                                         "Please check if jar file containing SizeOfAgent class is \n" +
030:                                         "specified in the java's \"-javaagent\" command line argument.");
031:                 }
032:                 return inst.getObjectSize(o);
033:         }
034:         
035:         /**
036:          * Calculates full size of object iterating over
037:          * its hierarchy graph.
038:          * @param obj object to calculate size of
039:          * @return object size
040:          */
041:         public static long fullSizeOf(Object obj) {
042:                 Map<Object, Object> visited = new IdentityHashMap<Object, Object>();
043:                 Stack<Object> stack = new Stack<Object>();
044: 
045:             long result = internalSizeOf(obj, stack, visited);
046:             while (!stack.isEmpty()) {
047:               result += internalSizeOf(stack.pop(), stack, visited);
048:             }
049:             visited.clear();
050:             return result;
051:         }               
052:           
053:     private static boolean skipObject(Object obj, Map<Object, Object> visited) {
054:             if (obj instanceof String) {
055:               // skip interned string
056:               if (obj == ((String) obj).intern()) {
057:                 return true;
058:               }
059:             }
060:             return (obj == null) // skip visited object
061:                 || visited.containsKey(obj);
062:          }
063: 
064:     private static long internalSizeOf(Object obj, Stack<Object> stack, Map<Object, Object> visited) {
065:             if (skipObject(obj, visited)){
066:                 return 0;
067:             }
068:             visited.put(obj, null);
069:             
070:             long result = 0;
071:             // get size of object + primitive variables + member pointers 
072:             result += SizeOfAgent.sizeOf(obj);
073:             
074:             // process all array elements
075:             Class clazz = obj.getClass();
076:             if (clazz.isArray()) {
077:               if(clazz.getName().length() != 2) {// skip primitive type array
078:                   int length =  Array.getLength(obj);
079:                           for (int i = 0; i < length; i++) {
080:                                   stack.add(Array.get(obj, i));
081:                       } 
082:               }       
083:               return result;
084:             }
085:             
086:             // process all fields of the object
087:             while (clazz != null) {
088:               Field[] fields = clazz.getDeclaredFields();
089:               for (int i = 0; i < fields.length; i++) {
090:                 if (!Modifier.isStatic(fields[i].getModifiers())) {
091:                   if (fields[i].getType().isPrimitive()) {
092:                           continue; // skip primitive fields
093:                   } else {
094:                     fields[i].setAccessible(true);
095:                     try {
096:                       // objects to be estimated are put to stack
097:                       Object objectToAdd = fields[i].get(obj);
098:                       if (objectToAdd != null) {                        
099:                         stack.add(objectToAdd);
100:                       }
101:                     } catch (IllegalAccessException ex) { 
102:                         assert false; 
103:                     }
104:                   }
105:                 }
106:               }
107:               clazz = clazz.getSuperclass();
108:             }
109:             return result;
110:          }
111: }

Basic idea is very similar to approach published by Dr. Heinz M. Kabutz http://www.javaspecialists.co.za/archive/Issue078.html. I even reused the skipObject method from there. The algorithm is built so that each object is counted only once to avoid cyclic references. Additionally it skips interned strings (see String.intern() for more information).

Disadvantages
The main disadvantage of the approach is that it can not be used in sandbox environments like unsigned applets or Web Start applications. This limitation is because reflection is used for accessing private class members and instrumentation agents may not work in sandbox environments as well.

Files

The file sizeofag.jar contains compiled class and Java sources inside. So you can just add sizeofag.jar into your JVM's -javaagent option and use SizeOfAgent as a normal class in your program. Enjoy it :)

转帖：http://www.jroller.com/maxim/entry/again_about_determining_size_of