Android DVM 記憶體管理研究分析

Andorid为applayer 所设计的 DalvikVirtual Matchine 真的是包山包海, DVM的功能有, 对象生命周期管理, 行程管理, 内存管理, 安全和例外处理以及跟内存有关的资源回收机制. 里面每一项功能都算是一门大学问, 在这里就针对DVM的内存管理和资源回收机制做分析.分析的重点方向会分为Initial跟Allocate来分析.

Initial

当DVM作初始化时, 会去呼叫JNI_CreateJavaVM函数来初始化一些VM需要的环境.就从CreateJavaVM函数开始分析.

//Jni.cpp
/*
 * Create a new VM instance.
 *
 * The current thread becomes the main VM thread.  We return immediately,
 * which effectively means the caller is executing in a native method.
 */
jint JNI_CreateJavaVM(JavaVM** p_vm, JNIEnv** p_env, void* vm_args) {
    // do something
    // 设定 JNIEnv 和 VM的数据结构
 
    // 获取一个新的 JNIEnv 的物件
    JNIEnvExt* pEnv = (JNIEnvExt*) dvmCreateJNIEnv(NULL);
 
    //初始化 VM
     gDvm.initializing = true;
    std::string status =
            dvmStartup(argc, argv.get(), args->ignoreUnrecognized,
                      (JNIEnv*)pEnv);
    gDvm.initializing = false;
    // do something
}
 
// Init.cpp
/*
 * VM initialization.  Pass in any options provided on the command line.
 * Do not pass in the class name or the options for the class.
 *
 * Returns 0 on success.
 */
std::string dvmStartup(int argc, const char* const argv[],
        bool ignoreUnrecognized, JNIEnv* pEnv)
{
     // VM init args
 
     /* mterp setup */
     // 设定VM值译器
     dvmQuasiAtomicsStartup
     dvmAllocTrackerStartup
    
     if (!dvmGcStartup()) {
        return "dvmGcStartup failed";
      }    
     // 一些跟 DVM 相关的功能依序启动
     //dvmThreadStartup, dvmInlineNativeStartup, dvmRegisterMapStartup, ....
}
 
// Alloc.cpp
/*
 * Initialize the GC universe.
 *
 * We're currently using a memory-mapped arena to keep things off of the
 * main heap.  This needs to be replaced with something real.
 */
bool dvmGcStartup()
{
    dvmInitMutex(&gDvm.gcHeapLock);
    pthread_cond_init(&gDvm.gcHeapCond, NULL);
    return dvmHeapStartup();
}
 
// Heap.cpp
/*
 * Initialize the GC heap.
 *
 * Returns true if successful, false otherwise.
 */
bool dvmHeapStartup()
{
    GcHeap *gcHeap;
 
    if (gDvm.heapGrowthLimit == 0) {
        gDvm.heapGrowthLimit = gDvm.heapMaximumSize;
    }
 
    gcHeap = dvmHeapSourceStartup(gDvm.heapStartingSize,
                                  gDvm.heapMaximumSize,
                                  gDvm.heapGrowthLimit);
    if (gcHeap == NULL) {
        return false;
    }
    gcHeap->ddmHpifWhen = 0;
    gcHeap->ddmHpsgWhen = 0;
    gcHeap->ddmHpsgWhat = 0;
    gcHeap->ddmNhsgWhen = 0;
    gcHeap->ddmNhsgWhat = 0;
    gDvm.gcHeap = gcHeap;
 
    /* Set up the lists we'll use for cleared reference objects.
     */
    gcHeap->clearedReferences = NULL;
 
    if (!dvmCardTableStartup(gDvm.heapMaximumSize, gDvm.heapGrowthLimit))     
    {
        LOGE_HEAP("card table startup failed.");
        return false;
    }
 
    return true;
}
 
 
// HeapSource.cpp
/*
 * Initializes the heap source; must be called before any other
 * dvmHeapSource*() functions.  Returns a GcHeap structure
 * allocated from the heap source.
 */
GcHeap* dvmHeapSourceStartup(size_t startSize, size_t maximumSize,
                             size_t growthLimit)
{
    //do something
    /* Create an unlocked dlmalloc mspace to use as
     * a heap source.
     */
    // 划分一块内存来当heap使用
    msp = createMspace(base, kInitialMorecoreStart, startSize);
 
    //初始化HeapSource. (HeapSource *hs)
 
    // 初始化 Heap memory 在 mspace 的区域中
    addInitialHeap(hs, msp, growthLimit)
 
    // Initialize Bitmap memory by heap memory
    dvmHeapBitmapInit(&hs->liveBits, base, length, "dalvik-bitmap-1")   
    dvmHeapBitmapInit(&hs->liveBits, base, length, "dalvik-bitmap-2")   
 
    // 为最坏的情况配置一块足够大的stack memory 作为可以存放对象用的.
    allocMarkStack(&gcHeap->markContext.stack, hs->maximumSize)
 
}
 
static bool allocMarkStack(GcMarkStack *stack, size_t maximumSize)
{
    const char *name = "dalvik-mark-stack";
    void *addr;
 
    assert(stack != NULL);
    stack->length = maximumSize * sizeof(Object*) /
        (sizeof(Object) + HEAP_SOURCE_CHUNK_OVERHEAD);
    addr = dvmAllocRegion(stack->length, PROT_READ | PROT_WRITE, name);
    if (addr == NULL) {
        return false;
    }
    stack->base = (const Object **)addr;
    stack->limit = (const Object **)((char *)addr + stack->length);
    stack->top = NULL;
    madvise(stack->base, stack->length, MADV_DONTNEED);
    return true;
}

由上面一路追下来, 可以得到以下初始化的结论.

1. 先划分一块mSpace的内存来做为Heapmemory使用. 以目前的程序代码来看

这块内存的大小为 16 MB.

2. 初始化 HeapSource 结构作为配置用. 以下是 HeapSource 的结构.

struct HeapSource {
        /* Target ideal heap utilization ratio; range 1..HEAP_UTILIZATION_MAX
         */
        size_t targetUtilization;
   
        /* The starting heap size.
         */
        size_t startSize;
   
        /* The largest that the heap source as a whole is allowed to grow.
         */
        size_t maximumSize;
   
        /*
         * The largest size we permit the heap to grow.  This value allows
         * the user to limit the heap growth below the maximum size.  This
         * is a work around until we can dynamically set the maximum size.
         * This value can range between the starting size and the maximum
         * size but should never be set below the current footprint of the
         * heap.
         */
        size_t growthLimit;
   
        /* The desired max size of the heap source as a whole.
         */
        size_t idealSize;
   
        /* The maximum number of bytes allowed to be allocated from the
         * active heap before a GC is forced.  This is used to "shrink" the
         * heap in lieu of actual compaction.
         */
        size_t softLimit;
   
        /* Minimum number of free bytes. Used with the target utilization when
         * setting the softLimit. Never allows less bytes than this to be free
         * when the heap size is below the maximum size or growth limit.
         */
        size_t minFree;
   
        /* Maximum number of free bytes. Used with the target utilization when
         * setting the softLimit. Never allows more bytes than this to be free
         * when the heap size is below the maximum size or growth limit.
         */
        size_t maxFree;
   
        /* The heaps; heaps[0] is always the active heap,
         * which new objects should be allocated from.
         */
        Heap heaps[HEAP_SOURCE_MAX_HEAP_COUNT];
   
        /* The current number of heaps.
         */
        size_t numHeaps;
   
        /* True if zygote mode was active when the HeapSource was created.
         */
        bool sawZygote;
   
        /*
         * The base address of the virtual memory reservation.
         */
        char *heapBase;
   
        /*
         * The length in bytes of the virtual memory reservation.
         */
        size_t heapLength;
   
        /*
         * The live object bitmap.
         */
        HeapBitmap liveBits;
   
        /*
         * The mark bitmap.
         */
        HeapBitmap markBits;
   
        /*
         * State for the GC daemon.
         */
        bool hasGcThread;
        pthread_t gcThread;
        bool gcThreadShutdown;
        pthread_mutex_t gcThreadMutex;
        pthread_cond_t gcThreadCond;
        bool gcThreadTrimNeeded;
    };

3. 设定好HeapSource结构中的一些属性, 就在mSpace这块内存中开始配置所要的Heapmemory.

4. 利用这块配置到的Heapmemory初始化两块名为dalvik-bitmap-1 和dalvik-bitmap-2的heapmemory.

5. 为这块配置到的Heapmemory 在配置一块足够大的mark stack memory来存放对象.

简单的说, 每一个程序中的DVM一旦启动之后就会为这个程序配置一块Heapmemory给使用. 而控制这块Heapmemory的结构就是 HeapSource.

Allocation

AndroidDVM 在配置内存时, 这时GC会检查目前的内存状态, 若内存的空间不足时, 就会开始启动回收机制. 何时会需要去配置内存, 当然就是程序中若有需要去产生一个新的对象使用时, 就会需要配置内存, 经过一番的研究, 配置内存源头是来自于 dvmAllocObject 这个函数.

// Alloc.cpp
/*
 * Create an instance of the specified class.
 *
 * Returns NULL and throws an exception on failure.
 */
Object* dvmAllocObject(ClassObject* clazz, int flags)
{
    Object* newObj;
 
    assert(clazz != NULL);
    assert(dvmIsClassInitialized(clazz) || dvmIsClassInitializing(clazz));
 
    /* allocate on GC heap; memory is zeroed out */
    newObj = (Object*)dvmMalloc(clazz->objectSize, flags);
    if (newObj != NULL) {
        DVM_OBJECT_INIT(newObj, clazz);
        dvmTrackAllocation(clazz, clazz->objectSize);   /* notify DDMS */
    }
 
    return newObj;
}
 
// Heap.cpp
/*
 * Allocate storage on the GC heap.  We guarantee 8-byte alignment.
 *
 * The new storage is zeroed out.
 *
 * Note that, in rare cases, this could get called while a GC is in
 * progress.  If a non-VM thread tries to attach itself through JNI,
 * it will need to allocate some objects.  If this becomes annoying to
 * deal with, we can block it at the source, but holding the allocation
 * mutex should be enough.
 *
 * In rare circumstances (JNI AttachCurrentThread) we can be called
 * from a non-VM thread.
 *
 * Use ALLOC_DONT_TRACK when we either don't want to track an allocation
 * (because it's being done for the interpreter "new" operation and will
 * be part of the root set immediately) or we can't (because this allocation
 * is for a brand new thread).
 *
 * Returns NULL and throws an exception on failure.
 *
 * TODO: don't do a GC if the debugger thinks all threads are suspended
 */
void* dvmMalloc(size_t size, int flags)
{
   void *ptr;
 
    dvmLockHeap();
 
    /* Try as hard as possible to allocate some memory.
     */
    ptr = tryMalloc(size);
 
    //do something
 
    dvmUnlockHeap();
 
    //do something
 
   return ptr;
}
 
/* Try as hard as possible to allocate some memory.
 */
static void *tryMalloc(size_t size)
{
    void *ptr;
   
    ptr = dvmHeapSourceAlloc(size);
   
    //do something
 
    /*
     * The allocation failed.  If the GC is running, block until it
     * completes and retry.
     */
    if (gDvm.gcHeap->gcRunning) {
        /*
         * The GC is concurrently tracing the heap.  Release the heap
         * lock, wait for the GC to complete, and retrying allocating.
         */
        dvmWaitForConcurrentGcToComplete();
    } else {
      /*
       * Try a foreground GC since a concurrent GC is not currently running.
       */
      gcForMalloc(false);
    }
 
 
    //do something
 
    return NULL;
}

到目前会发现在配置内存( dvmHeapSourceAlloc)的时候, 还会在呼叫一个( gcForMalloc)函数, 看似跟GC有关的动作. 先来分析 dvmHeapSourceAlloc函数

// HeapSource.cpp
/*
 * Allocates <n> bytes of zeroed data.
 */
void* dvmHeapSourceAlloc(size_t n)
{
   //检查heap memory是否存在.
   HS_BOILERPLATE();
   HeapSource *hs = gHs;
   Heap* heap = hs2heap(hs);
   // 检查配置长度 n 是否超过初始化的界线.
   if (heap->bytesAllocated + n > hs->softLimit) {
      //do something
      return NULL;
   }
   // 在heap memory中配置一块 n 长度的内存
   void* ptr = mspace_calloc(heap->msp, 1, n);
   // do something
   countAllocation(heap, ptr);
   // do something
   return ptr;
}
 
static void countAllocation(Heap *heap, const void *ptr)
{
    assert(heap->bytesAllocated < mspace_footprint(heap->msp));
 
    heap->bytesAllocated += mspace_usable_size(ptr) +
            HEAP_SOURCE_CHUNK_OVERHEAD;
    heap->objectsAllocated++;
    HeapSource* hs = gDvm.gcHeap->heapSource;
    dvmHeapBitmapSetObjectBit(&hs->liveBits, ptr);
 
    assert(heap->bytesAllocated < mspace_footprint(heap->msp));
}
 
// HeapBitmapInlines.h
/*
 * Sets the bit corresponding to <obj>, and widens the range of seen
 * pointers if necessary.  Does no range checking.
 */
static void dvmHeapBitmapSetObjectBit(HeapBitmap *hb, const void *obj)
{
    _heapBitmapModifyObjectBit(hb, obj, true, false);
}
 
/*
 * Internal function; do not call directly.
 */
static unsigned long _heapBitmapModifyObjectBit(HeapBitmap *hb, const void *obj,
                                                bool setBit, bool returnOld)
{
    const uintptr_t offset = (uintptr_t)obj - hb->base;
    const size_t index = HB_OFFSET_TO_INDEX(offset);
    const unsigned long mask = HB_OFFSET_TO_MASK(offset);
 
    assert(hb->bits != NULL);
    assert((uintptr_t)obj >= hb->base);
    assert(index < hb->bitsLen / sizeof(*hb->bits));
    if (setBit) {
        if ((uintptr_t)obj > hb->max) {
            hb->max = (uintptr_t)obj;
        }
        if (returnOld) {
            unsigned long *p = hb->bits + index;
            const unsigned long word = *p;
            *p |= mask;
            return word & mask;
        } else {
            hb->bits[index] |= mask;
        }
    } else {
        hb->bits[index] &= ~mask;
    }
    return false;
}

由上面的程序代码可以知道, dvmHeapSourceAlloc 函数除了配置一块长度为n 的内存之外, 还为这块内存设定mask. 方便之后GC做回收机制有个依据. 在此我们做个简单的归纳.

1. 一个Heapmemory管理一个mspace, 此mspace是一块连续可配置的内存.

2. 每当从Heapmemory配置一块内存时, 同时会有另外一块内存(HeapBitmap)来记录在HeapMemory 配置内存的状态.在Heapmemory中所配置的连续内存, 每8 bytes相当于在 HeapBitmap内存中的每一个bite. 也就是说若在Heap memory里离起始点第n个8 byte配置一块内存时, HeapBitmap内存里第n个bite就被设置起来.

继续分析跟GC有关的函数 gcForMalloc

//Heap.cpp
/* Do a full garbage collection, which may grow the
 * heap as a side-effect if the live set is large.
 */
static void gcForMalloc(bool clearSoftReferences)
{
    if (gDvm.allocProf.enabled) {
        Thread* self = dvmThreadSelf();
        gDvm.allocProf.gcCount++;
        if (self != NULL) {
            self->allocProf.gcCount++;
        }
    }
    /* This may adjust the soft limit as a side-effect.
     */
    const GcSpec *spec = clearSoftReferences ? GC_BEFORE_OOM :
                      GC_FOR_MALLOC;
    dvmCollectGarbageInternal(spec);
}
 
/*
 * Initiate garbage collection.
 *
 * NOTES:
 * - If we don't hold gDvm.threadListLock, it's possible for a thread to
 *   be added to the thread list while we work.  The thread should NOT
 *   start executing, so this is only interesting when we start chasing
 *   thread stacks.  (Before we do so, grab the lock.)
 *
 * We are not allowed to GC when the debugger has suspended the VM, which
 * is awkward because debugger requests can cause allocations.  The easiest
 * way to enforce this is to refuse to GC on an allocation made by the
 * JDWP thread -- we have to expand the heap or fail.
 */
void dvmCollectGarbageInternal(const GcSpec* spec)
{
    //do something
 
    // suspend all thread
    dvmSuspendAllThreads(SUSPEND_FOR_GC);
 
    dvmMethodTraceGCBegin();
    /* Set up the marking context.
     */
    if (!dvmHeapBeginMarkStep(spec->isPartial)) {
        LOGE_HEAP("dvmHeapBeginMarkStep failed; aborting");
        dvmAbort();
    }
    /* Mark the set of objects that are strongly reachable from the roots.
     */
    dvmHeapMarkRootSet();
    /* dvmHeapScanMarkedObjects() will build the lists of known
     * instances of the Reference classes.
     */
    assert(gcHeap->softReferences == NULL);
    assert(gcHeap->weakReferences == NULL);
    assert(gcHeap->finalizerReferences == NULL);
    assert(gcHeap->phantomReferences == NULL);
    assert(gcHeap->clearedReferences == NULL);
    /* Recursively mark any objects that marked objects point to strongly.
     * If we're not collecting soft references, soft-reachable
     * objects will also be marked.
     */
    dvmHeapScanMarkedObjects();
    /*
     * All strongly-reachable objects have now been marked.  Process
     * weakly-reachable objects discovered while tracing.
     */
    dvmHeapProcessReferences(&gcHeap->softReferences,
                             spec->doPreserve == false,
                             &gcHeap->weakReferences,
                             &gcHeap->finalizerReferences,
                             &gcHeap->phantomReferences);
    //Sweeping...
    dvmHeapSweepSystemWeaks();   
    /*
     * Live objects have a bit set in the mark bitmap, swap the mark
     * and live bitmaps.  The sweep can proceed concurrently viewing
     * the new live bitmap as the old mark bitmap, and vice versa.
     */
    dvmHeapSourceSwapBitmaps(); 
    /*
     * Walk through the list of objects that haven't been marked and free
     * them.  Assumes the bitmaps have been swapped.
     */
    dvmHeapSweepUnmarkedObjects(spec->isPartial, spec->isConcurrent,
                                &numObjectsFreed, &numBytesFreed);
    //Cleaning up...
    dvmHeapFinishMarkStep();
     /* Now's a good time to adjust the heap size, since
     * we know what our utilization is.
     *
     * This doesn't actually resize any memory;
     * it just lets the heap grow more when necessary.
     */
    dvmHeapSourceGrowForUtilization();
    dvmMethodTraceGCEnd();
    //do something
    dvmResumeAllThreads(SUSPEND_FOR_GC);
    //do something
}

最后由上面的程序代码可以知道 dvmCollectGarbageInternal 函数做了很多事, 粗体化的函数跟资源回收有关, 这些函数的功能归纳如下:

1.  suspend 此程序里所有的thread.

2.  初始化一个 Mark Stack用来作为追踪HeapBitmap中的bitmark.

3.  从root开始检查有强可及的对象都做个标记. 检查的地方如下:

    - System classes defined by rootclassloader

    - Foreach thread:

    -Interpreted stack, from top to "curFrame"

    -Dalvik registers (args + local vars)

    - JNIlocal references

    -Automatic VM local references (TrackedAlloc)

    -Associated Thread/VMThread object

    -ThreadGroups (could track & start with these instead of working upward from

      Threads)

    -Exception currently being thrown, if present

  - JNIglobal references

  -Interned string table

  -Primitive classes

  -Special objects

    -gDvm.outOfMemoryObj

  -Objects in debugger object registry

4.     将在Heapmemory中的所有的对象只要不是被soft reference 或是 weakreference 所参考, 全部都做个标记且在bitmapmemory中的位作设定. 并且将对象推进Mark Stack.

5.     开始处理heapmemory中的所有soft reference 跟 weakreference, 将这些的reference全部都设为NULL, 使其可及对象失去参考对象.

6.     将所有在Heapmemory中没被标记的系统对象全部回收.

7.     活动对象在markbitmap中有一个位设定, 就将livebitmap 跟 mark bitmap 互换. 反之亦然. 将有标记跟没标记的活动对象作分类.

8.     开始走访bitmapmemory检查位, 将有设定的位跟未设定的位取出来, 来找出未标记的对象并做回收.

9.     将bitmapmemory里的位全部从新初始化, 且删除MarkStack.

10.   利用HeapSource从新调整Heap memory size.

11.   Resume 此程序里所有的thread.

Conclusion

        由前面的分析可以归纳一下AndroidDVM 的内存管理是如何处理资源回收机制. 在android 一开机的时候就开始处理DVM的初始化, 在这初始化中, 做三件事.

1.     划分一块mSpace 的内存区域.

2.     在这个mSpace 内存区域中配置一块Heap memory, 由Heapsource来设定此块内存的属性. 且配置一块Bitmap memory来记录此Heap memory的状态.

3.     另外配置一块mark stack memory来存放所配置的对象.

当java程序需要实体化一个对象时, 其流程会从dvmMalloc函数开始. 一开始便是一旦在Heapmemory中配置了一块内存就会在Heap memory中的BitmapMemory中去设定位. 之后再由gcForMalloc去执行回收机制, 其回收机制就利用对象的标记与否来判定是否要回收. 在分析中有提到强可及对象,在java的语法中只要一般参考指向new出来的对象,这个参考就称为强参考或是强引用,而相对的对象就叫强可及物件.一旦参考若在程序某处被指定为NULL或是指向别的对象,此时原本所指的对象就退化为软可及物件.  强可及对象跟软可及对象是跟被指向的参考有关, Java的参考有分强, 软, 弱, 虚四种参考之分, 这是属于另外一个topic, 在网络上有很多高手在讨论. AndroidDVM 内存管理就分析到此为止, 这里只是把重点点了出来, 想要更了解详细作法还是需要搭配程序代码来研究.