谈谈fresco的bitmap内存分配

bitmap的内存分配主要有两个含义:
  1. 应用程序实现时针对bitmap的内存缓存;
  2. 从原始数据(byte[])经过decode生成bitmap过程中的内存分配问题;

其中第一个含义,在Fresco中实现对应的就是interface MemoryCache,这里略有提及,不会大费章节,主要关注decode过程中的内存分配问题。

WHAT?
http://frescolib.org/关于memory的介绍可以知道,在4.x及以下的系统上,Fresco的bitmap decode会把bitmap的pixel data(像素数据)放到一个“特殊的内存区域’,这个特殊的内存区域其实就是ashmem,至于ashmem是什么,详细的可以参考罗升阳的博客(http://blog.csdn.net/luoshengyang/article/details/6651971),这里不详述。

WHY?
那为什么在4.x及以下系统需要这样做?原因是如果Bitmap数量很多时会占用大量的内存(这里内存特指Java Heap),必然就会更加频繁的触发虚拟机进行GCGC会导致stop the world,就会出现卡顿,而5.0之后采用了art模式,对GC进行了优化,情况比较乐观(为什么这么说?另说),而在4.x及以下的手机就比较糟糕了,所以,在4.x及以下系统上Fresco选择了ashmem作为pixel data的存储区域。(对于想深究GC的问题,可自行分析dalvik和art的相关代码)

另外,Java Heap的GC问题既然会导致freeze的问题,那么我们也可以采用native heap,这个不受GC的影响,而且内存很大(基本可以认为只受物理内存大小影响),需要的只是特别注意去回收内存。那为什么不用native heap,而去选择ashmem呢?这个问题后面会讲解。

HOW?
Bitmap在Fresco这个框架是在decode时,内存被分配到ashmem中呢?接下来从代码上,慢慢分析这个过程。

PlatformDecoder
Fresco中,提供了PlatformDecoder这个接口用于处理bitmap的decode过程,下面包括了该接口的具体实现类;decoder-class-view.png

从类名就可以推测,Fresco根据系统版本不同,使用不同的decoder,从下面的代码也大致证实了这个推测(但有所差别):
  1. /** * * source : com/facebook/imagepipeline/core/ImagePipelineFactory.java * */
  2. /**
  3. * Provide the implementation of the PlatformDecoder for the current platform using the
  4. * provided PoolFactory
  5. *
  6. * @param poolFactory The PoolFactory
  7. * @return The PlatformDecoder implementation
  8. */
  9. public static PlatformDecoder buildPlatformDecoder(
  10. PoolFactory poolFactory,
  11. boolean directWebpDirectDecodingEnabled,
  12. BitmapCounterTracker bitmapCounterTracker) {
  13. if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
  14. int maxNumThreads = poolFactory.getFlexByteArrayPoolMaxNumThreads();
  15. return new ArtDecoder(
  16. poolFactory.getBitmapPool(),
  17. maxNumThreads,
  18. new Pools.SynchronizedPool<>(maxNumThreads));
  19. } else {
  20. if (directWebpDirectDecodingEnabled
  21. && Build.VERSION.SDK_INT < Build.VERSION_CODES.KITKAT) {
  22. return new GingerbreadPurgeableDecoder(bitmapCounterTracker);
  23. } else {
  24. return new KitKatPurgeableDecoder(
  25. poolFactory.getFlexByteArrayPool(), bitmapCounterTracker);
  26. }
  27. }
  28. }
1.1

分别是,5.0以上都使用ArtDecoder,4.4以下如果directWebpDirectDecodingEnabled为true,则使用GingerbreadPurgeableDecoder,不然其他都使用KitKatPurgeableDecoder这些decoder主要包含两个方法对原始数据进行decode:decodeFromEncodedImage和decodeJPEGFromEncodedImage。由于在decode环节上,这两个方法并无二致,为了方便,所以就选择decodeFromEncodedImage进行深入分析。

DalvikPurgeableDecoder
先看DalvikPurgeableDecoder的decodeFromEncodedImage方法:
  1. /** * * source : com/facebook/imagepipeline/platform/DalvikPurgeableDecoder.java * */
  2. @Override
  3. public CloseableReference<Bitmap> decodeFromEncodedImage(
  4. final EncodedImage encodedImage,
  5. Bitmap.Config bitmapConfig) {
  6. BitmapFactory.Options options = getBitmapFactoryOptions(
  7. encodedImage.getSampleSize(),
  8. bitmapConfig);
  9. CloseableReference<PooledByteBuffer> bytesRef = encodedImage.getByteBufferRef();
  10. Preconditions.checkNotNull(bytesRef);
  11. try {
  12. Bitmap bitmap = decodeByteArrayAsPurgeable(bytesRef, options);
  13. return pinBitmap(bitmap);
  14. } finally {
  15. CloseableReference.closeSafely(bytesRef);
  16. }
  17. }
  1. private static BitmapFactory.Options getBitmapFactoryOptions(
  2. int sampleSize,
  3. Bitmap.Config bitmapConfig) {
  4. BitmapFactory.Options options = new BitmapFactory.Options();
  5. options.inDither = true; // known to improve picture quality at low cost
  6. options.inPreferredConfig = bitmapConfig;
  7. // Decode the image into a 'purgeable' bitmap that lives on the ashmem heap
  8. options.inPurgeable = true;
  9. // Enable copy of of bitmap to enable purgeable decoding by filedescriptor
  10. options.inInputShareable = true;
  11. // Sample size should ONLY be different than 1 when downsampling is enabled in the pipeline
  12. options.inSampleSize = sampleSize;
  13. if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB) {
  14. options.inMutable = true; // no known perf difference; allows postprocessing to work
  15. }
  16. return options;
  17. }
2.1
着重关注和内存分配相关的两个设置:inPurgeable和inInputShareable。上面注释看,inPurgeable能使bitmap的内存分配到ashmem上,对于通过filedescriptor去decode的方式,还要设置inInputShareable为true,只能够使内存分配到ashmem上。先关注inPurgeable
  1. /**
  2. * If this is set to true, then the resulting bitmap will allocate its
  3. * pixels such that they can be purged if the system needs to reclaim
  4. * memory. In that instance, when the pixels need to be accessed again
  5. * (e.g. the bitmap is drawn, getPixels() is called), they will be
  6. * automatically re-decoded.
  7. *
  8. * <p>For the re-decode to happen, the bitmap must have access to the
  9. * encoded data, either by sharing a reference to the input
  10. * or by making a copy of it. This distinction is controlled by
  11. * inInputShareable. If this is true, then the bitmap may keep a shallow
  12. * reference to the input. If this is false, then the bitmap will
  13. * explicitly make a copy of the input data, and keep that. Even if
  14. * sharing is allowed, the implementation may still decide to make a
  15. * deep copy of the input data.</p>
  16. *
  17. * <p>While inPurgeable can help avoid big Dalvik heap allocations (from
  18. * API level 11 onward), it sacrifices performance predictability since any
  19. * image that the view system tries to draw may incur a decode delay which
  20. * can lead to dropped frames. Therefore, most apps should avoid using
  21. * inPurgeable to allow for a fast and fluid UI. To minimize Dalvik heap
  22. * allocations use the {@link #inBitmap} flag instead.</p>
  23. *
  24. * <p class="note"><strong>Note:</strong> This flag is ignored when used
  25. * with {@link #decodeResource(Resources, int,
  26. * android.graphics.BitmapFactory.Options)} or {@link #decodeFile(String,
  27. * android.graphics.BitmapFactory.Options)}.</p>
  28. */
  29. public boolean inPurgeable;
2.2
从注释看inPurgeable的作用是,在KITKAT及以下,使用该参数的话,当系统需要回收内存的时候,bitmap的pixels可以被清除。好在的是,当pixels需要被重新访问的时候(例如bitmap draw或者调用getPixels()的时候),它们又可以重新被decode出来。要重新decode出来的话,自然需要encoded data,encdoded data可能来源于对原始那份encoded data的引用,或者是对原始数据的拷贝。具体是引用或者拷贝,就是根据inInputShareable来决定的,如果是true那就是引用,不然就是deep copy,但是inInputShareable即使设置为true,不同的实现也可能是直接进行deep copy。而大的问题是,inPurgeable虽然能够避免在Java heap中分配内存,但是牺牲点了UI的流畅性,为什么?因为重新decode的过程是在UI线程进行的,这会导致一定的掉帧问题。

inPurgeable可以引出了两个问题:
  1. Decode过程是如何根据它来把pixels分配到ashmem?sdk文档中并没有提及到这一点,是不是真的如此?
  2. Decode为什么会导致drop frames,又该如何解决?

Pixels到ashmem过程
回到decodeFromEncodedImage,它主要是调用decodeByteArrayAsPurgeable进行decode的过程,为了方便(因为GingerbreadPurgeableDecoder的decode过程较为复杂,而且存在一定问题,留待后面解析),我们分析KitKatPurgeableDecoder的该方法:
  1. /**
  2. * Decodes a byteArray into a purgeable bitmap
  3. *
  4. * @param bytesRef the byte buffer that contains the encoded bytes
  5. * @return
  6. */
  7. @Override
  8. protected Bitmap decodeByteArrayAsPurgeable(
  9. CloseableReference<PooledByteBuffer> bytesRef,
  10. BitmapFactory.Options options) {
  11. final PooledByteBuffer pooledByteBuffer = bytesRef.get();
  12. final int length = pooledByteBuffer.size();
  13. final CloseableReference<byte[]> encodedBytesArrayRef = mFlexByteArrayPool.get(length);
  14. try {
  15. final byte[] encodedBytesArray = encodedBytesArrayRef.get();
  16. pooledByteBuffer.read(0, encodedBytesArray, 0, length);
  17. Bitmap bitmap = BitmapFactory.decodeByteArray(
  18. encodedBytesArray,
  19. 0,
  20. length,
  21. options);
  22. return Preconditions.checkNotNull(bitmap, "BitmapFactory returned null");
  23. } finally {
  24. CloseableReference.closeSafely(encodedBytesArrayRef);
  25. }
  26. }
3.1
很简单,就是就是copy数据到byte[],然后调用BitmapFactory的decodeByteArray进行decode。BitmapFactory的decodeByteArray最终通过jni的方式,调用BitmapFactory.cpp的nativeDecodeByteArray方法:
  1. static jobject nativeDecodeByteArray(JNIEnv* env, jobject, jbyteArray byteArray,
  2. int offset, int length, jobject options) {
  3. /* If optionsShareable() we could decide to just wrap the java array and
  4. share it, but that means adding a globalref to the java array object
  5. and managing its lifetime. For now we just always copy the array's data
  6. if optionsPurgeable(), unless we're just decoding bounds.
  7. */
  8. bool purgeable = optionsPurgeable(env, options) && !optionsJustBounds(env, options);
  9. AutoJavaByteArray ar(env, byteArray);
  10. SkMemoryStream* stream = new SkMemoryStream(ar.ptr() + offset, length, purgeable);
  11. SkAutoUnref aur(stream);
  12. return doDecode(env, stream, NULL, options, purgeable);
  13. }
3.2
从上面代码看,除非BitmapOptions设置了inJustDecodeBounds为true,不然由于inPurgeable为true,所以临时变量purgeable都为true。purgeable的值影响的是SkMemoryStream的区别:
  1. SkMemoryStream::SkMemoryStream(const void* src, size_t size, bool copyData) {
  2. fData = newFromParams(src, size, copyData);
  3. fOffset = 0;
  4. }
  1. static SkData* newFromParams(const void* src, size_t size, bool copyData) {
  2. if (copyData) {
  3. return SkData::NewWithCopy(src, size);
  4. } else {
  5. return SkData::NewWithProc(src, size, NULL, NULL);
  6. }
  7. }
  1. SkData* SkData::NewWithCopy(const void* data, size_t length) {
  2. if (0 == length) {
  3. return SkData::NewEmpty();
  4. }
  5. void* copy = sk_malloc_throw(length); // balanced in sk_free_releaseproc
  6. memcpy(copy, data, length);
  7. return new SkData(copy, length, sk_free_releaseproc, NULL);
  8. }
  1. SkData* SkData::NewWithProc(const void* data, size_t length,
  2. ReleaseProc proc, void* context) {
  3. return new SkData(data, length, proc, context);
  4. }
  1. SkData::SkData(const void* ptr, size_t size, ReleaseProc proc, void* context) {
  2. fPtr = ptr;
  3. fSize = size;
  4. fReleaseProc = proc;
  5. fReleaseProcContext = context;
  6. }
3.3
从上面代码看,区别就是如果是true,则copy一份数据,不然纯粹是引用。我们这里肯定是true,所以固定会copy一份数据,也就是copy一份encoded data。这内存的分配是在native heap上做的。
另外这个刚好反应上面说的,“但是inInputShareable即使设置为true,不同的实现也可能是直接进行deep copy

再看关键的doDecode方法,由于这个方法很长,只截取相关的一部分去看:
  1. // since we "may" create a purgeable imageref, we require the stream be ref'able
  2. // i.e. dynamically allocated, since its lifetime may exceed the current stack
  3. // frame.
  4. static jobject doDecode(JNIEnv* env, SkStreamRewindable* stream, jobject padding,
  5. jobject options, bool allowPurgeable, bool forcePurgeable = false) {
  6. int sampleSize = 1;
  7. SkImageDecoder::Mode mode = SkImageDecoder::kDecodePixels_Mode;
  8. SkBitmap::Config prefConfig = SkBitmap::kARGB_8888_Config;
  9. bool doDither = true;
  10. bool isMutable = false;
  11. float scale = 1.0f;
  12. bool isPurgeable = forcePurgeable || (allowPurgeable && optionsPurgeable(env, options));
  13. bool preferQualityOverSpeed = false;
  14. bool requireUnpremultiplied = false;
  15. jobject javaBitmap = NULL;//其实就是BitmapOptions的inBitmap
  16. if (options != NULL) {
  17. sampleSize = env->GetIntField(options, gOptions_sampleSizeFieldID);
  18. if (optionsJustBounds(env, options)) {
  19. mode = SkImageDecoder::kDecodeBounds_Mode;
  20. }
  21. ........................................
  22. //根据部分参数计算scale的的值(我们这里scale设置为1)
  23. if (env->GetBooleanField(options, gOptions_scaledFieldID)) {
  24. const int density = env->GetIntField(options, gOptions_densityFieldID);
  25. const int targetDensity = env->GetIntField(options, gOptions_targetDensityFieldID);
  26. const int screenDensity = env->GetIntField(options, gOptions_screenDensityFieldID);
  27. if (density != 0 && targetDensity != 0 && density != screenDensity) {
  28. scale = (float) targetDensity / density;
  29. }
  30. }
  31. }
  32. const bool willScale = scale != 1.0f;
  33. isPurgeable &= !willScale;//如果bitmap需要scale那么isPurgeable必然为false
  34. SkImageDecoder* decoder = SkImageDecoder::Factory(stream);
  35. if (decoder == NULL) {
  36. return nullObjectReturn("SkImageDecoder::Factory returned null");
  37. }
  38. ...........................................................
  39. SkAutoTDelete<SkBitmap> adb(outputBitmap == NULL ? new SkBitmap : NULL);
  40. if (outputBitmap == NULL) outputBitmap = adb.get();
  41. NinePatchPeeker peeker(decoder);
  42. decoder->setPeeker(&peeker);
  43. //如果isPurgeable,使用kDecodeBounds_Mode,用于先计算出图片的宽高等,而不是直接decode出pixels来
  44. SkImageDecoder::Mode decodeMode = isPurgeable ? SkImageDecoder::kDecodeBounds_Mode : mode;
  45. ...........................................................................
  46. // Only setup the decoder to be deleted after its stack-based, refcounted
  47. // components (allocators, peekers, etc) are declared. This prevents RefCnt
  48. // asserts from firing due to the order objects are deleted from the stack.
  49. SkAutoTDelete<SkImageDecoder> add(decoder);
  50. AutoDecoderCancel adc(options, decoder);
  51. // To fix the race condition in case "requestCancelDecode"
  52. // happens earlier than AutoDecoderCancel object is added
  53. // to the gAutoDecoderCancelMutex linked list.
  54. if (options != NULL && env->GetBooleanField(options, gOptions_mCancelID)) {
  55. return nullObjectReturn("gOptions_mCancelID");
  56. }
  57. SkBitmap decodingBitmap;
  58. //使用kDecodeBounds_Mode,先计算出图片的宽高等,而不是直接decode出pixels来
  59. if (!decoder->decode(stream, &decodingBitmap, prefConfig, decodeMode)) {
  60. return nullObjectReturn("decoder->decode returned false");
  61. }
  62. .............................................
  63. if (willScale) {
  64. .............
  65. .............
  66. } else {
  67. //把outputBitmap设置为刚才decode出的decodingBitmap
  68. outputBitmap->swap(decodingBitmap);
  69. }
  70. SkPixelRef* pr;
  71. if (isPurgeable) {
  72. pr = installPixelRef(outputBitmap, stream, sampleSize, doDither);
  73. } else {
  74. // if we get here, we're in kDecodePixels_Mode and will therefore
  75. // already have a pixelref installed.
  76. pr = outputBitmap->pixelRef();
  77. }
  78. if (pr == NULL) {
  79. return nullObjectReturn("Got null SkPixelRef");
  80. }
  81. if (!isMutable && javaBitmap == NULL) {
  82. // promise we will never change our pixels (great for sharing and pictures)
  83. pr->setImmutable();
  84. }
  85. // detach bitmap from its autodeleter, since we want to own it now
  86. adb.detach();
  87. if (javaBitmap != NULL) {
  88. bool isPremultiplied = !requireUnpremultiplied;
  89. GraphicsJNI::reinitBitmap(env, javaBitmap, outputBitmap, isPremultiplied);
  90. outputBitmap->notifyPixelsChanged();
  91. // If a java bitmap was passed in for reuse, pass it back
  92. return javaBitmap;
  93. }
  94. int bitmapCreateFlags = 0x0;
  95. if (isMutable) bitmapCreateFlags |= GraphicsJNI::kBitmapCreateFlag_Mutable;
  96. if (!requireUnpremultiplied) bitmapCreateFlags |= GraphicsJNI::kBitmapCreateFlag_Premultiplied;
  97. // now create the java bitmap
  98. return GraphicsJNI::createBitmap(env, outputBitmap, javaAllocator.getStorageObj(),
  99. bitmapCreateFlags, ninePatchChunk, layoutBounds, -1);
  100. }
  1. static SkPixelRef* installPixelRef(SkBitmap* bitmap, SkStreamRewindable* stream,
  2. int sampleSize, bool ditherImage) {
  3. SkImageRef* pr;
  4. // only use ashmem for large images, since mmaps come at a price
  5. if (bitmap->getSize() >= 32 * 1024) {
  6. pr = new SkImageRef_ashmem(stream, bitmap->config(), sampleSize);
  7. } else {
  8. pr = new SkImageRef_GlobalPool(stream, bitmap->config(), sampleSize);
  9. }
  10. pr->setDitherImage(ditherImage);
  11. bitmap->setPixelRef(pr)->unref();
  12. pr->isOpaque(bitmap);
  13. return pr;
  14. }
3.4
从上面红色标注可以大概知道(根据名字),当Bitmap的size大于32k时,bitmap会被分配到ashmem中,不然被分配到一个叫global pool的地方。所以前面提到,“当inPurgeable为ture时,bitmap的pixels会被分配到ashmem上”这个说法过于绝对,不太准确(至少官方kitkat-release这个分支上),准确上应该说“inPurgeable为ture且bitmap占用内存超过32k时,bitmap的pixels会被分配到ashmem上”。

而且,由于inPurgeable 为ture,所以decoder的SkImageDecoder::Mode decodeMode是kDecodeBounds_Mode,看decode函数:
  1. bool SkImageDecoder::decode(SkStream* stream, SkBitmap* bm,
  2. SkBitmap::Config pref, Mode mode) {
  3. // we reset this to false before calling onDecode
  4. fShouldCancelDecode = false;
  5. // assign this, for use by getPrefConfig(), in case fUsePrefTable is false
  6. fDefaultPref = pref;
  7. // pass a temporary bitmap, so that if we return false, we are assured of
  8. // leaving the caller's bitmap untouched.
  9. SkBitmap tmp;
  10. if (!this->onDecode(stream, &tmp, mode)) {
  11. return false;
  12. }
  13. bm->swap(tmp);
  14. return true;
  15. }
3.5
假设decoder是SkPNGImageDecoder,那么
  1. bool SkPNGImageDecoder::onDecode(SkStream* sk_stream, SkBitmap* decodedBitmap,
  2. Mode mode) {
  3. png_structp png_ptr;
  4. png_infop info_ptr;
  5. if (!onDecodeInit(sk_stream, &png_ptr, &info_ptr)) {
  6. return false;
  7. }
  8. if (setjmp(png_jmpbuf(png_ptr))) {
  9. return false;
  10. }
  11. PNGAutoClean autoClean(png_ptr, info_ptr);
  12. png_uint_32 origWidth, origHeight;
  13. int bitDepth, colorType, interlaceType;
  14. png_get_IHDR(png_ptr, info_ptr, &origWidth, &origHeight, &bitDepth,
  15. &colorType, &interlaceType, int_p_NULL, int_p_NULL);
  16. SkBitmap::Config config;
  17. bool hasAlpha = false;
  18. SkPMColor theTranspColor = 0; // 0 tells us not to try to match
  19. if (!this->getBitmapConfig(png_ptr, info_ptr, &config, &hasAlpha, &theTranspColor)) {
  20. return false;
  21. }
  22. const int sampleSize = this->getSampleSize();
  23. SkScaledBitmapSampler sampler(origWidth, origHeight, sampleSize);
  24. decodedBitmap->setConfig(config, sampler.scaledWidth(), sampler.scaledHeight());
  25. if (SkImageDecoder::kDecodeBounds_Mode == mode) {
  26. return true;
  27. }
  28. ................................
  29. }
3.6
可见,如果decode mode是kDecodeBounds_Mode,则不会分配实际内存,只是计算出诸如宽高等信息。而且上面的描述我们只是从函数名推断pixels会被分配到ashmem中,但是哪里会分配到实际的内存呢?但看完整个deDecode的过程,都没有发现有内存分配的地方,那究竟内存什么时候分配,在哪里分配?
回到前面注释说的那段话:当pixels需要被重新访问的时候(例如bitmap draw或者调用getPixels()的时候),它们又可以重新被decode出来。那么猜想会不会在draw bitmap的时候bitmap的内存进行分配的呢?找到Canvas的drawBitmap方法,由此走上开始一段代码copy的路,看:
  1. public void drawBitmap(@NonNull Bitmap bitmap, float left, float top, @Nullable Paint paint) {
  2. throwIfCannotDraw(bitmap);
  3. native_drawBitmap(mNativeCanvasWrapper, bitmap, left, top,
  4. paint != null ? paint.getNativeInstance() : 0, mDensity, mScreenDensity, bitmap.mDensity);
  5. }
  1. {"native_drawBitmap","(IIFFIIII)V",
  2. (void*) SkCanvasGlue::drawBitmap__BitmapFFPaint},
  1. static void drawBitmap__BitmapFFPaint(JNIEnv* env, jobject jcanvas,
  2. SkCanvas* canvas, SkBitmap* bitmap,
  3. jfloat left, jfloat top,
  4. SkPaint* paint, jint canvasDensity,
  5. jint screenDensity, jint bitmapDensity) {
  6. SkScalar left_ = SkFloatToScalar(left);
  7. SkScalar top_ = SkFloatToScalar(top);
  8. if (canvasDensity == bitmapDensity || canvasDensity == 0
  9. || bitmapDensity == 0) {
  10. if (screenDensity != 0 && screenDensity != bitmapDensity) {
  11. SkPaint filteredPaint;
  12. if (paint) {
  13. filteredPaint = *paint;
  14. }
  15. filteredPaint.setFilterBitmap(true);
  16. canvas->drawBitmap(*bitmap, left_, top_, &filteredPaint);
  17. } else {
  18. canvas->drawBitmap(*bitmap, left_, top_, paint);
  19. }
  20. } else {
  21. canvas->save();
  22. SkScalar scale = SkFloatToScalar(canvasDensity / (float)bitmapDensity);
  23. canvas->translate(left_, top_);
  24. canvas->scale(scale, scale);
  25. SkPaint filteredPaint;
  26. if (paint) {
  27. filteredPaint = *paint;
  28. }
  29. filteredPaint.setFilterBitmap(true);
  30. canvas->drawBitmap(*bitmap, 0, 0, &filteredPaint);
  31. canvas->restore();
  32. }
  33. }
  1. void SkCanvas::drawBitmap(const SkBitmap& bitmap, SkScalar x, SkScalar y,
  2. const SkPaint* paint) {
  3. SkDEBUGCODE(bitmap.validate();)
  4. if (NULL == paint || paint->canComputeFastBounds()) {
  5. SkRect bounds = {
  6. x, y,
  7. x + SkIntToScalar(bitmap.width()),
  8. y + SkIntToScalar(bitmap.height())
  9. };
  10. if (paint) {
  11. (void)paint->computeFastBounds(bounds, &bounds);
  12. }
  13. if (this->quickReject(bounds)) {
  14. return;
  15. }
  16. }
  17. SkMatrix matrix;
  18. matrix.setTranslate(x, y);
  19. this->internalDrawBitmap(bitmap, matrix, paint);
  20. }
  1. void SkCanvas::internalDrawBitmap(const SkBitmap& bitmap,
  2. const SkMatrix& matrix, const SkPaint* paint) {
  3. if (reject_bitmap(bitmap)) {
  4. return;
  5. }
  6. SkLazyPaint lazy;
  7. if (NULL == paint) {
  8. paint = lazy.init();
  9. }
  10. SkDEBUGCODE(bitmap.validate();)
  11. CHECK_LOCKCOUNT_BALANCE(bitmap);
  12. LOOPER_BEGIN(*paint, SkDrawFilter::kBitmap_Type)
  13. while (iter.next()) {
  14. iter.fDevice->drawBitmap(iter, bitmap, matrix, looper.paint());
  15. }
  16. LOOPER_END
  17. }
  1. void SkDevice::drawBitmap(const SkDraw& draw, const SkBitmap& bitmap,
  2. const SkMatrix& matrix, const SkPaint& paint) {
  3. draw.drawBitmap(bitmap, matrix, paint);
  4. }
  1. void SkDraw::drawBitmap(const SkBitmap& bitmap, const SkMatrix& prematrix,
  2. const SkPaint& origPaint) const {
  3. SkDEBUGCODE(this->validate();)
  4. // nothing to draw
  5. if (fRC->isEmpty() ||
  6. bitmap.width() == 0 || bitmap.height() == 0 ||
  7. bitmap.getConfig() == SkBitmap::kNo_Config) {
  8. return;
  9. }
  10. SkPaint paint(origPaint);
  11. paint.setStyle(SkPaint::kFill_Style);
  12. SkMatrix matrix;
  13. if (!matrix.setConcat(*fMatrix, prematrix)) {
  14. return;
  15. }
  16. if (clipped_out(matrix, *fRC, bitmap.width(), bitmap.height())) {
  17. return;
  18. }
  19. if (fBounder && just_translate(matrix, bitmap)) {
  20. SkIRect ir;
  21. int32_t ix = SkScalarRound(matrix.getTranslateX());
  22. int32_t iy = SkScalarRound(matrix.getTranslateY());
  23. ir.set(ix, iy, ix + bitmap.width(), iy + bitmap.height());
  24. if (!fBounder->doIRect(ir)) {
  25. return;
  26. }
  27. }
  28. if (bitmap.getConfig() != SkBitmap::kA8_Config &&
  29. just_translate(matrix, bitmap)) {
  30. //
  31. // It is safe to call lock pixels now, since we know the matrix is
  32. // (more or less) identity.
  33. //
  34. SkAutoLockPixels alp(bitmap);
  35. if (!bitmap.readyToDraw()) {
  36. return;
  37. }
  38. int ix = SkScalarRound(matrix.getTranslateX());
  39. int iy = SkScalarRound(matrix.getTranslateY());
  40. if (clipHandlesSprite(*fRC, ix, iy, bitmap)) {
  41. uint32_t storage[kBlitterStorageLongCount];
  42. SkBlitter* blitter = SkBlitter::ChooseSprite(*fBitmap, paint, bitmap,
  43. ix, iy, storage, sizeof(storage));
  44. if (blitter) {
  45. SkAutoTPlacementDelete<SkBlitter> ad(blitter, storage);
  46. SkIRect ir;
  47. ir.set(ix, iy, ix + bitmap.width(), iy + bitmap.height());
  48. SkScan::FillIRect(ir, *fRC, blitter);
  49. return;
  50. }
  51. }
  52. }
  53. ................................
  54. ................................
  55. }
  1. class SkAutoLockPixels : public SkNoncopyable {
  2. public:
  3. SkAutoLockPixels(const SkBitmap& bm, bool doLock = true) : fBitmap(bm) {
  4. fDidLock = doLock;
  5. if (doLock) {
  6. bm.lockPixels();
  7. }
  8. }
  9. ~SkAutoLockPixels() {
  10. if (fDidLock) {
  11. fBitmap.unlockPixels();
  12. }
  13. }
  14. private:
  15. const SkBitmap& fBitmap;
  16. bool fDidLock;
  17. };
  1. void SkBitmap::lockPixels() const {
  2. if (NULL != fPixelRef && 0 == sk_atomic_inc(&fPixelLockCount)) {
  3. fPixelRef->lockPixels();
  4. this->updatePixelsFromRef();
  5. }
  6. SkDEBUGCODE(this->validate();)
  7. }
  1. void SkPixelRef::lockPixels() {
  2. SkASSERT(!fPreLocked || SKPIXELREF_PRELOCKED_LOCKCOUNT == fLockCount);
  3. if (!fPreLocked) {
  4. SkAutoMutexAcquire ac(*fMutex);
  5. if (1 == ++fLockCount) {
  6. fPixels = this->onLockPixels(&fColorTable);
  7. }
  8. }
  9. }
3.7
最后一段代码,this->onLockPixels这里的this就是前面SkImageRef_ashmem对象,那么:
  1. void* SkImageRef_ashmem::onLockPixels(SkColorTable** ct) {
  2. SkASSERT(fBitmap.getPixels() == NULL);
  3. SkASSERT(fBitmap.getColorTable() == NULL);
  4. // fast case: check if we can just pin and get the cached data
  5. if (-1 != fRec.fFD) {//由于未分配内存,fFD为-1
  6. SkASSERT(fRec.fAddr);
  7. SkASSERT(!fRec.fPinned);
  8. int pin = ashmem_pin_region(fRec.fFD, 0, 0);
  9. if (ASHMEM_NOT_PURGED == pin) { // yea, fast case!
  10. fBitmap.setPixels(fRec.fAddr, fCT);
  11. fRec.fPinned = true;
  12. } else if (ASHMEM_WAS_PURGED == pin) {
  13. ashmem_unpin_region(fRec.fFD, 0, 0);
  14. // let go of our colortable if we lost the pixels. Well get it back
  15. // again when we re-decode
  16. if (fCT) {
  17. fCT->unref();
  18. fCT = NULL;
  19. }
  20. #if defined(DUMP_ASHMEM_LIFECYCLE) || defined(TRACE_ASH_PURGE)
  21. SkDebugf("===== ashmem purged %d\n", fBitmap.getSize());
  22. #endif
  23. } else {
  24. SkDebugf("===== ashmem pin_region(%d) returned %d\n", fRec.fFD, pin);
  25. // return null result for failure
  26. if (ct) {
  27. *ct = NULL;
  28. }
  29. return NULL;
  30. }
  31. } else {
  32. // no FD, will create an ashmem region in allocator
  33. }
  34. return this->INHERITED::onLockPixels(ct);
  35. }
  1. void* SkImageRef::onLockPixels(SkColorTable** ct) {
  2. if (NULL == fBitmap.getPixels()) {
  3. (void)this->prepareBitmap(SkImageDecoder::kDecodePixels_Mode);
  4. }
  5. if (ct) {
  6. *ct = fBitmap.getColorTable();
  7. }
  8. return fBitmap.getPixels();
  9. }
  1. bool SkImageRef::prepareBitmap(SkImageDecoder::Mode mode) {
  2. if (fErrorInDecoding) {
  3. return false;
  4. }
  5. /* As soon as we really know our config, we record it, so that on
  6. subsequent calls to the codec, we are sure we will always get the same
  7. result.
  8. */
  9. if (SkBitmap::kNo_Config != fBitmap.config()) {
  10. fConfig = fBitmap.config();
  11. }
  12. if (NULL != fBitmap.getPixels() ||
  13. (SkBitmap::kNo_Config != fBitmap.config() &&
  14. SkImageDecoder::kDecodeBounds_Mode == mode)) {
  15. return true;
  16. }
  17. SkASSERT(fBitmap.getPixels() == NULL);
  18. if (!fStream->rewind()) {
  19. SkDEBUGF(("Failed to rewind SkImageRef stream!"));
  20. return false;
  21. }
  22. SkImageDecoder* codec;
  23. if (fFactory) {
  24. codec = fFactory->newDecoder(fStream);
  25. } else {
  26. codec = SkImageDecoder::Factory(fStream);
  27. }
  28. if (codec) {
  29. SkAutoTDelete<SkImageDecoder> ad(codec);
  30. codec->setSampleSize(fSampleSize);
  31. codec->setDitherImage(fDoDither);
  32. if (this->onDecode(codec, fStream, &fBitmap, fConfig, mode)) {
  33. return true;
  34. }
  35. }
  36. #ifdef DUMP_IMAGEREF_LIFECYCLE
  37. if (NULL == codec) {
  38. SkDebugf("--- ImageRef: <%s> failed to find codec\n", this->getURI());
  39. } else {
  40. SkDebugf("--- ImageRef: <%s> failed in codec for %d mode\n",
  41. this->getURI(), mode);
  42. }
  43. #endif
  44. fErrorInDecoding = true;
  45. fBitmap.reset();
  46. return false;
  47. }
  1. bool SkImageRef_ashmem::onDecode(SkImageDecoder* codec, SkStream* stream,
  2. SkBitmap* bitmap, SkBitmap::Config config,
  3. SkImageDecoder::Mode mode) {
  4. if (SkImageDecoder::kDecodeBounds_Mode == mode) {
  5. return this->INHERITED::onDecode(codec, stream, bitmap, config, mode);
  6. }
  7. // Ashmem memory is guaranteed to be initialized to 0.
  8. codec->setSkipWritingZeroes(true);
  9. AshmemAllocator alloc(&fRec, this->getURI());
  10. codec->setAllocator(&alloc);
  11. bool success = this->INHERITED::onDecode(codec, stream, bitmap, config,
  12. mode);
  13. // remove the allocator, since its on the stack
  14. codec->setAllocator(NULL);
  15. if (success) {
  16. // remember the colortable (if any)
  17. SkRefCnt_SafeAssign(fCT, bitmap->getColorTable());
  18. return true;
  19. } else {
  20. if (fRec.fPinned) {
  21. ashmem_unpin_region(fRec.fFD, 0, 0);
  22. fRec.fPinned = false;
  23. }
  24. this->closeFD();
  25. return false;
  26. }
  27. }
  1. bool SkImageRef::onDecode(SkImageDecoder* codec, SkStream* stream,
  2. SkBitmap* bitmap, SkBitmap::Config config,
  3. SkImageDecoder::Mode mode) {
  4. return codec->decode(stream, bitmap, config, mode);
  5. }
  1. bool SkImageDecoder::decode(SkStream* stream, SkBitmap* bm,
  2. SkBitmap::Config pref, Mode mode) {
  3. // we reset this to false before calling onDecode
  4. fShouldCancelDecode = false;
  5. // assign this, for use by getPrefConfig(), in case fUsePrefTable is false
  6. fDefaultPref = pref;
  7. // pass a temporary bitmap, so that if we return false, we are assured of
  8. // leaving the caller's bitmap untouched.
  9. SkBitmap tmp;
  10. if (!this->onDecode(stream, &tmp, mode)) {
  11. return false;
  12. }
  13. bm->swap(tmp);
  14. return true;
  15. }
3.8
假设图片是png图片,那么使用的decoder就是SkPNGImageDecoder:
  1. bool SkPNGImageDecoder::onDecode(SkStream* sk_stream, SkBitmap* decodedBitmap,
  2. Mode mode) {
  3. png_structp png_ptr;
  4. png_infop info_ptr;
  5. if (!onDecodeInit(sk_stream, &png_ptr, &info_ptr)) {
  6. return false;
  7. }
  8. if (setjmp(png_jmpbuf(png_ptr))) {
  9. return false;
  10. }
  11. PNGAutoClean autoClean(png_ptr, info_ptr);
  12. png_uint_32 origWidth, origHeight;
  13. int bitDepth, colorType, interlaceType;
  14. png_get_IHDR(png_ptr, info_ptr, &origWidth, &origHeight, &bitDepth,
  15. &colorType, &interlaceType, int_p_NULL, int_p_NULL);
  16. SkBitmap::Config config;
  17. bool hasAlpha = false;
  18. SkPMColor theTranspColor = 0; // 0 tells us not to try to match
  19. if (!this->getBitmapConfig(png_ptr, info_ptr, &config, &hasAlpha, &theTranspColor)) {
  20. return false;
  21. }
  22. const int sampleSize = this->getSampleSize();
  23. SkScaledBitmapSampler sampler(origWidth, origHeight, sampleSize);
  24. decodedBitmap->setConfig(config, sampler.scaledWidth(), sampler.scaledHeight());
  25. if (SkImageDecoder::kDecodeBounds_Mode == mode) {
  26. return true;
  27. }
  28. // from here down we are concerned with colortables and pixels
  29. // we track if we actually see a non-opaque pixels, since sometimes a PNG sets its colortype
  30. // to |= PNG_COLOR_MASK_ALPHA, but all of its pixels are in fact opaque. We care, since we
  31. // draw lots faster if we can flag the bitmap has being opaque
  32. bool reallyHasAlpha = false;
  33. SkColorTable* colorTable = NULL;
  34. if (colorType == PNG_COLOR_TYPE_PALETTE) {
  35. decodePalette(png_ptr, info_ptr, &hasAlpha, &reallyHasAlpha, &colorTable);
  36. }
  37. SkAutoUnref aur(colorTable);
  38. if (!this->allocPixelRef(decodedBitmap,
  39. SkBitmap::kIndex8_Config == config ? colorTable : NULL)) {
  40. return false;
  41. }
  42. .......................................
  43. .......................................
  44. }
  1. bool SkImageDecoder::allocPixelRef(SkBitmap* bitmap,
  2. SkColorTable* ctable) const {
  3. return bitmap->allocPixels(fAllocator, ctable);
  4. }
  1. bool SkBitmap::allocPixels(Allocator* allocator, SkColorTable* ctable) {
  2. HeapAllocator stdalloc;
  3. if (NULL == allocator) {
  4. allocator = &stdalloc;
  5. }
  6. return allocator->allocPixelRef(this, ctable);
  7. }
3.9
这里的allocator就是AshmemAllocator:
  1. class AshmemAllocator : public SkBitmap::Allocator {
  2. public:
  3. AshmemAllocator(SkAshmemRec* rec, const char name[])
  4. : fRec(rec), fName(name) {}
  5. virtual bool allocPixelRef(SkBitmap* bm, SkColorTable* ct) {
  6. const size_t size = roundToPageSize(bm->getSize());
  7. int fd = fRec->fFD;
  8. void* addr = fRec->fAddr;
  9. SkASSERT(!fRec->fPinned);
  10. if (-1 == fd) {
  11. SkASSERT(NULL == addr);
  12. SkASSERT(0 == fRec->fSize);
  13. fd = ashmem_create_region(fName, size);
  14. #ifdef DUMP_ASHMEM_LIFECYCLE
  15. SkDebugf("=== ashmem_create_region %s size=%d fd=%d\n", fName, size, fd);
  16. #endif
  17. if (-1 == fd) {
  18. SkDebugf("------- imageref_ashmem create failed <%s> %d\n",
  19. fName, size);
  20. return false;
  21. }
  22. int err = ashmem_set_prot_region(fd, PROT_READ | PROT_WRITE);
  23. if (err) {
  24. SkDebugf("------ ashmem_set_prot_region(%d) failed %d\n",
  25. fd, err);
  26. close(fd);
  27. return false;
  28. }
  29. addr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
  30. if (-1 == (long)addr) {
  31. SkDebugf("---------- mmap failed for imageref_ashmem size=%d\n",
  32. size);
  33. close(fd);
  34. return false;
  35. }
  36. fRec->fFD = fd;
  37. fRec->fAddr = addr;
  38. fRec->fSize = size;
  39. } else {
  40. SkASSERT(addr);
  41. SkASSERT(size == fRec->fSize);
  42. (void)ashmem_pin_region(fd, 0, 0);
  43. }
  44. bm->setPixels(addr, ct);
  45. fRec->fPinned = true;
  46. return true;
  47. }
  48. private:
  49. // we just point to our caller's memory, these are not copies
  50. SkAshmemRec* fRec;
  51. const char* fName;
  52. };
3.10
好了,到此为止,我们终于找到对于isPurgeable为true的图片,内存分配的时机和流程,只有当bitmap进行draw或者getPixels(这个大抵一致)的时候,bitmap 的pixels才进行实际的内存分配,而且也发现了前面说的当bitmap重新进行decode的时候会导致drop frames的原因,因为从上面代码看,bitmap draw的整个流程看,操作都处于UI线程,由于存在耗时的操作,自然就会导致不流畅的问题。

Why drop frames?
isPurgeable为ture的图片在UI线程上decode会出现掉帧问题,那怎么办?Fresco的思路是(个人推测),既然不管是初始decode还是被清除掉后re-decode,都会有卡顿问题,那么可以提前把该内存区域锁定住,既做到自行预先进行pixels的内存分配解决初始decode的卡顿,又可以防止该区域内存在系统内存资源紧张下被回收后re-decode时卡顿问题。恰恰,系统确实提供了这个方法,就是AndroidBitmap_lockPixels,把这个行为称作pin

在Fresco的DalvikPurgeableDecoder中,
  1. /**
  2. * Creates a bitmap from encoded bytes.
  3. *
  4. * @param encodedImage the encoded image with reference to the encoded bytes
  5. * @param bitmapConfig the {@link android.graphics.Bitmap.Config}
  6. * used to create the decoded Bitmap
  7. * @return the bitmap
  8. * @throws TooManyBitmapsException if the pool is full
  9. * @throws java.lang.OutOfMemoryError if the Bitmap cannot be allocated
  10. */
  11. @Override
  12. public CloseableReference<Bitmap> decodeFromEncodedImage(
  13. final EncodedImage encodedImage,
  14. Bitmap.Config bitmapConfig) {
  15. BitmapFactory.Options options = getBitmapFactoryOptions(
  16. encodedImage.getSampleSize(),
  17. bitmapConfig);
  18. CloseableReference<PooledByteBuffer> bytesRef = encodedImage.getByteBufferRef();
  19. Preconditions.checkNotNull(bytesRef);
  20. try {
  21. Bitmap bitmap = decodeByteArrayAsPurgeable(bytesRef, options);
  22. return pinBitmap(bitmap);
  23. } finally {
  24. CloseableReference.closeSafely(bytesRef);
  25. }
  26. }
4.1
每次decode完后,都会进行pinBitmap的操作,那么pinBitmap干什么?
  1. /**
  2. * Pins the bitmap
  3. */
  4. public CloseableReference<Bitmap> pinBitmap(Bitmap bitmap) {
  5. try {
  6. // Real decoding happens here - if the image was corrupted, this will throw an exception
  7. Bitmaps.pinBitmap(bitmap);
  8. } catch (Exception e) {
  9. bitmap.recycle();
  10. throw Throwables.propagate(e);
  11. }
  12. if (!mUnpooledBitmapsCounter.increase(bitmap)) {
  13. bitmap.recycle();
  14. throw new TooManyBitmapsException();
  15. }
  16. return CloseableReference.of(bitmap, mUnpooledBitmapsCounter.getReleaser());
  17. }
  1. /**
  2. * Pin the bitmap so that it cannot be 'purged'. Only makes sense for purgeable bitmaps
  3. * WARNING: Use with caution. Make sure that the pinned bitmap is recycled eventually. Otherwise,
  4. * this will simply eat up ashmem memory and eventually lead to unfortunate crashes.
  5. * We *may* eventually provide an unpin method - but we don't yet have a compelling use case for
  6. * that.
  7. * @param bitmap the purgeable bitmap to pin
  8. */
  9. public static void pinBitmap(Bitmap bitmap) {
  10. Preconditions.checkNotNull(bitmap);
  11. nativePinBitmap(bitmap);
  12. }
  1. { "nativePinBitmap",
  2. "(Landroid/graphics/Bitmap;)V",
  3. (void*) Bitmaps_pinBitmap },
  1. /**
  2. * Pins bitmap's pixels.
  3. *
  4. * <p> Throws RuntimeException if unable to pin.
  5. */
  6. static void Bitmaps_pinBitmap(
  7. JNIEnv* env,
  8. jclass clazz,
  9. jobject bitmap) {
  10. UNUSED(clazz);
  11. int rc = AndroidBitmap_lockPixels(env, bitmap, 0);
  12. if (rc != ANDROID_BITMAP_RESULT_SUCCESS) {
  13. safe_throw_exception(env, "Failed to pin Bitmap");
  14. }
  15. }
  1. int AndroidBitmap_lockPixels(JNIEnv* env, jobject jbitmap, void** addrPtr) {
  2. if (NULL == env || NULL == jbitmap) {
  3. return ANDROID_BITMAP_RESULT_BAD_PARAMETER;
  4. }
  5. SkBitmap* bm = GraphicsJNI::getNativeBitmap(env, jbitmap);
  6. if (NULL == bm) {
  7. return ANDROID_BITMAP_RESULT_JNI_EXCEPTION;
  8. }
  9. bm->lockPixels();
  10. void* addr = bm->getPixels();
  11. if (NULL == addr) {
  12. bm->unlockPixels();
  13. return ANDROID_BITMAP_RESULT_ALLOCATION_FAILED;
  14. }
  15. if (addrPtr) {
  16. *addrPtr = addr;
  17. }
  18. return ANDROID_BITMAP_RESULT_SUCCESS;
  19. }
4.2
lockPixels的代码前面已经分析过,就是实际进行内存分配的操作,所以pin的操作实际上就是预先分配好pixels的内存,避免在UI线程进行decode,另外再看一下lockPixels函数:
  1. void SkBitmap::lockPixels() const {
  2. if (NULL != fPixelRef && 0 == sk_atomic_inc(&fPixelLockCount)) {
  3. fPixelRef->lockPixels();
  4. this->updatePixelsFromRef();
  5. }
  6. SkDEBUGCODE(this->validate();)
  7. }
4.3
该内存区域会采用引用计数的方法去管理,所以每一次lockPixels,fPixelLockCount都会自增,这样能够保证该内存区域不会被系统自动回收掉,这样就可以避免出现re-decode导致的卡顿问题,但是这就需要自己主动管理该内存区域。

讲到这里终于可以回顾之前的两个问题,并得出结论:
  1. Q:Decode过程是如何根据它来把pixels分配到ashmem?sdk文档中并没有提及到这一点,是不是真的如此?
    A:
    isPurgeable为ture的情况下,bitmap的pixels如果超过32k情况下,会被分配到ashmem,分配的时机可以是在bitmap draw或者手动调用getPixels等方法时,而Fresco采用的是,主动执行“pin”操作去让pixels预先分配到ashmem。32k以下的bitmap会分配到global pool中(这里没有做详细分析,有兴趣的另行分析)。所以说,Fresco的bitmap在4.4及以下系统会被分配到ashmem的说法过于绝对。
  2. Q:Decode为什么会导致drop frames,又该如何解决?
    A:导致drop frames的原因是,
    isPurgeable为ture的bitmap会在UI线程中进行decode,过于耗时,所以自然可能存在卡顿问题。解决的办法就是,预先进行“pin”操作去预先decode出bitmap的pixels到内存(ashmem或者global pool),并可以锁定该内存区域,防止系统回收掉该内存区域,导致re-decode。

GingerbreadPurgeableDecoder
前面对KitKatPurgeableDecoder进行了详细的分析,至于GingerbreadPurgeableDecoder,由于BitmapOptions的isPurgeable都为true,所以在使用系统API(主要是BitmapFactory的decode系列方法)进行decode的过程基本和KitKatPurgeableDecoder是完全一致的。但是,GingerbreadPurgeableDecoder当前master分支上的代码非常让人费解(可能是issue或者可能已经deprecated),其中有这样的一个函数:
  1. /** * * source : com/facebook/imagepipeline/platform/GingerbreadPurgeableDecoder.java * */
  2. protected Bitmap decodeFileDescriptorAsPurgeable(
  3. CloseableReference<PooledByteBuffer> bytesRef,
  4. int inputLength,
  5. byte[] suffix,
  6. BitmapFactory.Options options) {
  7. MemoryFile memoryFile = null;
  8. try {
  9. memoryFile = copyToMemoryFile(bytesRef, inputLength, suffix);
  10. FileDescriptor fd = getMemoryFileDescriptor(memoryFile);
  11. Bitmap bitmap = sWebpBitmapFactory.decodeFileDescriptor(fd, null, options);
  12. return Preconditions.checkNotNull(bitmap, "BitmapFactory returned null");
  13. } catch (IOException e) {
  14. throw Throwables.propagate(e);
  15. } finally {
  16. if (memoryFile != null) {
  17. memoryFile.close();
  18. }
  19. }
  20. }
5.1
它的decode方法依赖于sWebpBitmapFactory这样的一个WebpBitmapFactory对象。为什么说费解?前面说道,4.4及以下如果directWebpDirectDecodingEnabled为true,则使用GingerbreadPurgeableDecoder。而directWebpDirectDecodingEnabled == mConfig.getExperiments().isWebpSupportEnabled()这个值,是对ImagePipelineConfig初始化的时候应用设置的,含义是是否支持Webp格式的图片,假设希望支持,那么设置为true。那么我们再看下面在ImagePipelineConfig的构造函数的部分代码:
  1. /** * * source : com/facebook/imagepipeline/core/ImagePipelineConfig.java * */
  2. // Here we manage the WebpBitmapFactory implementation if any
  3. WebpBitmapFactory webpBitmapFactory = mImagePipelineExperiments.getWebpBitmapFactory();
  4. if (webpBitmapFactory != null) {
  5. BitmapCreator bitmapCreator = new HoneycombBitmapCreator(getPoolFactory());
  6. setWebpBitmapFactory(webpBitmapFactory, mImagePipelineExperiments, bitmapCreator);
  7. } else {
  8. // We check using introspection only if the experiment is enabled
  9. if (mImagePipelineExperiments.isWebpSupportEnabled() &&
  10. WebpSupportStatus.sIsWebpSupportRequired) {
  11. webpBitmapFactory = WebpSupportStatus.loadWebpBitmapFactoryIfExists();
  12. if (webpBitmapFactory != null) {
  13. BitmapCreator bitmapCreator = new HoneycombBitmapCreator(getPoolFactory());
  14. setWebpBitmapFactory(webpBitmapFactory, mImagePipelineExperiments, bitmapCreator);
  15. }
  16. }
  17. }
5.2
  1. /** * * source : com/facebook/common/webp/WebpSupportStatus.java * */
  2. public static final boolean sIsWebpSupportRequired =
  3. Build.VERSION.SDK_INT <= Build.VERSION_CODES.JELLY_BEAN_MR1;
5.3
粗体部分意思是,如果应用设置了isWebpSupportEnabled而且SDK_INT <= 17那么会初始化一个WebpBitmapFactory实例,SDK_INT>=18该实例为NULL。但是回过头去看前面说的“4.4以下如果directWebpDirectDecodingEnabled为true,则使用GingerbreadPurgeableDecoder”,那问题就出现了,SDK_INT=18的虽然会使用GingerbreadPurgeableDecoder去decode bitmap,但是sWebpBitmapFactory却为NULL,然后就会出现NullPointerException。拿了一部API 18的手机实测,该情况确实如此,只不过可能由于webp support部分还是处于“实验”阶段,出现issue也正常。
抛开issue回到GingerbreadPurgeableDecoder上继续说。从2.1代码可知道,GingerbreadPurgeableDecoderdecode前会先把原始数据(encoded data)会copy到MemoryFile中去:
  1. private static MemoryFile copyToMemoryFile(
  2. CloseableReference<PooledByteBuffer> bytesRef,
  3. int inputLength,
  4. @Nullable byte[] suffix) throws IOException {
  5. int outputLength = inputLength + (suffix == null ? 0 : suffix.length);
  6. MemoryFile memoryFile = new MemoryFile(null, outputLength);
  7. memoryFile.allowPurging(false);
  8. PooledByteBufferInputStream pbbIs = null;
  9. LimitedInputStream is = null;
  10. OutputStream os = null;
  11. try {
  12. pbbIs = new PooledByteBufferInputStream(bytesRef.get());
  13. is = new LimitedInputStream(pbbIs, inputLength);
  14. os = memoryFile.getOutputStream();
  15. ByteStreams.copy(is, os);
  16. if (suffix != null) {
  17. memoryFile.writeBytes(suffix, 0, inputLength, suffix.length);
  18. }
  19. return memoryFile;
  20. } finally {
  21. CloseableReference.closeSafely(bytesRef);
  22. Closeables.closeQuietly(pbbIs);
  23. Closeables.closeQuietly(is);
  24. Closeables.close(os, true);
  25. }
  26. }
5.4
从中的好处就是,借助MemoryFile把encoded data拷贝到ashmem中去,尽量避免在Java Heap上分配内存而造成频繁GC的问题,这相对于KitKatPurgeableDecoder直接把encoded data拷贝到byte array,也就是Java Heap上是一个优势。(至于MemoryFile是如何能够映射数据到ashmem的细节,可以参看前面说的罗升阳说的那边文章。
详细看一下GingerbreadPurgeableDecoder的decode过程:
  1. @Override
  2. public Bitmap decodeFileDescriptor(
  3. FileDescriptor fd,
  4. Rect outPadding,
  5. BitmapFactory.Options opts) {
  6. return hookDecodeFileDescriptor(fd, outPadding, opts);
  7. }
  1. @DoNotStrip
  2. public static Bitmap hookDecodeFileDescriptor(
  3. FileDescriptor fd,
  4. Rect outPadding,
  5. BitmapFactory.Options opts) {
  6. StaticWebpNativeLoader.ensure();
  7. Bitmap bitmap;
  8. boolean isWebp = false;
  9. long originalSeekPosition = nativeSeek(fd, 0, false);
  10. if (originalSeekPosition != -1) {
  11. InputStream inputStream = wrapToMarkSupportedStream(new FileInputStream(fd));
  12. try {
  13. byte[] header = getWebpHeader(inputStream, opts);
  14. isWebp = isWebpHeader(header, 0, HEADER_SIZE);
  15. boolean isWebpSupported = isWebpSupportedByPlatform(header, 0, HEADER_SIZE);
  16. if (isWebp && !isWebpSupported) {
  17. bitmap = nativeDecodeStream(
  18. inputStream,
  19. opts,
  20. getScaleFromOptions(opts),
  21. getInTempStorageFromOptions(opts));
  22. // We send error if the direct decode failed
  23. sendWebpErrorLog("webp_direct_decode_fd", bitmap);
  24. setPaddingDefaultValues(outPadding);
  25. setWebpBitmapOptions(bitmap, opts);
  26. } else {
  27. nativeSeek(fd, originalSeekPosition, true);
  28. bitmap = originalDecodeFileDescriptor(fd, outPadding, opts);
  29. if (bitmap == null && isWebp) {
  30. // Notify that the native decode has failed and that we're trying to decode directly
  31. sendWebpErrorLog("webp_native_decode_fd_fallback", bitmap);
  32. // We fallback into our code for decoding
  33. bitmap = nativeDecodeStream(
  34. new FileInputStream(fd),
  35. opts,
  36. getScaleFromOptions(opts),
  37. getInTempStorageFromOptions(opts));
  38. // Notify that the direct decoder failed after native decoder
  39. sendWebpErrorLog("webp_direct_decode_fd_fallback", bitmap);
  40. setWebpBitmapOptions(bitmap, opts);
  41. }
  42. }
  43. } finally {
  44. try {
  45. inputStream.close();
  46. } catch (Throwable t) {
  47. /* ignore */
  48. }
  49. }
  50. } else {
  51. bitmap = hookDecodeStream(new FileInputStream(fd), outPadding, opts);
  52. if (bitmap == null && isWebp) {
  53. // Notify that the native decoding was wrong
  54. sendWebpErrorLog("webp_native_decode_out_seek_fd_fallback", bitmap);
  55. // We fallback into our code for decoding
  56. bitmap = nativeDecodeStream(
  57. new FileInputStream(fd),
  58. opts,
  59. getScaleFromOptions(opts),
  60. getInTempStorageFromOptions(opts));
  61. // Notify if the direct decoding has failed after native decoder
  62. sendWebpErrorLog("webp_direct_decode_out_seek_fd_fallback", bitmap);
  63. setWebpBitmapOptions(bitmap, opts);
  64. }
  65. setPaddingDefaultValues(outPadding);
  66. }
  67. return bitmap;
  68. }
5.5
hookDecodeFileDescriptor函数,判断如果是webp格式的图片,而且判断当前系统不支持decode该webp图片格式(android系统对多媒体格式支持的可以看https://developer.android.com/guide/topics/media/media-formats.html

不支持的话,就调用自己的nativeDecodeStream方法去解析webp图片,如果系统本身就支持本webp图片,那么直接调用originalDecodeFileDescriptor方法:
  1. @DoNotStrip
  2. private static Bitmap originalDecodeFileDescriptor(
  3. FileDescriptor fd,
  4. Rect outPadding,
  5. BitmapFactory.Options opts) {
  6. return BitmapFactory.decodeFileDescriptor(fd, outPadding, opts);
  7. }
5.6
直接调用BitmapFactory.decodeFileDescriptor的decode过程,就和上面KitKatPurgeableDecoder的decode过程没有太大差异了,唯一不同就是inInputShareable的处理:
  1. static jobject nativeDecodeFileDescriptor(JNIEnv* env, jobject clazz, jobject fileDescriptor,
  2. jobject padding, jobject bitmapFactoryOptions) {
  3. NPE_CHECK_RETURN_ZERO(env, fileDescriptor);
  4. jint descriptor = jniGetFDFromFileDescriptor(env, fileDescriptor);
  5. struct stat fdStat;
  6. if (fstat(descriptor, &fdStat) == -1) {
  7. doThrowIOE(env, "broken file descriptor");
  8. return nullObjectReturn("fstat return -1");
  9. }
  10. bool isPurgeable = optionsPurgeable(env, bitmapFactoryOptions);
  11. bool isShareable = optionsShareable(env, bitmapFactoryOptions);
  12. bool weOwnTheFD = false;
  13. if (isPurgeable && isShareable) {
  14. int newFD = ::dup(descriptor);
  15. if (-1 != newFD) {
  16. weOwnTheFD = true;
  17. descriptor = newFD;
  18. }
  19. }
  20. SkAutoTUnref<SkData> data(SkData::NewFromFD(descriptor));
  21. if (data.get() == NULL) {
  22. return nullObjectReturn("NewFromFD failed in nativeDecodeFileDescriptor");
  23. }
  24. SkAutoTUnref<SkMemoryStream> stream(new SkMemoryStream(data));
  25. /* Allow purgeable iff we own the FD, i.e., in the puregeable and
  26. shareable case.
  27. */
  28. return doDecode(env, stream, padding, bitmapFactoryOptions, weOwnTheFD);
  29. }

只有isPurgeableisShareable都为true时,才能去dup fd,不然就是引用旧的fd。

至于调用Fresco的nativeDecodeStream方法去解析webp图片的过程,这里就不在赘述了,感兴趣可以自行分析(依然是把pixels放到ashmem中去,通过BitmapCreator

ArtDecoder
最后简单的看看ArtDecoder,该decoder没有使用什么“trick”去做一些黑科技,直接使用BitmapFactory的decode方法:
  1. protected CloseableReference<Bitmap> decodeStaticImageFromStream(
  2. InputStream inputStream,
  3. BitmapFactory.Options options) {
  4. Preconditions.checkNotNull(inputStream);
  5. int sizeInBytes = BitmapUtil.getSizeInByteForBitmap(
  6. options.outWidth,
  7. options.outHeight,
  8. options.inPreferredConfig);
  9. final Bitmap bitmapToReuse = mBitmapPool.get(sizeInBytes);
  10. if (bitmapToReuse == null) {
  11. throw new NullPointerException("BitmapPool.get returned null");
  12. }
  13. options.inBitmap = bitmapToReuse;
  14. Bitmap decodedBitmap;
  15. ByteBuffer byteBuffer = mDecodeBuffers.acquire();
  16. if (byteBuffer == null) {
  17. byteBuffer = ByteBuffer.allocate(DECODE_BUFFER_SIZE);
  18. }
  19. try {
  20. options.inTempStorage = byteBuffer.array();
  21. decodedBitmap = BitmapFactory.decodeStream(inputStream, null, options);
  22. } catch (RuntimeException re) {
  23. mBitmapPool.release(bitmapToReuse);
  24. throw re;
  25. } finally {
  26. mDecodeBuffers.release(byteBuffer);
  27. }
  28. if (bitmapToReuse != decodedBitmap) {
  29. mBitmapPool.release(bitmapToReuse);
  30. decodedBitmap.recycle();
  31. throw new IllegalStateException();
  32. }
  33. return CloseableReference.of(decodedBitmap, mBitmapPool);
  34. }
  1. /**
  2. * Options returned by this method are configured with mDecodeBuffer which is GuardedBy("this")
  3. */
  4. private static BitmapFactory.Options getDecodeOptionsForStream(
  5. EncodedImage encodedImage,
  6. Bitmap.Config bitmapConfig) {
  7. final BitmapFactory.Options options = new BitmapFactory.Options();
  8. // Sample size should ONLY be different than 1 when downsampling is enabled in the pipeline
  9. options.inSampleSize = encodedImage.getSampleSize();
  10. options.inJustDecodeBounds = true;
  11. // fill outWidth and outHeight
  12. BitmapFactory.decodeStream(encodedImage.getInputStream(), null, options);
  13. if (options.outWidth == -1 || options.outHeight == -1) {
  14. throw new IllegalArgumentException();
  15. }
  16. options.inJustDecodeBounds = false;
  17. options.inDither = true;
  18. options.inPreferredConfig = bitmapConfig;
  19. options.inMutable = true;
  20. return options;
  21. }
6.1
使用了BitmapOptions的inBitmap和inTempStorage去优化内存使用。inBitmap是由上层的BitmapPool去分配内存,inTempStorage是由SynchronizedPool分配内存,都是用缓存池的方式分配和回收内存,做到对这些区域的内存可管理,减少各个不同地方自行分配内存

Unpin
既然存在“pin”操作,自然就应该会有unpin的操作,因为pin操作导致了内存区域的引用计数增加,只有通过unpin操作,才能把相应的引用计数减少,保证内存能够被回收,不然就会导致oom。

Fresco中确实提供了这样的方法,进行“unpin”操作:
  1. public static void releaseByteBuffer(Bitmap bitmap) {
  2. Preconditions.checkNotNull(bitmap);
  3. nativeReleaseByteBuffer(bitmap);
  4. }
6.2
对应JNI的方法就是:
  1. static void Bitmaps_releaseByteBuffer(
  2. JNIEnv* env,
  3. jclass clazz,
  4. jobject bitmap) {
  5. UNUSED(clazz);
  6. int rc = AndroidBitmap_unlockPixels(env, bitmap);
  7. if (rc != ANDROID_BITMAP_RESULT_SUCCESS) {
  8. safe_throw_exception(env, "Failed to unlock Bitmap pixels");
  9. }
  10. }
6.3
其实就是AndroidBitmap_unlockPixels
  1. int AndroidBitmap_unlockPixels(JNIEnv* env, jobject jbitmap) {
  2. if (NULL == env || NULL == jbitmap) {
  3. return ANDROID_BITMAP_RESULT_BAD_PARAMETER;
  4. }
  5. SkBitmap* bm = GraphicsJNI::getNativeBitmap(env, jbitmap);
  6. if (NULL == bm) {
  7. return ANDROID_BITMAP_RESULT_JNI_EXCEPTION;
  8. }
  9. // notifyPixelsChanged() needs be called to apply writes to GL-backed
  10. // bitmaps. Note that this will slow down read-only accesses to the
  11. // bitmaps, but the NDK methods are primarily intended to be used for
  12. // writes.
  13. bm->notifyPixelsChanged();
  14. bm->unlockPixels();
  15. return ANDROID_BITMAP_RESULT_SUCCESS;
  16. }
  1. void SkBitmap::unlockPixels() const {
  2. SkASSERT(NULL == fPixelRef || fPixelLockCount > 0);
  3. if (NULL != fPixelRef && 1 == sk_atomic_dec(&fPixelLockCount)) {
  4. fPixelRef->unlockPixels();
  5. this->updatePixelsFromRef();
  6. }
  7. SkDEBUGCODE(this->validate();)
  8. }
  1. void SkPixelRef::unlockPixels() {
  2. SkASSERT(!fPreLocked || SKPIXELREF_PRELOCKED_LOCKCOUNT == fLockCount);
  3. if (!fPreLocked) {
  4. SkAutoMutexAcquire ac(*fMutex);
  5. SkASSERT(fLockCount > 0);
  6. if (0 == --fLockCount) {
  7. this->onUnlockPixels();
  8. fPixels = NULL;
  9. fColorTable = NULL;
  10. }
  11. }
  12. }
6.4
如果是SkPixelRef是SkImageRef_ashmem,则:
  1. void SkImageRef_ashmem::onUnlockPixels() {
  2. this->INHERITED::onUnlockPixels();
  3. if (-1 != fRec.fFD) {
  4. SkASSERT(fRec.fAddr);
  5. SkASSERT(fRec.fPinned);
  6. //回收bitmap在ashmem的内存
  7. ashmem_unpin_region(fRec.fFD, 0, 0);
  8. fRec.fPinned = false;
  9. }
  10. // we clear this with or without an error, since we've either closed or
  11. // unpinned the region
  12. //设置pixels为NULL
  13. fBitmap.setPixels(NULL, NULL);
  14. }
6.5
可见,调用releaseByteBuffer用就能够回收掉bitmap占用的内存,但前提是没有别的地方对这块内存进行过lock操作,因为lock操作会导致引用计数增加。但是,奇怪的是,在Fresco中,没有任何一个地方会调用该方法,那不会导致OOM吗?
找了很久,终于发现,我们不一定需要执行unpin操作进行内存回收,直接对Bitmap调用recycle方法也是可以的,这也是我们最常用的回收Bitmap的方法,而且是系统提供的API。看Bitmap的recycle方法:
  1. /**
  2. * Free the native object associated with this bitmap, and clear the
  3. * reference to the pixel data. This will not free the pixel data synchronously;
  4. * it simply allows it to be garbage collected if there are no other references.
  5. * The bitmap is marked as "dead", meaning it will throw an exception if
  6. * getPixels() or setPixels() is called, and will draw nothing. This operation
  7. * cannot be reversed, so it should only be called if you are sure there are no
  8. * further uses for the bitmap. This is an advanced call, and normally need
  9. * not be called, since the normal GC process will free up this memory when
  10. * there are no more references to this bitmap.
  11. */
  12. public void recycle() {
  13. if (!mRecycled) {
  14. if (nativeRecycle(mNativeBitmap)) {
  15. // return value indicates whether native pixel object was actually recycled.
  16. // false indicates that it is still in use at the native level and these
  17. // objects should not be collected now. They will be collected later when the
  18. // Bitmap itself is collected.
  19. mBuffer = null;
  20. mNinePatchChunk = null;
  21. }
  22. mRecycled = true;
  23. }
  24. }
6.6
nativeRecycle对应的JNI方法是:
  1. static jboolean Bitmap_recycle(JNIEnv* env, jobject, SkBitmap* bitmap) {
  2. #ifdef USE_OPENGL_RENDERER
  3. if (android::uirenderer::Caches::hasInstance()) {
  4. return android::uirenderer::Caches::getInstance().resourceCache.recycle(bitmap);
  5. }
  6. #endif // USE_OPENGL_RENDERER
  7. bitmap->setPixels(NULL, NULL);
  8. return true;
  9. }
  1. void SkBitmap::setPixels(void* p, SkColorTable* ctable) {
  2. if (NULL == p) {
  3. this->setPixelRef(NULL, 0);
  4. return;
  5. }
  6. Sk64 size = this->getSize64();
  7. SkASSERT(!size.isNeg() && size.is32());
  8. this->setPixelRef(new SkMallocPixelRef(p, size.get32(), ctable, false))->unref();
  9. // since we're already allocated, we lockPixels right away
  10. this->lockPixels();
  11. SkDEBUGCODE(this->validate();)
  12. }
  1. SkPixelRef* SkBitmap::setPixelRef(SkPixelRef* pr, size_t offset) {
  2. // do this first, we that we never have a non-zero offset with a null ref
  3. if (NULL == pr) {
  4. offset = 0;
  5. }
  6. if (fPixelRef != pr || fPixelRefOffset != offset) {
  7. if (fPixelRef != pr) {
  8. this->freePixels();
  9. SkASSERT(NULL == fPixelRef);
  10. SkSafeRef(pr);
  11. fPixelRef = pr;
  12. }
  13. fPixelRefOffset = offset;
  14. this->updatePixelsFromRef();
  15. }
  16. SkDEBUGCODE(this->validate();)
  17. return pr;
  18. }
  1. void SkBitmap::freePixels() {
  2. // if we're gonna free the pixels, we certainly need to free the mipmap
  3. this->freeMipMap();
  4. if (fColorTable) {
  5. fColorTable->unref();
  6. fColorTable = NULL;
  7. }
  8. if (NULL != fPixelRef) {
  9. if (fPixelLockCount > 0) {
  10. fPixelRef->unlockPixels();
  11. }
  12. fPixelRef->unref();
  13. fPixelRef = NULL;
  14. fPixelRefOffset = 0;
  15. }
  16. fPixelLockCount = 0;
  17. fPixels = NULL;
  18. }
6.7
调用recycle方法,不管是引用计数的多少,直接把pixels回收掉,引用计数也置为0。所以Fresco是用recycle的方法进行bitmap的内存回收:

  1. public BitmapCounter(int maxCount, int maxSize) {
  2. Preconditions.checkArgument(maxCount > 0);
  3. Preconditions.checkArgument(maxSize > 0);
  4. mMaxCount = maxCount;
  5. mMaxSize = maxSize;
  6. mUnpooledBitmapsReleaser = new ResourceReleaser<Bitmap>() {
  7. @Override
  8. public void release(Bitmap value) {
  9. try {
  10. decrease(value);
  11. } finally {
  12. value.recycle();
  13. }
  14. }
  15. };
  16. }
6.8
结束语
到此为止,Fresco对bitmap decode的内存分配问题已经算是比较深入分析了,但是还留待一点思考的空间:
  1. Art模式下,究竟对GC做了什么优化的地方,不会导致stop-the-world的问题呢?
  2. Fresco的开发人员是有多了解Bitmap的decode和android平台特性,才能够找到以isPurgeable为切入点,让Bitmap的pixels分配到ashmem上,并知道drop frames问题并提出解决方法,OMG。
展开阅读全文

没有更多推荐了,返回首页