Fresco介绍：Android的一个新图片库

翻译自：https://code.facebook.com/posts/366199913563917

快速有效的展示图片对Facebook Android客户端非常重要。可是该团队多年来在有效存储图片时遇到了很多问题。图片很大，可是设备却很小。每个像素需要占用4字节的数据----red,green,blue和alpha值各占一字节。如果手机屏幕的尺寸是480*800的话，一张全屏的图片会占用1.5M的内存。而手机的内存是有限的，Android设备给它的众多应用程序分配各自的内存空间。在有些设备中，Facebook应用程序只被分配了16M的内存，这样的情况下，一张图片就几乎占据了十分之一的内存。

当应用程序使用完内存会发生什么情况呢？它会崩溃。因此，Facebook团队创建了Fresco库----它负责管理图片及其使用的内存。

Android系统的内存区域

首先，我们需要了解Android系统提供的不同内存区域。

1）Java heap: 该区域被设备开发商对每个应用程序设定了严格的空间限制。所有使用Java语言的new操作符创建的对象都保存在这里。此处的内存会被Java的垃圾回收机制处理，即当App不再使用该区域的某块内存时，系统会自动回收它。

不幸的是，垃圾回收过程本身也是一个问题。当回收内存较多时，Android系统会完全停止应用程序以便执行回收。这也是应用程序冻屏或卡顿的最常见的应用。这些很影响用户的体验，他们此时可能正在滑动或点击按钮，却只能等待应用程序做出反应。

2）Native heap： 使用C++语音的new操作符创建的对象放在这个区域。在该区域App只受设备的物理内存的限制。并且没有垃圾回收机制，不用担心应用程序卡顿的问题。但是，C++应用程序需要负责释放分配的所有内存，否则会引起内存泄漏，最终导致应用程序崩溃。

3）ashmem:也叫匿名共享内存。它的行为与native heap很像，但是多了额外的系统调用。Android系统可以“unpin"该内存而不用释放它。只是一种懒惰模式的释放，内存只有在系统趋势需要更多的内存时才真正被释放。当Android系统"pin"该内存后，如果该内存没有被释放过的话，旧的数据仍然存在。

可清除的位图（Purgeable bitmaps)

一般情况下，Java应用程序不能够直接对Ashmem区域进行操作，但是有些情况例外，图片就是其一。当插件一个解压后的图片（即未被压缩的图片）时，即Bitmap， Android API提供了purgeable选项：

BitmapFactory.Options = new BitmapFactory.Options();
options.inPurgeable = true;Bitmap bitmap = BitmapFactory.decodeByteArray(jpeg, 0, jpeg.length, options);

设置该选项的图片保存在ashmem内存区。因此，垃圾回收机制不会自动回收它们。Android的系统库在绘制系统需要渲染该图片时，对该图片的内存区域执行"pin"操作，结束后执行”unpin". 被unpin过的内存会随时被系统回收。当unpin过的图片需要重绘时，系统会即时重新解压该图片。

这似乎是一个很完美的解决方案，但问题是即时的解压发生在UI线程中。解压是一个很耗费CUP的操作，会引起UI卡顿。基于上述原因，Google现在不再建议使用这一特性，而推荐使用另一个选项：inBitmap. 该选项的问题是Android3.0之后才出现。并且在Android 4.4之前只在图片的大小相同时才起作用。Fresco就是解决这个问题的，该库甚至可以在Android 2.3系统上使用。

Fresco的解决方法

找到了一种解决上述问题----快速的UI和快速的内存的方法。如果提前在非UI线程中pin内存，并保证不会被unpin的话,这样既能把图片保存在ashmem中又不会引起UI卡顿。Fresco团队发现Android Native Development Kit (NDK)中的函数AndroidBitmap_lockPixels实现了pin功能，但是Android系统中使用该函数后会使用unlockPixels将内存unpin.Fresco的突破是调用lockPixels却不使用配对的unlockPixels, 这样创建的图片保存在Java heap之外并且不会影响UI线程的速度。

用Java写代码时，像C++一样考虑内存的释放问题

使用pin操作过的可清除位图既不会被垃圾回收又不享受ashmem内置的回收机制以防止内存泄漏。它完全依靠程序开发人员来维护。

在C++中，常用的解决方案是使用智能指针来实现引用计数。它利用了C++语言的各种机制---拷贝构造函数，assignment操作符，及析构函数。这些在Java语言中都不存在，因此我们需要找出一种方法使得使用Java语言也可以得到C++语言类似的保证。

Fresco使用了两个类来实现上述方法。一个是SharedReference类， 该类有两个方法：addReference和deleteReference, 调用者在使用被引用的对象或不再使用该对象时必须调用这两个方法。当引用数为0时，被引用的对象被回收（如使用Bitmap.recycle).
很明显，java开发人员很容易忘记调用上述方法，因为避免开发人员自己管理对象的释放是java的一大特性。因此，在SharedReference类上，Fresco又创建了CloseableReference类，它不仅实现了java的Closeable接口，还实现了Cloneable接口。构造函数和clone()方法调用addReference()，而close()方法调用deleteReference()。 因此Java开发者只需要准从如下两个简单的规则：
   1.在将一个CloseableReference对象赋值给新对象时，使用.clone()。
   2.在退出对象的作用域之前，调用.close(), 通常在finally模块中使用。
上述规则既有效的防止了内存泄漏，又让我们在大型java应用程序中享受native内存管理。

Fresco不仅仅是个图片加载器，而是管道

将图片展示到移动设备需要如下步骤：

很多已知的开源库也采用这样的步骤，如Picasso, Universal Image Loader, Glide,Volley等。所有这些库对Android开发做出了重要的贡献，而Fresco在一些重要的方面走得更远。

将这些步骤看作管道而不是加载器就是一个重要的不同点。每一个步骤都尽可能地与其他步骤独立，每个步骤都接收一个输入和一些参数，然后产出一个输出。它使得并行做这些操作成为可能，而不是像其他库那样只能顺序操作。有些操作只在特定的条件下执行，有些操作可以对执行的线程做一些特殊的要求。很多用户在网上很慢的情况下使用Facebook，这时候我们希望用户能尽可能快的看到图片，甚至在图片还没有完全下载完之前。

停止担心，享受流的概念

Java中的异步代码通常通过类似 Future的机制来执行。代码被提交到另外一个线程执行，使用一个类似Future的对象去确认结果是否已经准备好了。该操作的一个缺陷是假设只有一个结果。当处理渐进性图片时，我们需要一系列正在处理的结果。

Fresco的解决方案是使用Future的更通用的版本，DataSource类。它提供了一个订阅方法，在该方法中，调用者必须传递一个DataSubscriber对象和一个Executor对象。该DataSubscriber对象接收来自DataSource的所有中间结果及最后的结果，并提供一个简单的方法区分它们。因为我们会频繁使用需要显示close调用的对象，因此DataSource本身是一个Closeable对象。

在后台，使用了新的叫做Producer/Consumer的框架来实现上述方案。它是从ReactiveX框架获取的灵感，使用与RxJava类似的接口，但是更适合移动端并且含有内在的Closeables支持。

接口很简单，Producer类只有一个方法produceResults，该方法使用Consumer对象为参数。对应地， Consumer 类有一个onNewResult方法。

就像Java的输入输出流一样，我们可以将多个生产者串联起来做成一个生产链。假设我们有一个生产者，它的任务是将I类型转换为O类型。可以通过类似下图的方法来生成生产链：

public class OutputProducer<I, O> implements Producer<O> {

  private final Producer<I> mInputProducer;

  public OutputProducer(Producer<I> inputProducer) {
    this.mInputProducer = inputProducer;
  }

  public void produceResults(Consumer<O> outputConsumer, ProducerContext context) {
    Consumer<I> inputConsumer = new InputConsumer(outputConsumer);
    mInputProducer.produceResults(inputConsumer, context);
  }

  private static class InputConsumer implements Consumer<I> {
    private final Consumer<O> mOutputConsumer;

    public InputConsumer(Consumer<O> outputConsumer) {
      mOutputConsumer = outputConsumer;
    }

    public void onNewResult(I newResult, boolean isLast) {
      O output = doActualWork(newResult);
      mOutputConsumer.onNewResult(output, isLast);      
    }
  }}

这使得我们可以把一系列的步骤链接起来，而同时却保持它们的逻辑独立性。

动画——从一到多

Stickers, 一种使用GIF和WebP格式保存的动画，深受Facebook用户的喜爱。在移动端支持该动画会引起新的挑战。动画不仅是一个位图而是一系列的位图，每张都需要解码，保存在内存中，然后展现。将每一帧动画都保存在内存中对大型动画来说是不现实的。

Fresco创建了AnimatedDrawable类，具备渲染动画能力的图片类，及两个支持类---一个支持GIF，另一个支持WebP。AnimatedDrawable类实现了标准的AndroidAnimatable接口，因此调用者可以随时启动和终止动画。为优化内存使用，只在所有帧都足够小时才全部缓存，如果太大的话，就即时解码。该行为对调用者是完全可行的。

两个支持类是使用C++编码实现的。我们只保存解码后的数据和解析后的元数据（如宽度和高度等）的一份拷贝，并保存对数据的引用计数，以便在Java端允许多个Drawable对象同时读取同一个WebP图片。

Drawee——图片的展示

当图片正在从网络上下载时，我们希望现实一个占位图。如果下载失败的话，显示一个错误指示图。当图片下载完成后，我们做一个快速的fade-in动画。

我们经常需要放大图片，或甚至应用一个展示矩阵去使用硬件加速以固定尺寸来渲染图片。我们并不是每次都以图片中心点为焦点来进行放大，焦点可以在任何地方。有时我们需要圆角的图片，甚至圆形的图片。所有这些操作都需要快速平滑。

最开始的实现是使用Android的View对象——当图片下载完成后，替换占位页面为一个ImageView。该方案被证明速度非常慢。改变页面会促使Android系统执行整个的布局过程（layout pass)，而这是用户滑动时最不希望看到的。可行的方案是使用Android的图片类（Drawable)，可以被快速的替换。

因此Fresco创建了Drawee，它是一个MVC风格的展示图片的框架。使用DraweeHierarchy类来实现该框架，该类实现了图片的分层制度，每层对应一个具体的功能——对原始图片进行展示，分层，淡入或放大等功能。

DraweeControllers类连接图片管道（或任何图片加载器）并负责后台的图片操作。他们从管道中接收事件并决定如何处理它们。他们控制DraweeHierachy的展示——是占位图，还是错误指示图，或者下载完成的图片。

DraweeViews 类只提供有些的功能，但它提供的功能却是决定性的。它监听Android系统中页面是否还在显示的事件。当页面不再显示时通知DraweeController关闭图片使用的资源。这避免了内存泄漏。另外，上述controller会通知页面管道取消网络请求，如果该请求还没有完成的话。因此，滑动很长的图片列表，不会引起网络阻塞。

上述机制解决了展示图片的大部分工作。调用代码只需要创建一个DraweeView对象，指定URI，并且选择性的添加也许参数即可。开发者不需要担心如何管理图片内存或更新图片的问题。所有这些事情都由Fresco库做了处理。

原文如下，写得非常好，提供给愿意看原文的：

Displaying images quickly and efficiently on Facebook for Android is important. Yet we have had many problems storing images effectively over the years. Images are large, but devices are small. Each pixel takes up 4 bytes of data — one for each of red, green, blue, and alpha. If a phone has a screen size of 480 x 800 pixels, a single full-screen image will take up 1.5 MB of memory. Phones often have very little memory, and Android devices divide up what memory they have among multiple apps. On some devices, the Facebook app is given as little as 16 MB — and just one image could take up a tenth of that!

What happens when your app runs out of memory? It crashes. We set out to solve this by creating a library we're calling Fresco — it manages images and the memory they use. Crashes begone.

Regions of memory

To understand what Facebook did here, we need to understand the different heaps of memory available on Android.

The Java heap is the one subject to the strict, per-application limits set by the device manufacturer. All objects created using the Java language's new operator go here. This is a relatively safe area of memory to use. Memory is garbage-collected, so when the app has finished with memory, the system will automatically reclaim it.

Unfortunately, this process of garbage collection is precisely the problem. To do more than basic reclamations of memory, Android must halt the application completely while it carries out the garbage collection. This is one of the most common causes of an app appearing to freeze or stall briefly while you are using it. It's frustrating for people using the app, and they may try to scroll or press a button — only to see the app wait inexplicably before responding.

In contrast, the native heap is the one used by the C++ new operator. There is much more memory available here. The app is limited only by the physical memory available on the device. There is no garbage collection and nothing to slow things down. However, C++ programs are responsible for freeing every byte of memory they allocate, or they will leak memory and eventually crash.

Android has another region of memory, called ashmem. This operates much like the native heap, but has additional system calls. Android can “unpin” the memory rather than freeing it. This is a lazy free; the memory is freed only if the system actually needs more memory. When Android “pins” the memory back, old data will still be there if it hasn't been freed.

Purgeable bitmaps

Ashmem is not directly accessible to Java applications, but there are a few exceptions, and images are one of them. When you create a decoded (uncompressed) image, known as a bitmap, the Android API allows you to specify that the image be purgeable:

BitmapFactory.Options = new BitmapFactory.Options();
options.inPurgeable = true;Bitmap bitmap = BitmapFactory.decodeByteArray(jpeg, 0, jpeg.length, options);

Purgeable bitmaps live in ashmem. However, the garbage collector does not automatically reclaim them. Android's system libraries “pin” the memory when the draw system is rendering the image, and “unpin” it when it's finished. Memory that is unpinned can be reclaimed by the system at any time. If an unpinned image ever needs to be drawn again, the system will just decode it again, on the fly.

This might seem like a perfect solution, but the problem is that the on-the-fly decode happens on the UI thread. Decoding is a CPU-intensive operation, and the UI will stall while it is being carried out. For this reason, Google now advises against using the feature. They now recommend using a different flag, inBitmap. However, this flag did not exist until Android 3.0. Even then, it was not useful unless most of the images in the app were the same size, which definitely isn't the case for Facebook. It was not until Android 4.4 that this limitation was removed. However, we needed a solution that would work for everyone using Facebook, including those running Android 2.3.

Having our cake and eating it too

We found a solution that allows us to have the best of both worlds — both a fast UI and fast memory. If we pinned the memory in advance, off the UI thread, and made sure it was never unpinned, then we could keep the images in ashmem but not suffer the UI stalls. As luck would have it, the Android Native Development Kit (NDK) has a function that does precisely this, calledAndroidBitmap_lockPixels. The function was originally intended to be followed by a call tounlockPixels to unpin the memory again.

Our breakthrough came when we realized we didn't have to do that. If we called lockPixels without a matching unlockPixels, we created an image that lived safely off the Java heap and yet never slowed down the UI thread. A few lines of C++ code, and we were home free.

Write code in Java, but think like C++

As we learned from Spider-Man, “With great power comes great responsibility.” Pinned purgeable bitmaps have neither the garbage collector's nor ashmem's built-in purging facility to protect them from memory leaks. We are truly on our own.

In C++, the usual solution is to build smart pointer classes that implement reference counting. These make use of C++ language facilities — copy constructors, assignment operators, and deterministic destructors. This syntactic sugar does not exist in Java, where the garbage collector is assumed to be able to take care of everything. So we have to somehow find a way to implement C++-style guarantees in Java.

We made use of two classes to do this. One is simply called SharedReference. This has two methods, addReference and deleteReference, which callers must call whenever they take the underlying object or let it out of scope. Once the reference count goes to zero, resource disposal (such as Bitmap.recycle) takes place.

Yet, obviously, it would be highly error-prone to require Java developers to call these methods. Java was chosen as a language to avoid doing this! So on top of SharedReference, we builtCloseableReference. This implements not only the Java Closeable interface, but Cloneable as well. The constructor and the clone() method call addReference(), and the close() method callsdeleteReference(). So Java developers need only follow two simple rules:

1. On assigning a CloseableReference to a new object, call .clone().
2. Before going out of scope, call .close(), usually in a finally block.

These rules have been effective in preventing memory leaks, and have let us enjoy native memory management in large Java applications like Facebook for Android and Messenger for Android.

It's more than a loader — it's a pipeline

There are many steps involved in showing an image on a mobile device:

Several excellent open source libraries exist that perform these sequences — Picasso, Universal Image Loader, Glide, and Volley, to name a few. All of these have made important contributions to Android development. We believe our new library goes further in several important ways.

Thinking of the steps as a pipeline rather than as a loader in itself makes a difference. Each step should be as independent of the others as possible, taking an input and some parameters and producing an output. It should be possible to do some operations in parallel, others in serial. Some execute only in specific conditions. Several have particular requirements as to which threads they execute on. Moreover, the entire picture becomes more complex when we consider progressive images. Many people use Facebook over very slow Internet connections. We want these users to be able to see their images as quickly as possible, often even before the image has actually finished downloading.

Stop worrying, love streaming

Asynchronous code on Java has traditionally been executed through mechanisms like Future. Code is submitted for execution on another thread, and an object like a Future can be checked to see if the result is ready. This, however, assumes that there is only one result. When dealing with progressive images, we want there to be an entire series of continuous results.

Our solution was a more generalized version of Future, called DataSource. This offers a subscribe method, to which callers must pass a DataSubscriber and an Executor. The DataSubscriber receives notifications from the DataSource on both intermediate and final results, and offers a simple way to distinguish between them. Because we are so often dealing with objects that require an explicit close call, DataSource itself is a Closeable.

Behind the scenes, each of the boxes above is implemented using a new framework, called Producer/Consumer. Here we drew inspiration from ReactiveX frameworks. Our system has interfaces similar to RxJava, but more appropriate for mobile and with built-in support for Closeables.

The interfaces are kept simple. Producer has a single method, produceResults, which takes aConsumer object. Consumer, in turn, has an onNewResult method.

We use a system like this to chain producers together. Suppose we have a producer whose job is to transform type I to type O. It would look like this:

 
public class OutputProducer<I, O> implements Producer<O> {

  private final Producer<I> mInputProducer;

  public OutputProducer(Producer<I> inputProducer) {
    this.mInputProducer = inputProducer;
  }

  public void produceResults(Consumer<O> outputConsumer, ProducerContext context) {
    Consumer<I> inputConsumer = new InputConsumer(outputConsumer);
    mInputProducer.produceResults(inputConsumer, context);
  }

  private static class InputConsumer implements Consumer<I> {
    private final Consumer<O> mOutputConsumer;

    public InputConsumer(Consumer<O> outputConsumer) {
      mOutputConsumer = outputConsumer;
    }

    public void onNewResult(I newResult, boolean isLast) {
      O output = doActualWork(newResult);
      mOutputConsumer.onNewResult(output, isLast);      
    }
  }}

This lets us chain together a very complex series of steps and still keep them logically independent.

Animations — from one to many

Stickers, which are animations stored in the GIF and WebP formats, are well liked by people who use Facebook. Supporting them poses new challenges. An animation is not one bitmap but a whole series of them, each of which must be decoded, stored in memory, and displayed. Storing every single frame in memory is not tenable for large animations.

We built AnimatedDrawable, a Drawable capable of rendering animations, and two backends for it — one for GIF, the other for WebP. AnimatedDrawable implements the standard Android Animatableinterface, so callers can start and stop the animation whenever they want. To optimize memory usage, we cache all the frames in memory if they are small enough, but if they are too large for that, we decode on the fly. This behavior is fully tunable by the caller.

Both backends are implemented in C++ code. We keep a copy of both the encoded data and parsed metadata, such as width and height. We reference count the data, which allows multiple Drawables on the Java side to access a single WebP image simultaneously.

How do I love thee? Let me Drawee the ways . . .

When images are being downloaded from the network, we want to show a placeholder. If they fail to download, we show an error indicator. When the image does arrive, we do a quick fade-in animation. Often we scale the image, or even apply a display matrix, to render it at a desired size using hardware acceleration. And we don't always scale around the center of the image — the useful focus point may well be elsewhere. Sometimes we want to show the image with rounded corners, or even as a circle. All of these operations need to be fast and smooth.

Our previous implementation involved using Android View objects — swapping out a placeholder View for an ImageView when the time came. This turned out to be quite slow. Changing Views forces Android to execute an entire layout pass, definitely not something you want to happen while users are scrolling. A more sensible approach would be to use Android's Drawables, which can be swapped out on the fly.

So we built Drawee. This is an MVC-like framework for the display of images. The model is calledDraweeHierarchy. It is implemented as a hierarchy of Drawables, each of which applies a specific function — imaging, layering, fade-in, or scaling — to the underlying image.

DraweeControllers connect to the image pipeline — or to any image loader — and take care of backend image manipulation. They receive events back from the pipeline and decide how to handle them. They control what the DraweeHierarchy actually displays — whether a placeholder, error condition, or finished image.

DraweeViews have only limited functionality, but what they provide is decisive. They listen for Android system events that signal that the view is no longer being shown on-screen. When going off-screen, the DraweeView can tell the DraweeController to close the resources used by the image. This avoids memory leaks. In addition, the controller will tell the image pipeline to cancel the network request, if it hasn't gone out yet. Thus, scrolling through a long list of images, as Facebook often does, will not break the network bank.

With these facilities, the hard work of displaying images is gone. Calling code need only instantiate a DraweeView, specify a URI, and, optionally, name some other parameters. Everything else happens automatically. Developers don't need to worry about managing image memory or streaming the updates to the image. Everything is done for them by the libraries.

Fresco

Having built this elaborate tool set for image display and manipulation, we wanted to share it with the Android developer community. We are pleased to announce that, as of today, this project is now available as open source.

A  fresco is a painting technique that has been popular around the world for centuries. With this name, we honor the many great artists who have used this form, from Italian Renaissance masters like Raphael to the Sigiriya artists of Sri Lanka. We don't pretend to be on that level. We do hope that Android app developers enjoy using our library as much as we've enjoyed building it.