Android图形子系统详解

Android的图形系统发展经历了通过CPU模拟图形操作，和GPU专门进行图形处理的阶段，因为这方面硬件技术发展很快，因此Android图形系统也在不断调整，以期提供更加快速流畅的UI体验。

图形操作可以有两种方式实现：一是利用通用CPU模拟图形操作；二是利用GPU专门做图形操作。前者会增加CPU的负担，在现在高分辨率已经是普遍现象的时候，让通用处理器来完成大量的图形计算已经不现实。Android图形系统的发展过程也验证了这一观点。

为了达到高效的图形处理效果，是必须紧密结合软件和硬件的。这篇文章主要介绍跟Android的图形子系统。以后可能会对这些主题进行更加深入的探讨。

Android图形系统的软件构成

下面的示意图，展示了Android上负责图形处理的软件模块。

一个典型Android应用中各个图形系统组件的关系图

GPU:

GPU专门设计用于加速图形操作。GPU不同于CPU，它的一个设计目的就是高度的并行化，并行化是大部分图形计算的共同特征。

Android 刚刚问世的时候，GPU还是可选的，最近发布的版本中，GPU已经是一个必配硬件。如果系统中没有GPU，系统使用的OpenGL ES就包含了libagl和pixelflinger，通过软件实现OpenGL ES协议接口，有时也有硬件支持的CopyBit。但是不幸的是，Android通过软件模拟OpenGL，并不支持OpenGL ES 2.0。现在，Android系统中的不少组件使用了OpenGL ES 2.0，比如HWUI、Renderscript、SurfaceTexture。平板电脑都有很高的分辨率，纯软件的模拟支持并不能保证图形的填充需 求，也就不能为用户提供流畅的UI体验。厂商如果想制造基于ICS或者更高版本Android系统的设备，就必须具有支持OpenGL ES 2.0 的GPU。

Canvas:

画布是应用程序用来绘制Widget或图形等元素的地 方。Froyo和Gingerbread上，画布通过Skia来绘制。Honeycomb及以后的版本，HWUI被加入了进来，提供了GPU加速支持。在 Ice Cream Sandwich及以后的版本上，HWUI缺省用于图形的绘制。

Skia:

Skia是一组2D绘图的API，它完全通过软件实现。由于性能方面的原因，Skia逐渐被HWUI所替代。

HWUI

HWUI 可以使UI组件使用GPU加速。HWUI是在Honeycomb中引入进来的，目的是使交互更加快速，及时响应，流畅。在大分辨率的平板电脑上，通过 Skia来绘制动画，会占用很高的CPU资源，进而拖慢整个系统。HWUI需要支持OpenGL ES 2.0的GPU，不能通过软件模拟。

Renderscript

Renderscript 同样也是Honeycomb引入的新的API，它的设计为了同时解决移植和性能问题。应用程序员用Renderscript（基于C99）编写代码，然后 一个LLVM的交叉编译器把它编译为机器独立的bit code，应用程序员再将其打包到apk中。当用户下载apk时，设备上的编译器（基于LLVM，位于/system/lib/libbcc.so）将 bit code编译为目标机器上的指令。

Renderscript在Froyo和Gingerbread上也存在，但是不是公开的API。只有Android的一些wallpaper使用了它。那时它的实现也非常粗糙，功能有限。

Surface:

一 个Surface对应一个屏幕外缓冲区，应用程序用来渲染窗口内容。一个游戏程序，它可能使用OpenGL在Surface上绘制3D对象，一个普通应用 程序，它可能使用Skia来绘制Widget或者文本，它也可能使用HWUI库来启用GPU加速。从ICS开始，Surface通过一个后端的 SurfaceTexture实现，这就意味着Surface对应的不再是一个缓冲区，而是一个纹理（texture）。

Android平台的图形栈

SurfaceFlinger:

SurfaceFlinger是一个合成器，它管理来自于不同应用的Surface。比如，可能有许多应用同时存在，与此对应的，存在许多独立的Surface需要被渲染。SurfaceFlinger决定屏幕上显示的内容，那些需要被覆盖，进行裁剪。

SurfaceFlinger使用的是OpenGL ES 1.1标准中的函数。为什么呢？如果使用OpenGL ES 2.0，就必须需要支持OpenGL ES 2.0的硬件GPU，这会使系统的启动更加复杂，也会使模拟器的实现更加困难。

HW Composer:

硬件合成器是Honeycomb引入的一个HAL，SurfaceFlinger使用它，利用硬件资源来加速Surface的合成，比如3D GPU和2D的图形引擎。

CopyBit:

CopyBit也是一个HAL。它允许使用特殊硬件来加速一些图形操作，比如复制（blitting）。它设计的初衷是在没有3D GPU的系统上加速软件的渲染过程。CopyBit在ICS中被删除了，因为GPU已经成为一个必备硬件，没有必要专门设计一个加速部件。

Libagl/PixelFlinger:

libagl 是一个通过软件实现了OpenGL ES 1.0和1.1版本API的组件。它使用PixelFlinger来实现OpenGL调用。为了加速使用PixelFlinger的渲染过程，JIT被引 入了进来，称为CodeFling。CodeFling生成机器代码，它急剧加速了许多类型的像素操作。

可以看出，Android的图形系统在不断的调整，目的是为了提供更加快速流畅的UI体验。这就是Android版本中图形相关代码变动很大的原因。

Learning about Android Graphics Subsystem

By Bhanu Chetlapalli
Software Engineer, MIPS Technologies

Graphics Performance is the most important component in defining any user interface (UI) experience. The smoothness of the UI interactions and the crispness of the images are all directly dependent on the graphics capabilities of the underlying platform. Customers expect rapid visual feedback when buttons and keys are pressed.  Having a dedicated graphics engine and an operating system that correctly utilizes that engine will significantly reduce the workload on the CPU, vastly improving user experience.

Here at MIPS we support many open source and commercial operating systems with versions that are optimized to take full advantage of the MIPS processor architecture.  Android is one of those open source operating system that we have put significant work into porting to our architecture.  Early in the life of Android many cell phone companies shipped “bare bones” products running Android.  Many of those products did not have the responsiveness and crisp graphics customers were expecting because the products did not have a complete graphics engine and relied on the CPU to “emulate” graphics operations.  Since then almost all smart phones now have graphics engines that fully support the requirements of Android which takes a large burden off the CPU.

Moving onto the specifics of efficient graphics processing, the hardware and software have to work in tandem to enable this. This post aims to briefly explain a few related topics so that engineers can have a better understanding of Android’s Graphics subsystem. I will cover some of these topics in more detail in my future posts.

The Android Graphics Software Components

The following chart shows the software components in the Android operating system responsible for graphics.  I will go through a discussion of most of these and provide some insight into their purpose.

Role of GPUs:

A Graphics Processing Unit (GPU) is a specialized hardware engine which greatly accelerates graphics operations. They are different from CPUs in that they are designed to do highly parallel work – which is typical of most graphics operations.

When Android first started, GPUs were somewhat optional, but with newer releases, GPUs became a mandatory requirement. Systems without GPUs use a software OpenGL ES stack consisting of libagl & pixelflinger, sometimes with hardware support from CopyBIT. Unfortunately software emulation of OpenGL on Android does not support the OpenGL ES 2.0 standard.  Today large parts of the Android operating system use OpenGL ES 2.0 through components like HWUI, Renderscript, SurfaceTextures, etc.  Tablets devices all have high resolution and software simply cannot keep up with the fill rates required to provide a smooth user experience.  Manufacturers who plan to launch new devices that support Android Ice Cream Sandwich (ICS) or later releases must have an OpenGL ES 2.0 GPU.

Canvas:

Canvas is the Android class which application developers would use to draw widgets, pictures etc. In Android versions Froyo and Gingerbread Canvas would do the drawing using Skia.  Android Honeycomb and onward, HWUI was added as an alternate GPU-accelerated option. Android Ice Cream Sandwich (ICS) and onward, HWUI is the default.

Skia:

Skia is a 2D drawing API which is used by applications and works completely in software.  For performance reasons Skia is slowly being replaced by HWUI.

HWUI

The HWUI library enables UI components to be accelerated using the GPU. HWUI was introduced in Honeycomb to make the user interface fast, responsive and smooth.  Since tablets had very high resolutions, using Skia for effects like animation would have been too CPU intensive and slow. HWUI requires an OpenGL ES 2.0 compliant GPU which cannot be emulated by software on Android.

Renderscript

Renderscript was a new API introduced in Honeycomb* to address portability and performance at the same time. The application developer writes the code in the Renderscript language (which is based on C99), and an LLVM cross compiler on the host converts it to machine-independent IR called bit code, the developer then packages the bit code files into the Android application (APK).  When the user downloads the APK, an on-device compiler (LLVM based, located in /system/lib/libbcc.so) compiles it from bit code to the actual machine instructions for the target platform.

* Renderscript was available in Froyo & Gingerbread but was not a published API. Only a few wallpapers in the Android source used it. Its implementation was also fairly trivial & constrained.

Surface:

A Surface in Android corresponds to an off screen buffer into which an application renders the contents. An application might be a game which uses OpenGL to draw 3D objects directly into a surface or a normal application which uses Skia to draw widgets, text, etc. It could even use HWUI library to enable a GPU accelerated user interface. From Android ICS, surfaces are implemented with a SurfaceTexture backend which means instead of a buffer, a texture is used.

SurfaceFlinger:

SurfaceFlinger is the compositor that manages multiple surfaces from various apps. For example, there could be many running applications with independent Surfaces that are being rendered. SurfaceFlinger determines what will be shown on the screen and does overlays as needed.

SurfaceFlinger uses only the functions in the OpenGL ES 1.1 standard.  If SurfaceFlinger used functions in OpenGL ES 2.0, it would require GPU hardware and drivers to be working – which would complicate bring-up on new devices and the Android emulator.

HW Composer:

HW Composer is a Hardware Abstraction Layer (HAL) introduced in Honeycomb that is used by SurfaceFlinger to efficiently perform composition using hardware resources like a 3D GPU or a 2D graphics engine.

CopyBit:

CopyBit was a hardware abstraction layer (HAL) which allowed the use of special hardware that can speed up some graphics operations like blitting etc. It was used to accelerate software rendering on systems without 3D GPUs. CopyBit support was removed in ICS, since GPUs became mandatory for supporting the newer user interface.

Libagl/PixelFlinger:

libagl is the library which implements the OpenGL ES 1.0 & 1.1 API in software. It uses PixelFlinger to implement the OpenGL calls. To speed up the rendering using PixelFlinger, a just in time compiler (JIT) was introduced called CodeFlinger. CodeFlinger generates direct machine code which drastically speeds up many types of pixel operations.

Conclusion

I hope this provided you with a general summary of all the different graphics components in Android.  In my next article I will go into more details about the operation of some of these components.  MIPS continues to lead the industry in highly optimized versions of the Android operating system.  For more information about Android on MIPS go to http://developer.mips.com or http://www.github.com/mips