Android学习之graphic rendering 那些事

原文地址:https://plus.google.com/105051985738280261832/posts/2FXDCz8x93s

这篇文章是从google group中找到的, android的开发组的一员 Dianne Hackborn 对android中的graphic的机制的一个解释及释疑. 作为曾经接触过多媒体及图形图像处理的我(可悲的是为了生活,妥协的不能再从事这个领域的我)还是觉得自己有必要关注一下android的中多媒体及图形图像的知识, 特此,翻译此篇文章, 以示纪念!

How about some Android graphics true facts?

关于Android graphics的一些真相

I get tired of seeing so much misinformation posted and repeated all over the place about how graphics rendering works on Android. Here is some truth:

我在网上看到太多对于android中的图形宣染的一些错误理解及解释. 这里我阐述一些真正的真相:

• Android has always used some hardware accelerated drawing. Since before 1.0 all window compositing to the display has been done with hardware.

Android总是使用一些硬件来加上绘图. 从android 1.0开始, 从所有的窗口合并处理到显示都由硬件来完成. (译者, 至少在真实的device上是这样, 模拟器上的实现,据我所知从4.0开始才有所改变, 由软件逐步过度到使用pc的图形芯片帮助处理, 具体参看google I/O 2011中关于android tools的section的描述)

• This means that many of the animations you see have always been hardware accelerated: menus being shown, sliding the notification shade, transitions between activities, pop-ups and dialogs showing and hiding, etc.

这意味着你在设备上看到的animation都是由硬件加速的: 显示menu, notification的滑动, activitiy之间的转换, dialog的显示与小时等等.

• Android did historically use software to render the contents of each window. For example in a UI likehttp://www.simplemobilereview.com/wp-content/uploads/2010/12/2-home-menu.png there are four windows: the status bar, the wallpaper, the launcher on top of the wallpaper, and the menu. If one of the windows updates its contents, such as highlighting a menu item, then (prior to 3.0) software is used to draw the new contents of that window; however none of the other windows are redrawn at all, and the re-composition of the windows is done in hardware. Likewise, any movement of the windows such as the menu going up and down is all hardware rendering.

历史上使用软件来宣染每个window中的contents. 例如, 在如http://www.simplemobilereview.com/wp-content/uploads/2010/12/2-home-menu.png上的UI, 包含4个window: status bar, wallpaper, 在wallpaper之上的launcher和menu. 如果其中一个window更新了它的contents, 比如高亮了一个menu item, 3.0之前都会用软件来描画这个window的新的content; 而其它的window都不会被重画, 这个window的重新组装是被硬件完成. 同样的, windows的移动,如menu的显示与隐藏都是由硬件来宣染的.

• Looking at drawing inside of a window, you don’t necessarily need to do this in hardware to achieve full 60fps rendering. This depends very much on the number of pixels in your display and the speed of your CPU. For example, Nexus S has no trouble doing 60fps rendering of all the normal stuff you see in the Android UI like scrolling lists on its 800x480 screen. The original Droid however struggled with a similar screen resolution.

来看看描化window内部的情况, 你不必依靠硬件来达到60fps的宣染要求. 这依赖于你所显示的界面上的pixels数及CPU的速度. 例如, 在Nexus S的800*400的屏幕上显示常见的Android UI是没有问题的, 而最初的Droid在这样的分辨率下就不能胜任了.

• "Full" hardware accelerated drawing within a window was added in Android 3.0. The implementation in Android 4.0 is not any more full than in 3.0. Starting with 3.0, if you set the flag in your app saying that hardware accelerated drawing is allowed, then all drawing to the application’s windows will be done with the GPU. The main change in this regard in Android 4.0 is that now apps that are explicitly targeting 4.0 or higher will have acceleration enabled by default rather than having to put android:handwareAccelerated="true" in their manifest. (And the reason this isn’t just turned on for all existing applications is that some types of drawing operations can’t be supported well in hardware and it also impacts the behavior when an application asks to have a part of its UI updated. Forcing hardware accelerated drawing upon existing apps will break a significant number of them, from subtly to significantly.)

全硬件加速宣染是在Android 3.0才加入的. 4.0的实现与3.0相比没有做更多的事情. 从3.0开始, 在你的app中声明允许硬件加速宣染, 所有应用程序的windows的宣染都会由GPU完成. 4.0中最主要的变化是:从4.0之后硬件宣染的选项默认是打开的, 这意味着你不需要在manifest中申明android:handwareAccelerated="true". (这么改动的原因不止是默认的为所有现存的app打开这个选项, 这里的app中有一些描化动作是不能很好的由硬件支持, 它还影响一个应用程序要求部分更新它的UI. 在现存的apps上强制硬件加速宣染, 可能导致或轻微或严重的问题.)

• Hardware accelerated drawing is not all full of win. For example on the PVR drivers of devices like the Nexus S and Galaxy Nexus, simply starting to use OpenGL in a process eats about 8MB of RAM. Given that our process overhead is about 2MB, this is pretty huge. That RAM takes away from other things, such as the number of background processes that can be kept running, potentially slowing down things like app switching.

硬件加速描画也有它的不足. 例如, 在以PVR 驱动的设备, 如Nexus S/ Galaxy Nexus, 简单使用OpenGL会消耗process中8MB的RAM. 考虑到我们的process的消耗大约是2MB, 这个数值就相当的大了. 这8MB RAM会从其它的部分获取, 例如,限制了你所能保持的后台process, 这就降低了app的转换速度.

• Because of the overhead of OpenGL, one may very well not want to use it for drawing. For example some of the work we are doing to make Android 4.0 run well on the Nexus S has involved turning off hardware accelerated drawing in parts of the UI so we don’t lose 8MB of RAM in the system process, another 8MB in the phone process, another 8MB in the system UI process, etc. Trust me, you won’t notice -- there is just no benefit on that device in using OpenGL to draw something like the status bar, even with fancy animations going on in there.

由于OpenGL对系统资源的消耗, 你可能不像用它来描画. 例如, 在4.0中对于Nexus S我们对描绘一些UI的动作关闭了硬件加速, 这样我们就不会损失8MB系统内存, phone process中的8MB内存以及系统UI process中的8MB内存,诸如此类. 相信我, 你不需要注意它--在这样的device上使用OpenGL来描画status bar, 即使在其上有一些很绚的animation, 得不到什么好处.

• Hardware accelerated drawing is not a magical silver bullet to butter-smooth UI. There are many different efforts that have been going on towards this, such as improved scheduling of foreground vs. background threads in 1.6, rewriting the input system in 2.3, strict mode, concurrent garbage collection, loaders, etc. If you want to achieve 60fps, you have 20 milliseconds to handle each frame. This is not a lot of time. Just touching the flash storage system in the thread that is running the UI can in some cases introduce a delay that puts you out of that timing window, especially if you are writing to storage.

硬件加速描画不是魔法银蛋,它不能带来细腻流畅的UI体验. 在提高UI上有一些其它不同的尝试, 例如1.6中该经了前台和后台线程的切换, 2.3中重写了输入系统, strict mode, 并行垃圾收集, 加载器等等. 如果想要达到60fps, 你需要20毫秒来处理每个frame. 这时间不长. 只是简单的操作需要访问flash storage system的UI线程就很容易的会超过这个时间窗口, 尤其是在你要往flash storage写操作时.

• A recent example of the kinds of interesting things that impact UI smoothness: we noticed that ICS on Nexus S was actually less smooth when scrolling through lists than it was on Gingerbread. It turned out that the reason for this was due to subtle changes in timing, so that sometimes in ICS as the app was retrieving touch events and drawing the screen, it would go to get the next event slightly before it was ready, causing it to visibly miss a frame while tracking the finger even though it was drawing the screen at a solid 60fps. (Edit: for those who need this made clear, yes of course this particular issue is fixed.)

最近我们发现了一些有趣的事情, 它们会影响UI的平滑: 在Nexus S上ICS的scrolling没有在Gingerbread上流畅. 这个问题表示出ICS中对时间的一些微调,导致了问题. (目前已经修正了)

• When people have historically compared web browser scrolling between Android and iOS, most of the differences they are seeing are not due to hardware accelerated drawing. Originally Android went a different route for its web page rendering and made different compromises: the web page is turned in to a display list, which is continually rendered to the screen, instead of using tiles. This has the benefit that scrolling and zooming never have artifacts of tiles that haven’t yet been drawn. Its downside is that as the graphics on the web page get more complicated to draw the frame rate goes down. As of Android 3.0, the browser now uses tiles, so it can maintain a consistent frame rate as you scroll or zoom, with the negative of having artifacts when newly needed tiles can’t be rendered quickly enough. The tiles themselves are rendered in software, which I believe is the case for iOS as well. (And this tile-based approach could be used prior to 3.0 without hardware accelerated drawing; as mentioned previously, the Nexus S CPU can easily draw the tiles to the window at 60fps.)

当人们比较Android和iOS上web browser的滚动效果时, 大部分的差异不是由于硬件加速描绘造成的. 原来的Android对于web page的宣染和组织采用了与现在不同的方式: web page是被按照list的方式显示的. 它是连续的在屏幕上宣染, 而不是使用tiles. 这样做的好处是在滚动和放大缩小时, 不会有还没有被描画的假的tiles被显示出来. 缺点是如果web page上有复杂的图像,那么会降低frame rate. 从3.0开始, browser开始使用tile的方式来宣染, 它保证了在滚动和放大缩小时能够得到一致的frame rate, 但是会使得这时候有可能会显示artifact tiles. 这些来不及宣染的artifact tiles是通过软件来宣染的. 我猜iOS也是这么做的.(折中基于tile的方式也可以被3.0之前的系统使用; 就像之前说的, Nexus S的CPU足够达到60fps.)

• Hardware accleration does not magically make drawing performance problems disappear. There is still a limit to how much the GPU can do. A recent interesting example of this is tablets built with Tegra 2 -- that GPU can touch every pixel of a 1280x800 screen about 2.5 times at 60fps. Now consider the Android 3.0 tablet home screen where you are switching to the all apps list: you need to draw the background (1x all pixels), then the layer of shortcuts and widgets (let’s be nice and say this is .5x all pixels), then the black background of all apps (1x all pixels), and the icons and labels of all apps (.5x all pixels). We’ve already blown our per-pixel budget, and we haven’t even composited the separate windows to the final display yet. To get 60fps animation, Android 3.0 and later use a number of tricks. A big one is that it tries to put all windows into overlays instead of having to copy them to the framebuffer with the GPU. In the case here even with that we are still over-budget, but we have another trick: because the wallpaper on Android is in a separate window, we can make this window larger than the screen to hold the entire bitmap. Now, as you scroll, the movement of the background doesn’t require any drawing, just moving its window... and because this window is in an overlay, it doesn’t even need to be composited to the screen with the GPU.

硬件加速不能使描绘效率的问题神奇的消失. GPU能做到什么程度依然有所限制. 最近的有趣例子就是基于Teara 2的tablet--它的GPU可以达到1280*800的每个pixel进行2.5倍于60fps的操作.  下面的文字阐述了在显示上进行的一些tricks. 主要是利用window和overlay的关系来减少对GPU的使用.  比如把所有的windows都描绘到overlay上, 而不是用GPU复制到对应的framebuffer上

• As device screen resolution goes up, achieving a 60fps UI is closely related to GPU speed and especially the GPU’s memory bus bandwidth. In fact, if you want to get an idea of the performance of a piece of hardware, always pay close attention to the memory bus bandwidth. There are plenty of times where the CPU (especially with those wonderful NEON instructions) can go a lot faster than the memory bus.

随着device的分辨率的提高, 要到到60fps的UI就更依赖GPU的速符及GPU的memory的带宽. 事实上, 如果你想获得一个硬件的理想性能, 你总是需要特别关注它的memory总线带宽. 因为总是会有一些CPU的处理速度(尤其是那些支持NEON指令的CPU)要远高于内存总线的带宽.

EDIT:

Wow this has generated a lot more discussion, and I won't be able to address nearly everything that has been raised. But I'll try to expand on things a bit here to better address what I think are some of the more interesting points.

哇哦,这会衍生更多的讨论, 我不能解决所有东西. 但是我想在这里展示一些我认为更有趣的观点.

Some have raised points along the lines of Samsung Galaxy S2 phones already having a smoother UI and indicating that they are doing something different vs. the Galaxy Nexus. When comparing individual devices though you really need to look at all of the factors. For example, the S2's screen is 480x800 vs. the Galaxy Nexus at 720x1280. If the Nexus S could already do 60fps for simple UIs on its 480x800, the CPU in the S2's is even better off.

The real important difference between these two screens is just that the Galaxy Nexus has 2.4x as many pixels that need to be drawn as the S2. This means that to achieve the same efficiency at drawing the screen, you need a CPU that can run a single core at 2.4x the speed (and rendering a UI for a single app is essentially not parallelizable, so multiple cores isn't going to save you).

This is where hardware accelerated rendering really becomes important: as the number of pixels goes up, GPUs can generally scale much better to handle them, since they are more specialized at their task. In fact this was the primary incentive for implementing hardware accelerated drawing in Android -- at 720x1280 we are well beyond the point where current ARM CPUs can provide 60fps. (And this is a reason to be careful about making comparisons between the Galaxy Nexus and other devices like the S2 -- if you are running third party apps, there is a good chance today that the app is not enabling hardware acceleration, so your comparison is doing CPU rendering on the Galaxy Nexus which means you almost certainly aren't going to get 60fps out of it, because it needs to hit 2.4x as many pixels as the S2 does.)

GPU可以对更高的分辨率提供更高的宣染效率.

To be complete, there is another big advantage that the GPU gives you -- many more drawing effects become feasible. For example, if you are drawing a bitmap in software, you basically can't do anything to it except apply an offset. Just trying to scale it is going to make rendering significantly slower. On a GPU, applying transformations well beyond simple scales is basically free. This is why in the new default Holo themes in Android we have background images -- with hardware accelerated drawing, we can afford to draw (and scale) them. In fact, if the hardware path is not enabled by the app, these background images will be turned off.

GPU可以带来更多的优化选择. 比如, 对于一个图片的放大缩小, GPU可以更好的完成这个动作, 而如果用软件则太慢了.