UNDERSTANDING ANDROID GRAPHICS INTERNALS – GRAPHICS BASICS (I)

In this post, it is presumed that you know about frame buffer driver, display control, display overlay, frame buffer, frame rate, common color space schemes, graphics pipeline ,EGL, Direct Rendering Infrastructure; If you do not, Wikipedia is an excellent source to learn these concepts.

The graphics subsystem underwent significant changes from ICS to Jelly Bean release which introduces vsync and triple buffering to improve on frame rate and animation smoothness along with major code refactoring.

• Surfaces and centralized surface composition

Android graphics runs on top of the Linux graphics DRM driver and EGL, understandable that it did not port from X11 which services remote network login. SurfaceFlinger provides the compositor service and a UI application does not draw to the frame buffer directly, thus must obtain a surface of depth to draw graphics from SurfaceFlinger.

From UI application point of view, a surface is a rectangular canvas window on the screen with 2D geometrical metrics and a particular color scheme. The depth of the surface is specified upon acquisition.

The SurfaceFlinger determines when and whether a surface shows on the screen, partially or entirely; regions of a surface which is lower than other surfaces will not be displayed or will be blended with colors of higher surfaces; regions of a surface out of the scope of the display screen will not be drawn at all. SurfaceFlinger drivers the composition of surface areas and the rendering to the display screen.

• What is a window on android? (ANativeWindow)

Each OS platform defines its distinct window structure and window types in EGL are derived from OS native window types. On android, the abstract window prototype is defined in system/core/include/system/window.h, namely ANativeWindow or its synonym EGLNativeWindowType in EGL as defined in frameworks/native/opengl/include/EGL/eglplatform.h, essentially two interchangeable terms in two domains.

Innately, a window is associated a certain count of graphic buffers which are used to contain drawing artifacts.

The struct ANativeWindow encompasses static attributes such as 2D resolutions and dynamic ones such as scaling modes, transforms, buffer swap interval range, a collection of callback function pointers.

The (*query)() callback is used to retrieve attributes from the native window.

The (*setSwapInterval) callback is to set swap interval.

(*queueBuffer), (*dequeueBuffer), (*cancellBuffer) are buffer operation routines; in JellyBean, the file descriptor fenceFd is the newly added argument linked to the fence in EGL extension API and is used for buffer idle signaling. When the window needs a fresh buffer to draw on, this callback is invoked to acquire one; when the current buffer is ready for rending, (*queueBuffer) is invoked to post the buffer for rendering. (*cancellBuffer) is invoked to cancel a buffer already dequeued but not used.

Another special (*perform) callback is invoked to perform customized operations. Valid operations include NATIVE_WINDOW_SET_USAGE, NATIVE_WINDOW_SET_CROP, NATIVE_WINDOW_SET_BUFFER_COUNT, NATIVE_WINDOW_SET_BUFFERS_TRANSFORM, NATIVE_WINDOW_SET_BUFFERS_FORMAT, NATIVE_WINDOW_SET_SCALING_MODE, NATIVE_WINDOW_LOCK,NATIVE_WINDOW_UNLOCK_AND_POST, NATIVE_WINDOW_API_CONNECT, NATIVE_WINDOW_API_DISCONNECT, NATIVE_WINDOW_SET_BUFFERS_USER_DIMENSIONS, NATIVE_WINDOW_SET_POST_TRANSFORM_CROP.

More operation codes may be added in the future.

Both ANativeWindow and ANativeWindowBuffer has a field common of type  struct android_native_base_t for reference counting checking.

• The native graphic buffer handle

In Android, a surface is allocated in the SurfaceFlinger and is passed over binder interface to the client ui app. The buffer affiliated with the surface is also allocated in SurfaceFlinger and passes to client over binder interface. As a note here, if not using ion driver, a surface graphic buffer is consistently allocated in ashmem driver and linked with a process-unique file descriptor. The memory address for the buffer can be restored by re-mapping the file descriptor. The details of preserving the file descriptor and restoring the memory address varies from one platform hardware vendor to another. Android only defines a generic native graphic handle as follows insystem/core/include/cutils/native_handle.h; vendor-specific private handle extends this structure in the gralloc.c.

typedef struct native_handle
{
int version;        /* sizeof(native_handle_t) */
int numFds;         /* number of file-descriptors at &data[0] */
int numInts;        /* number of ints at &data[numFds] */
int data[0];        /* numFds + numInts ints */
} native_handle_t;


In window.h the actually used buffer is typedefed as

typedef const native_handle_t* buffer_handle_t;

• The native graphic buffer ANativeWindowBuffer

The ANativeWindowBuffer is structured in the same file as ANativeWindow.

typedef struct ANativeWindowBuffer {

#ifdef __cplusplus     ANativeWindowBuffer() {

        common.magic = ANDROID_NATIVE_BUFFER_MAGIC;         common.version = sizeof(ANativeWindowBuffer);         memset(common.reserved, 0, sizeof(common.reserved));}

// Implement the methods that sp<ANativeWindowBuffer> expects so that it     // can be used to automatically refcount ANativeWindowBuffer's.    

void incStrong(const void* id) const {         common.incRef(const_cast<android_native_base_t*>(&common));     }    

void decStrong(const void* id) const {         common.decRef(const_cast<android_native_base_t*>(&common));     }

#endif

struct android_native_base_t common;

int width;    

int height;    

int stride;    

int format;    

int usage;

void* reserved[2];

buffer_handle_t handle;

void* reserved_proc[8]; }

ANativeWindowBuffer_t;

• Flattenable

For easy traversing over binder interface, an  graphic object may opts to implement the Flattenable interface whose declaration is quoted below.

/*  * The Flattenable interface allows an object to serialize itself out  * to a byte-buffer and an array of file descriptors.  */

class Flattenable {

public:    

// size in bytes of the flattened object    

virtual size_t getFlattenedSize() const = 0;

// number of file descriptors to flatten    

virtual size_t getFdCount() const = 0;

// flattens the object into buffer.    

// size should be at least of getFlattenedSize()  

// file descriptors are written in the fds[] array but ownership is    

// not transfered (ie: they must be dupped by the caller of    

// flatten() if needed).    

virtual status_t flatten(void* buffer, size_t size,             int fds[], size_t count) const = 0;

// unflattens the object from buffer.    

// size should be equal to the value of getFlattenedSize() when the    

// object was flattened.    

// unflattened file descriptors are found in the fds[] array and    

// don't need to be dupped(). ie: the caller of unflatten doesn't    

// keep ownership. If a fd is not retained by unflatten() it must be    

// explicitly closed.    

virtual status_t unflatten(void const* buffer,   size_t size, int fds[], size_t count) = 0;

protected:     virtual ~Flattenable() = 0;

};