Android Surface创建流程

Android Surface创建使用流程

刚刚梳理了一遍Android上UI绘制所依赖的Surface的创建过程，把一些重点记录下来，有一些概念需要先清楚：
1. 一个Activity一般都会对应到一个Window, 对应了一个DecorView, ViewRoot
2. ViewRoot中有一个Surface, 就是App可以用于绘制UI的Surface了，在Native层也对应了一个Surface, 在SurfaceFlinger对应的是一个Layer，通过Layer中的Producer可以真正的申请Buffer用于绘制。App中通过Surface中的Canvas的类似lockcanvas接口触发dequeue buffer流程。
3. 一个ViewRoot在WMS端对应了一个WindowState, WindowState又通过一系列引用关联到一个SurfaceControl, Native层也有一个SurfaceControl。这个可以用于控制窗口的一些属性。
4. WMS native层的SurfaceComposerClient与SurfaceFlinger之间通过ISurfaceComposerClient建立联系。一个App在SurfaceFlinger端对应了一个Client,用于处理该App layer的创建等请求。

Surface的创建使用过程需要按如下几个步骤来完成
- 1. APP与WMS, WMS与SurfaceFlinger建立联系
- 2. WMS端SurfaceControl创建过程
- 3. App端Surface创建过程
- 4. Application端Surface的使用

---

整体类图

1. App, WMS, SurfaceFlinger建立联系

App与WMS之间通过IWindowSession和IWindow进行夸进程交互。每个App和WMS之间建立了一个Session。

App与WMS的Session是通过WindowManagerGlobal建立的，WindowManagerGlobal是单例，每个process一个。

public static IWindowSession getWindowSession() {
if (sWindowSession == null) {
try {
InputMethodManager imm = InputMethodManager.getInstance();
IWindowManager windowManager = getWindowManagerService();
sWindowSession = windowManager.openSession(…)
}
return sWindowSession;
}
}

WindowManagerService为每个App建立一个Session，后续Application进程与WMS的交互都通过这个Session进行。
@Override
public IWindowSession openSession(IWindowSessionCallback callback, IInputMethodClient client,
IInputContext inputContext) {
Session session = new Session(this, callback, client, inputContext);
return session;
}

WindowManager与SurfaceFlinger的交互稍微复杂一些，首先WMS需要通过JNI在Java层SurfaceSession和Native层的SurfaceComposerClient之间建立联系，
然后再让SurfaceComposerClient和SurfaceFlinger进程内的Client建立联系。

大致过程如下，一般是Application端通过SetContentView起请求, 在ViewRoot中通过已经创建好的Session向WMS请求加入Window.

/**
* We have one child
*/
public void setView(View view, WindowManager.LayoutParams attrs, View panelParentView) {
res = mWindowSession.addToDisplay(mWindow, mSeq, mWindowAttributes,
getHostVisibility(), mDisplay.getDisplayId(),
mAttachInfo.mContentInsets, mAttachInfo.mStableInsets,
mAttachInfo.mOutsets, mInputChannel);

@Override
public int addToDisplay(IWindow window, int seq, WindowManager.LayoutParams attrs,
int viewVisibility, int displayId, Rect outContentInsets, Rect outStableInsets,
Rect outOutsets, InputChannel outInputChannel) {
return mService.addWindow(this, window, seq, attrs, viewVisibility, displayId,
outContentInsets, outStableInsets, outOutsets, outInputChannel);
}

WMS收到App端请求，建立Window
每个Window对应一个WindowState.
public int addWindow(Session session, IWindow client, int seq,
WindowManager.LayoutParams attrs, int viewVisibility, int displayId,
Rect outContentInsets, Rect outStableInsets, Rect outOutsets,
InputChannel outInputChannel) {
WindowState win = new WindowState(this, session, client, token,
attachedWindow, appOp[0], seq, attrs, viewVisibility, displayContent);
win.attach();

void attach() {
if (WindowManagerService.localLOGV) Slog.v(
TAG, “Attaching ” + this + ” token=” + mToken
+ “, list=” + mToken.windows);
mSession.windowAddedLocked();
}

Session
1. 每个App都与WMS有一个Session
2. 所有App的Session都在WMS中保存
3. 创建SurfaceSession
同一个App内部的所有WindowState使用相同的Session,也是对应同一个SurfaceSession.

void windowAddedLocked() {
if (mSurfaceSession == null) {
if (WindowManagerService.localLOGV) Slog.v(
TAG_WM, “First window added to ” + this + “, creating SurfaceSession”);
mSurfaceSession = new SurfaceSession();
if (SHOW_TRANSACTIONS) Slog.i(
TAG_WM, ” NEW SURFACE SESSION ” + mSurfaceSession);
mService.mSessions.add(this);
if (mLastReportedAnimatorScale != mService.getCurrentAnimatorScale()) {
mService.dispatchNewAnimatorScaleLocked(this);
}
}
mNumWindow++;
}

SurfaceSession通过JNI接口与WMS的native层建立联系，创建了一个SurfaceComposerClient.

/* Create a new connection with the surface flinger. /
public SurfaceSession() {
mNativeClient = nativeCreate();
}

JNI: android_view_SurfaceSession

static jlong nativeCreate(JNIEnv* env, jclass clazz) {
SurfaceComposerClient* client = new SurfaceComposerClient();
client->incStrong((void*)nativeCreate);
return reinterpret_cast(client);
}

SurfaceComposerClient在初始化过程中与SurfaceFlinger直接建立联系，SurfaceFlinger为每个App创建了一个Client对象。

SurfaceComposerClient::SurfaceComposerClient()
: mStatus(NO_INIT), mComposer(Composer::getInstance())
{
}

void SurfaceComposerClient::onFirstRef() {
sp sm(ComposerService::getComposerService());
if (sm != 0) {
sp conn = sm->createConnection();
if (conn != 0) {
mClient = conn;
mStatus = NO_ERROR;
}
}
}

SurfaceFlinger创建Client返回给SurfaceComposerClient.
SurfaceComposerClinet与Client一一对应。
每个App可能有多个Activity, 分别对应多个Window,对应多个ViewRoot, 但与WMS交互的只有一个Session.
在WMS中每个Activity都有对应的WindowState, 但都对应了同一个Session,同一个SurfaceSession,同一个SurfaceComposerClient, 同一个Composer.
在SurfaceFlinger中对应一个Clinet，至此，App与WMS, SurfaceFlinger建立联系的过程完成。
sp SurfaceFlinger::createConnection()
{
sp bclient;
sp client(new Client(this));
status_t err = client->initCheck();
if (err == NO_ERROR) {
bclient = client;
}
return bclient;
}

2. SurfaceControl建立过程

Application端的relayoutWindow触发SurfaceControl创建过程，主要有如下一些点：
1. jave层创建SurfaceControl对象，Native层创建SurfaceControl对象，对应。
2. SF内部Client通知SF创建Layer.
此步骤完成后，SF内部建立了Layer, WMS内部建立了SurfaceControl,可以实际对Surface的控制
但App层的Surface还没有建立联系。

private int relayoutWindow(WindowManager.LayoutParams params, int viewVisibility,
boolean insetsPending) throws RemoteException {

int relayoutResult = mWindowSession.relayout(
        mWindow, mSeq, params,
        (int) (mView.getMeasuredWidth() * appScale + 0.5f),
        (int) (mView.getMeasuredHeight() * appScale + 0.5f),
        viewVisibility, insetsPending ? WindowManagerGlobal.RELAYOUT_INSETS_PENDING : 0,
        mWinFrame, mPendingOverscanInsets, mPendingContentInsets, mPendingVisibleInsets,
        mPendingStableInsets, mPendingOutsets, mPendingBackDropFrame, mPendingConfiguration,
        mSurface);

public int relayout(IWindow window, int seq, WindowManager.LayoutParams attrs,
int requestedWidth, int requestedHeight, int viewFlags,
int flags, Rect outFrame, Rect outOverscanInsets, Rect outContentInsets,
Rect outVisibleInsets, Rect outStableInsets, Rect outsets, Rect outBackdropFrame,
Configuration outConfig, Surface outSurface) {
int res = mService.relayoutWindow(this, window, seq, attrs,
requestedWidth, requestedHeight, viewFlags, flags,
outFrame, outOverscanInsets, outContentInsets, outVisibleInsets,
outStableInsets, outsets, outBackdropFrame, outConfig, outSurface);

return res;

}

WMS通过IWindow找到对应的WindowState
通过WindowState找到WindowAnimator
public int relayoutWindow(Session session, IWindow client, int seq,
WindowManager.LayoutParams attrs, int requestedWidth,
int requestedHeight, int viewVisibility, int flags,
Rect outFrame, Rect outOverscanInsets, Rect outContentInsets,
Rect outVisibleInsets, Rect outStableInsets, Rect outOutsets, Rect outBackdropFrame,
Configuration outConfig, Surface outSurface) {
WindowState win = windowForClientLocked(session, client, false);
WindowStateAnimator winAnimator = win.mWinAnimator;
if (viewVisibility == View.VISIBLE &&
(win.mAppToken == null || !win.mAppToken.clientHidden)) {
result = relayoutVisibleWindow(outConfig, result, win, winAnimator, attrChanges,
oldVisibility);
try {
result = createSurfaceControl(outSurface, result, win, winAnimator);

if (viewVisibility == View.VISIBLE && winAnimator.hasSurface()) {
// We already told the client to go invisible, but the message may not be
// handled yet, or it might want to draw a last frame. If we already have a
// surface, let the client use that, but don’t create new surface at this point.
winAnimator.mSurfaceController.getSurface(outSurface);

通过WindowAnimator去创建SurfaceControl

private int createSurfaceControl(Surface outSurface, int result, WindowState win,
WindowStateAnimator winAnimator) {
WindowSurfaceController surfaceController = winAnimator.createSurfaceLocked();
if (surfaceController != null) {
surfaceController.getSurface(outSurface);

return result;

}

每个WindowState都有一个WindowStateAnimator
每个WindowStateAnimator都对应有一个WindowSurfaceController
WindowSurfaceController createSurfaceLocked() {
mSurfaceController = new WindowSurfaceController(mSession.mSurfaceSession,
attrs.getTitle().toString(),
width, height, format, flags, this);
mSurfaceController.setPositionAndLayer(mTmpSize.left, mTmpSize.top, layerStack, mAnimLayer);

return mSurfaceController;

}

WindowSurfaceController
每个WindowState都对应一个SurfaceControl
WindowSurfaceController中有个Java层的SurfaceControl对象
Java层的SurfaceControl对象通过mNativeObject与native层的SurfaceControl对象对应起来。
Java层对窗口控制的请求，都会传递给native层的SurfaceControl处理。
public WindowSurfaceController(SurfaceSession s,
String name, int w, int h, int format, int flags, WindowStateAnimator animator) {
if (animator.mWin.isChildWindow() &&
animator.mWin.mSubLayer < 0 &&
animator.mWin.mAppToken != null) {
mSurfaceControl = new SurfaceControlWithBackground(s,
name, w, h, format, flags, animator.mWin.mAppToken);

} else {
    mSurfaceControl = new SurfaceControl(
            s, name, w, h, format, flags);
}

}

public SurfaceControl(SurfaceSession session,
String name, int w, int h, int format, int flags)
throws OutOfResourcesException {
mName = name;
mNativeObject = nativeCreate(session, name, w, h, format, flags);
}

JNI: android_view_SurfaceControl
static jlong nativeCreate(JNIEnv* env, jclass clazz, jobject sessionObj,
jstring nameStr, jint w, jint h, jint format, jint flags) {
ScopedUtfChars name(env, nameStr);
sp client(android_view_SurfaceSession_getClient(env, sessionObj));
sp surface = client->createSurface(
String8(name.c_str()), w, h, format, flags);

}

SurfaceComposerClient内部创建SurfaceControl
1. 通知Client端创建layer
2. 由SurfaceComposerClient创建出native层的SurfaceControl对象，返回给Java层
sp SurfaceComposerClient::createSurface(
const String8& name,
uint32_t w,
uint32_t h,
PixelFormat format,
uint32_t flags)
{
if (mStatus == NO_ERROR) {
sp handle;
sp gbp;
status_t err = mClient->createSurface(name, w, h, format, flags,
&handle, &gbp);
if (err == NO_ERROR) {
sur = new SurfaceControl(this, handle, gbp);
}
}
return sur;
}

status_t Client::createSurface(
const String8& name,
uint32_t w, uint32_t h, PixelFormat format, uint32_t flags,
sp* handle,
sp* gbp)
{
class MessageCreateLayer : public MessageBase {
SurfaceFlinger* flinger;
Client* client;
sp* handle;
sp* gbp;
status_t result;
const String8& name;
uint32_t w, h;
PixelFormat format;
uint32_t flags;
public:
MessageCreateLayer(SurfaceFlinger* flinger,
const String8& name, Client* client,
uint32_t w, uint32_t h, PixelFormat format, uint32_t flags,
sp* handle,
sp* gbp)
: flinger(flinger), client(client),
handle(handle), gbp(gbp), result(NO_ERROR),
name(name), w(w), h(h), format(format), flags(flags) {
}
status_t getResult() const { return result; }
virtual bool handler() {
result = flinger->createLayer(name, client, w, h, format, flags,
handle, gbp);
return true;
}
};
sp msg = new MessageCreateLayer(mFlinger.get(),
name, this, w, h, format, flags, handle, gbp);
mFlinger->postMessageSync(msg);
return static_cast

3. Surface建立过程

WMS在创建好SurfaceCntrol后，通过SurfaceControl去获取Surface

ViewRootImpl中可以看到Application端传入了一个mSurface的参数过去接收WMS的返回结果。

private int relayoutWindow(WindowManager.LayoutParams params, int viewVisibility,
boolean insetsPending) throws RemoteException {

int relayoutResult = mWindowSession.relayout(
        mWindow, mSeq, params,
        (int) (mView.getMeasuredWidth() * appScale + 0.5f),
        (int) (mView.getMeasuredHeight() * appScale + 0.5f),
        viewVisibility, insetsPending ? WindowManagerGlobal.RELAYOUT_INSETS_PENDING : 0,
        mWinFrame, mPendingOverscanInsets, mPendingContentInsets, mPendingVisibleInsets,
        mPendingStableInsets, mPendingOutsets, mPendingBackDropFrame, mPendingConfiguration,
        mSurface);

在WMS内，在createSurfaceControl时带入了outSurface, 所以看起来在创建SurfaceControl的过程中会给Application的Surface赋值。
public int relayoutWindow(Session session, IWindow client, int seq,
WindowManager.LayoutParams attrs, int requestedWidth,
int requestedHeight, int viewVisibility, int flags,
Rect outFrame, Rect outOverscanInsets, Rect outContentInsets,
Rect outVisibleInsets, Rect outStableInsets, Rect outOutsets, Rect outBackdropFrame,
Configuration outConfig, Surface outSurface) {
result = createSurfaceControl(outSurface, result, win, winAnimator);

看到创建好SurfaceControl后，会调用SurfaceControl.getSurface完成对Surface赋值。
private int createSurfaceControl(Surface outSurface, int result, WindowState win,
WindowStateAnimator winAnimator) {
if (!win.mHasSurface) {
result |= RELAYOUT_RES_SURFACE_CHANGED;
}
WindowSurfaceController surfaceController = winAnimator.createSurfaceLocked();
if (surfaceController != null) {
surfaceController.getSurface(outSurface);

具体看看WindowSurfaceController中getSurface的过程：
void getSurface(Surface outSurface) {
outSurface.copyFrom(mSurfaceControl);
}

1. 通过SurfaceControl再创建新的native层Surface对象
2. 把Native层的Surface指针给Java层Surface的mNativeObject.
   public void copyFrom(SurfaceControl other) {
   long surfaceControlPtr = other.mNativeObject;
   long newNativeObject = nativeCreateFromSurfaceControl(surfaceControlPtr);
   synchronized (mLock) {
   setNativeObjectLocked(newNativeObject);
   }
   }

android_view_surface.cpp

static jlong nativeCreateFromSurfaceControl(JNIEnv* env, jclass clazz,
jlong surfaceControlNativeObj) {
sp ctrl(reinterpret_cast

4. Surface使用

1. App端使用Surface去绘图，可以直接通知到native层的Surface完成Buffer的dequeue & enqueue.
2. WMS端使用SurfaceControl完成窗口属性的控制

通过Java层Surface的lockCanvas & unlockCanvasAndPost可以分别去dequeue buffer & enqueue buffer.
ViewRootImpl

/**
* @return true if drawing was successful, false if an error occurred
*/
private boolean drawSoftware(Surface surface, AttachInfo attachInfo, int xoff, int yoff,
boolean scalingRequired, Rect dirty) {

// Draw with software renderer.
final Canvas canvas;
try {
    final int left = dirty.left;
    final int top = dirty.top;
    final int right = dirty.right;
    final int bottom = dirty.bottom;
    canvas = mSurface.lockCanvas(dirty);

mView.draw(canvas);
} finally {
try {
surface.unlockCanvasAndPost(canvas);

public Canvas lockCanvas(Rect inOutDirty)
throws Surface.OutOfResourcesException, IllegalArgumentException {
synchronized (mLock) {

    mLockedObject = nativeLockCanvas(mNativeObject, mCanvas, inOutDirty);
    return mCanvas;
}

}

static jlong nativeLockCanvas(JNIEnv* env, jclass clazz,
jlong nativeObject, jobject canvasObj, jobject dirtyRectObj) {
sp surface(reinterpret_cast