【surfaceflinger源码分析】surface与surfaceflinger之间的关系

本篇文章带着以下问题继续分析surfaceflinger的源码:

1. 什么是surface ? surface与图形数据之间是什么关系？
2. surface和surfaceflinger之间是什么关系？

Surface定义

先看看Surface这个类的定义，主要是定义了很多与GraphicBuffer相关的操作。

class Surface
    : public ANativeObjectBase<ANativeWindow, Surface, RefBase>
{
public:

    /*
     * creates a Surface from the given IGraphicBufferProducer (which concrete
     * implementation is a BufferQueue).
     *
     * Surface is mainly state-less while it's disconnected, it can be
     * viewed as a glorified IGraphicBufferProducer holder. It's therefore
     * safe to create other Surfaces from the same IGraphicBufferProducer.
     *
     * However, once a Surface is connected, it'll prevent other Surfaces
     * referring to the same IGraphicBufferProducer to become connected and
     * therefore prevent them to be used as actual producers of buffers.
     *
     * the controlledByApp flag indicates that this Surface (producer) is
     * controlled by the application. This flag is used at connect time.
     */
    explicit Surface(const sp<IGraphicBufferProducer>& bufferProducer,
            bool controlledByApp = false);
    ...............................;
    virtual int dequeueBuffer(ANativeWindowBuffer** buffer, int* fenceFd);
    virtual int cancelBuffer(ANativeWindowBuffer* buffer, int fenceFd);
    virtual int queueBuffer(ANativeWindowBuffer* buffer, int fenceFd);
    ..............................;
    sp<IGraphicBufferProducer> mGraphicBufferProducer;
    ......................;

来翻译下这段代码中的注释：

creates a Surface from the given IGraphicBufferProducer (which concrete implementation is a BufferQueue).

Surface is mainly state-less while it’s disconnected, it can be viewed as a glorified IGraphicBufferProducer holder. It’s therefore safe to > create other Surfaces from the same IGraphicBufferProducer.

However, once a Surface is connected, it’ll prevent other Surfaces referring to the same IGraphicBufferProducer to become connected and therefore prevent them to be used as actual producers of buffers.

通过指定的IGraphicBufferProducer来创建一个Surface，而IGraphicBufferProducer是由具体的BufferQueue实现的。
当未连接时，Surface基本上是无状态的，可以把Surface看成是IGraphicBufferProducer光荣的持有者。因此此时可以使用相同的IGraphicBufferProducer去创建其他Surface。
当时一旦Surface已经连接，它将会阻止其他持有相同IGraphicBufferProducer的Surface编程连接状态，从而阻止他们被用作buffers的实际生产者。

从上面的注释来看，Surface需要一个IGraphicBufferProducer，而IGraphicBufferProducer由BufferQueue实现的。Surface在使用之前还需要connect。一堆问题随之而来:

1. Surface一般是怎么创建的呢？
2. BufferQueue是在哪里创建的呢？BufferQueue中的IGraphicBufferProducer给了Surface， 那Consumer是谁？
3. Surface connect动作，是要connect谁呢 ？

surface创建

在网络上找到一个surface的demo:https://github.com/yrzroger/NativeSFDemo
将代码整理后，其主要代码逻辑如下:

//1. 创建surfaceComposerClient
sp<SurfaceComposerClient> surfaceComposerClient = new SurfaceComposerClient
//2. 通过surfaceComposerClient创建Surface
sp<SurfaceControl> surfaceControl = surfaceComposerClient->createSurface(mName, .......);
//3. 设置surface的状态
SurfaceComposerClient::Transaction{}
            .setLayer(surfaceControl, std::numeric_limits<int32_t>::max())
            .show(surfaceControl)
            .apply();
 sp<ANativeWindow> nativeWindow = mSurfaceControl->getSurface();
 //4. 连接surface
 native_window_api_connect(nativeWindow, NATIVE_WINDOW_API_CPU);
 ative_window_set_xxxxxx(.........);
 while(!quit) {
     //5. 获取一个input buffer
     nativeWindow->dequeueBuffer(nativeWindow, &nativeBuffer, &releaseFenceFd);
     sp<GraphicBuffer> buf(GraphicBuffer::from(nativeBuffer));
     // 6.写input buffer
     memcp(buf, .....);
     // 7.将写好的input buffer存入buffer queue
     nativeWindow->queueBuffer(nativeWindow, buf->getNativeBuffer(), acquireFenceFd);
 }

从上面的代码流程来看，Surface是包在surfaceComposerClient内部的。先看看surfaceComposerClient的代码。

sp<SurfaceControl> SurfaceComposerClient::createSurface(const String8& name, uint32_t w, uint32_t h,
                                                        PixelFormat format, uint32_t flags,
                                                        SurfaceControl* parent,
                                                        LayerMetadata metadata,
                                                        uint32_t* outTransformHint) {
    sp<SurfaceControl> s;
    createSurfaceChecked(name, w, h, format, &s, flags, parent, std::move(metadata),
                         outTransformHint);
    return s;
}
status_t SurfaceComposerClient::createSurfaceChecked(const String8& name, uint32_t w, uint32_t h,
                                                     PixelFormat format,
                                                     sp<SurfaceControl>* outSurface, uint32_t flags,
                                                     SurfaceControl* parent, LayerMetadata metadata,
                                                     uint32_t* outTransformHint) {
    sp<SurfaceControl> sur;
    status_t err = mStatus;
    if (mStatus == NO_ERROR) {
        sp<IBinder> handle;
        sp<IGraphicBufferProducer> gbp;
        uint32_t transformHint = 0;
        //下面这句代码，可以获得IGraphicBufferProducer，卧槽，这不就是创建Surface需要的么 ？
        // mClient是个什么玩意 ？后面再深入分析
        err = mClient->createSurface(name, w, h, format, flags, parentHandle, std::move(metadata),
                                     &handle, &gbp, &transformHint);
        if (outTransformHint) {
            *outTransformHint = transformHint;
        }
        if (err == NO_ERROR) {
            //创建一个SurfaceControl，看来Surface的创建在SurfaceControl中了。
            *outSurface = new SurfaceControl(this, handle, gbp, transformHint);
        }
    }
    return err;
}

SurfaceControl中创建Surface对象

sp<Surface> SurfaceControl::getSurface() const
{
    Mutex::Autolock _l(mLock);
    if (mSurfaceData == nullptr) {
        return generateSurfaceLocked();
    }
    return mSurfaceData;
}
//将mGraphicBufferProducer作为参数创建Surface
sp<Surface> SurfaceControl::generateSurfaceLocked() const
{
    // This surface is always consumed by SurfaceFlinger, so the
    // producerControlledByApp value doesn't matter; using false.
    mSurfaceData = new Surface(mGraphicBufferProducer, false);

    return mSurfaceData;
}

至此，Surface的创建已经找到地方了。那么继续找BufferQueue是在哪里创建的？
上面SurfaceComposerClient由这句代码可以获取IGraphicBufferProducer，那么BufferQueue肯定与此相关，那么来看看这个是什么。

mClient->createSurface(…）

sp<ISurfaceComposerClient>  mClient;
//实际上这玩意是一个Binder对象，是一个代理，其实体在surfaceflinger进程中。
void SurfaceComposerClient::onFirstRef() {
    //获取surfacecomposer的代理，其实体就是SurfaceFlinger这个类
    //class SurfaceFlinger : public BnSurfaceComposer,
    //                  public PriorityDumper,
    //                   public ClientCache::ErasedRecipient,
    //                   private IBinder::DeathRecipient,
    //                   private HWC2::ComposerCallback,
    //                   private ISchedulerCallback {
    //               .............;
    //        };
    // sf->createConnection() 实际上是执行sp<ISurfaceComposerClient> SurfaceFlinger::createConnection()
    // 这里涉及Android binder的知识，暂不深究。后面会出独立篇幅来研究Binder
    sp<ISurfaceComposer> sf(ComposerService::getComposerService());
    if (sf != nullptr && mStatus == NO_INIT) {
        sp<ISurfaceComposerClient> conn;
        //在surfaceflinger中创建SurfaceComposerClient的实体，返回SurfaceComposerClient的代理
        conn = sf->createConnection();
        if (conn != nullptr) {
            mClient = conn;
            mStatus = NO_ERROR;
        }
    }
}

sp<ISurfaceComposerClient> SurfaceFlinger::createConnection() {
    const sp<Client> client = new Client(this);
    return client->initCheck() == NO_ERROR ? client : nullptr;
}

上面代码已经解释SurfaceComposerClient中sp mClient为surfaceflinger中Client的代理。
那么mClient->createSurface(…）实际执行的是surfaceflinger的Client接口:Client::createSurface(…)
那么看看此接口中是如何创建IGraphicBufferProducer的？

status_t Client::createSurface(const String8& name, uint32_t w, uint32_t h, PixelFormat format,
                               uint32_t flags, const sp<IBinder>& parentHandle,
                               LayerMetadata metadata, sp<IBinder>* handle,
                               sp<IGraphicBufferProducer>* gbp, uint32_t* outTransformHint) {
    // We rely on createLayer to check permissions.
    return mFlinger->createLayer(name, this, w, h, format, flags, std::move(metadata), handle, gbp,
                                 parentHandle, nullptr, outTransformHint);
}

从下面的代码看是BufferQueueLayer中创建IGraphicBufferProducer

status_t SurfaceFlinger::createLayer(const String8& name, const sp<Client>& client, uint32_t w,
                                     uint32_t h, PixelFormat format, uint32_t flags,
                                     LayerMetadata metadata, sp<IBinder>* handle,
                                     sp<IGraphicBufferProducer>* gbp,
                                     const sp<IBinder>& parentHandle, const sp<Layer>& parentLayer,
                                     uint32_t* outTransformHint) {
   
    sp<Layer> layer;
    std::string uniqueName = getUniqueLayerName(name.string());
    bool primaryDisplayOnly = false;

    switch (flags & ISurfaceComposerClient::eFXSurfaceMask) {
        case ISurfaceComposerClient::eFXSurfaceBufferQueue:
            //一般普通的surface对应的是BufferQueueLayer
            result = createBufferQueueLayer(client, std::move(uniqueName), w, h, flags,
                                            std::move(metadata), format, handle, gbp, &layer);

            break;
        case ISurfaceComposerClient::eFXSurfaceBufferState:
            result = createBufferStateLayer(client, std::move(uniqueName), w, h, flags,
                                            std::move(metadata), handle, &layer);
            break;
        case ISurfaceComposerClient::eFXSurfaceEffect:
            result = createEffectLayer(client, std::move(uniqueName), w, h, flags,
                                       std::move(metadata), handle, &layer);
            break;
        case ISurfaceComposerClient::eFXSurfaceContainer:
            result = createContainerLayer(client, std::move(uniqueName), w, h, flags,
                                          std::move(metadata), handle, &layer);
            break;
       ...........;
    }
    result = addClientLayer(client, *handle, *gbp, layer, parentHandle, parentLayer,
                            addToCurrentState, outTransformHint);
    .........................;
    return result;
}

继续看下IGraphicBufferProducer的代码:

status_t SurfaceFlinger::createBufferQueueLayer(const sp<Client>& client, std::string name,
                                                uint32_t w, uint32_t h, uint32_t flags,
                                                LayerMetadata metadata, PixelFormat& format,
                                                sp<IBinder>* handle,
                                                sp<IGraphicBufferProducer>* gbp,
                                                sp<Layer>* outLayer) {
    .........................................;
    sp<BufferQueueLayer> layer;
    LayerCreationArgs args(this, client, std::move(name), w, h, flags, std::move(metadata));
    args.textureName = getNewTexture();
    {
        Mutex::Autolock lock(mStateLock);
        //创建BufferQueueLayer
        layer = getFactory().createBufferQueueLayer(args);
    }
    status_t err = layer->setDefaultBufferProperties(w, h, format);
    if (err == NO_ERROR) {
        *handle = layer->getHandle();
        //获取IGraphicBufferProducer
        *gbp = layer->getProducer();
        *outLayer = layer;
    }
    return err;
}

sp<BufferQueueLayer> DefaultFactory::createBufferQueueLayer(const LayerCreationArgs& args) {
    return new BufferQueueLayer(args);
}

layer的继承关系:

class BufferQueueLayer : public BufferLayer {
    .........;
}
class BufferLayer : public Layer {
    .........;
}

class Layer : public virtual RefBase, compositionengine::LayerFE {
    ............;
}

BufferQueueLayer在哪里创建BufferQueue的呢 ？

void BufferQueueLayer::onFirstRef() {
    BufferLayer::onFirstRef();
    // Creates a custom BufferQueue for SurfaceFlingerConsumer to use
    sp<IGraphicBufferProducer> producer;
    sp<IGraphicBufferConsumer> consumer;
    //创建BufferQueue
    mFlinger->getFactory().createBufferQueue(&producer, &consumer, true);
    mProducer = mFlinger->getFactory().createMonitoredProducer(producer, mFlinger, this);
    mConsumer =
            mFlinger->getFactory().createBufferLayerConsumer(consumer, mFlinger->getRenderEngine(),
    ................................;
}

void DefaultFactory::createBufferQueue(sp<IGraphicBufferProducer>* outProducer,
                                       sp<IGraphicBufferConsumer>* outConsumer,
                                       bool consumerIsSurfaceFlinger) {
    BufferQueue::createBufferQueue(outProducer, outConsumer, consumerIsSurfaceFlinger);
}

void BufferQueue::createBufferQueue(sp<IGraphicBufferProducer>* outProducer,
        sp<IGraphicBufferConsumer>* outConsumer,
        bool consumerIsSurfaceFlinger) {
    sp<BufferQueueCore> core(new BufferQueueCore());
    sp<IGraphicBufferProducer> producer(new BufferQueueProducer(core, consumerIsSurfaceFlinger));
    sp<IGraphicBufferConsumer> consumer(new BufferQueueConsumer(core));
    *outProducer = producer;
    *outConsumer = consumer;
}

所以用于创建Surface的IGraphicBufferProducer实际为BufferQueueProducer， 而IGraphicBufferConsumer实际为BufferQueueConsumer储存在BufferQueueLayer中。

surface connect

native_window_api_connect(nativeWindow, NATIVE_WINDOW_API_CPU);
这个c接口最终是调用下面Surface::connect(…)

int Surface::connect(int api, const sp<IProducerListener>& listener, bool reportBufferRemoval) {
     ...................;
    int err = mGraphicBufferProducer->connect(listener, api, mProducerControlledByApp, &output);
    ....................;
    return err;
}

status_t BufferQueueProducer::connect(const sp<IProducerListener>& listener,
        int api, bool producerControlledByApp, QueueBufferOutput *output) {
    //如果已经连接过，就不允许再链接。也就是BufferQueueCore只允许由一个connected producer
    if (mCore->mConnectedApi != BufferQueueCore::NO_CONNECTED_API) {
        BQ_LOGE("connect: already connected (cur=%d req=%d)",
                mCore->mConnectedApi, api);
        return BAD_VALUE;
    }
    ..........................;
    int status = NO_ERROR;
    switch (api) {
        case NATIVE_WINDOW_API_EGL:
        case NATIVE_WINDOW_API_CPU:
        case NATIVE_WINDOW_API_MEDIA:
        case NATIVE_WINDOW_API_CAMERA:
            mCore->mConnectedApi = api;
            output->width = mCore->mDefaultWidth;
            output->height = mCore->mDefaultHeight;
            output->transformHint = mCore->mTransformHintInUse = mCore->mTransformHint;
            output->numPendingBuffers =
                    static_cast<uint32_t>(mCore->mQueue.size());
            output->nextFrameNumber = mCore->mFrameCounter + 1;
            output->bufferReplaced = false;
            output->maxBufferCount = mCore->mMaxBufferCount;
          ................................;
    }
    ..........................................;
    return status;
}

从上面的代码逻辑来看，surface connect的对象是surface中的producer对应的BufferQueueCore，BufferQueueCore同一时间只能链接一个producer因此可以阻止多次链接的问题。另外在connect的时候会获取BufferQueueCore的一些信息。

画图总结

问题解答

1. 什么是surface ? surface与图形数据之间是什么关系？
   surface是bufferqueue中bufferproducer的一个holder, 一个surface对应surfaceflinger中的一个layer， layer作为bufferqueue中buffer consumer的holder。app的UI/视频/camera图形数据通过surface的bufferproducer将数据发送到surfaceflinger中对应layer中的bufferproducer中，最终由layer中的buffer consumer取出图形数据进行合成显示。
   surface是图形数据传输到surfaceflinger的一个通道。
2. surface和surfaceflinger之间是什么关系？
   一个surface与surfaceflinger中一个layer一一对应。layer中有bufferqueue用于接收surface通过producer发过来的图形数据。

关系图