Android openCamera流程

从 Application 连接到 CameraService,这涉及到 Android 架构中的三个层次:APP 层,frameworks 层,Runtime 层。 其中,APP 层直接调用 frameworks 层所封装的方法,而 frameworks 层需要通过 Binder 远程调用 Runtime 中 CameraService 的函数。

下面以 android P 开始跟踪代码。

从APP到CameraService

这部分主要的调用逻辑如下图所示(图片来源 StoneDemo CSDN博客)。
图片来源 StoneDemo CSDN博客

APP

在应用中使用camera,将需要创建 CameraManager 对象实例,而后通过 CameraManager::openCamera() 打开camera设备。

openCamera(String cameraId, final CameraDevice.StateCallback callback, Handler handler)

其中:

  • cameraId 是需要打开的camera设备
  • callback 是camera设备状态的回调
  • handler 是回调函数需要使用的 Handler

frameworks

CameraManager

文件路径:android/frameworks/base/core/java/android/hardware/camera2/CameraManager.java

通过 CameraManager::openCamera() 方法可以知道,最终也将是调用 openCameraDeviceUserAsync(cameraId, callback, executor, clientUid) 方法,其中 clientUid 为 USE_CALLING_UID。

    private CameraDevice openCameraDeviceUserAsync(String cameraId,
            CameraDevice.StateCallback callback, Executor executor, final int uid)
            throws CameraAccessException {
        CameraCharacteristics characteristics = getCameraCharacteristics(cameraId);
        CameraDevice device = null;

        synchronized (mLock) {

            ICameraDeviceUser cameraUser = null;

            /* 实例化CameraDeviceImpl,该对象的构造函数将保存参数信息 */
            android.hardware.camera2.impl.CameraDeviceImpl deviceImpl =
                    new android.hardware.camera2.impl.CameraDeviceImpl(
                        cameraId,
                        callback,
                        executor,
                        characteristics,
                        mContext.getApplicationInfo().targetSdkVersion);

            /* CameraDeviceImpl 对象包含的 CameraDeviceCallbacks 实例,
             * 每个 CameraDeviceImpl 都将创建一个 CameraDeviceCallbacks,
             * 这是提供给远端连接到 CameraDeviceImpl 的接口
             */
            ICameraDeviceCallbacks callbacks = deviceImpl.getCallbacks();

            try {
                /* 根据 API1/API2 进行不同的区分,以API2进行跟踪 */
                if (supportsCamera2ApiLocked(cameraId)) {
                    // Use cameraservice's cameradeviceclient implementation for HAL3.2+ devices
                    /* 通过同一文件的 CameraManagerGlobal 类方法获得CameraService代理。
                     * 实际上是通过ServiceManager获得"media.camera" 的
                     * CameraService Binder代理,然后转变为ICameraService
                     */
                    ICameraService cameraService = CameraManagerGlobal.get().getCameraService();
                    if (cameraService == null) {
                        throw new ServiceSpecificException(
                            ICameraService.ERROR_DISCONNECTED,
                            "Camera service is currently unavailable");
                    }
                    /* 通过 ICameraService::connectDevice() 连接设备,
                     * 返回的 cameraUser 实际上指向的是远端 CameraDeviceClient
                     * 的本地接口
                     */
                    cameraUser = cameraService.connectDevice(callbacks, cameraId,
                            mContext.getOpPackageName(), uid);
                } else {
                    // Use legacy camera implementation for HAL1 devices
                    ...
                }
            } catch (ServiceSpecificException e) {
                ...
            }

            // TODO: factor out callback to be non-nested, then move setter to constructor
            // For now, calling setRemoteDevice will fire initial
            // onOpened/onUnconfigured callbacks.
            // This function call may post onDisconnected and throw CAMERA_DISCONNECTED if
            // cameraUser dies during setup.
            /* 将 CameraDeviceClient 设置到 CameraDeviceImpl 中进行管理 */
            deviceImpl.setRemoteDevice(cameraUser);
            device = deviceImpl;
        }

        return device;
    }
CameraDeviceImpl

文件路径:android/frameworks/base/core/java/android/hardware/camera2/impl/CameraDeviceImpl.java

    /**
     * Set remote device, which triggers initial onOpened/onUnconfigured callbacks
     *
     * <p>This function may post onDisconnected and throw CAMERA_DISCONNECTED if remoteDevice dies
     * during setup.</p>
     *
     */
    public void setRemoteDevice(ICameraDeviceUser remoteDevice) throws CameraAccessException {
        synchronized(mInterfaceLock) {
            // TODO: Move from decorator to direct binder-mediated exceptions
            // If setRemoteFailure already called, do nothing
            if (mInError) return;

            /* 将远端设备增加一层封装 */
            mRemoteDevice = new ICameraDeviceUserWrapper(remoteDevice);

            IBinder remoteDeviceBinder = remoteDevice.asBinder();
            ...

            /* 将会触发 onOpened 和 onUnconfigured 这两个回调,
             * 这两个回调是在创建 CameraDeviceImpl 实例对象时传递
             * 进来的callback,而该callback只是APP在调用 openCamera()
             * 时传递进来的,应用一般实现为开启预览等操作
             */
            mDeviceExecutor.execute(mCallOnOpened);
            mDeviceExecutor.execute(mCallOnUnconfigured);
        }
    }
CameraService

文件路径:android/frameworks/av/services/camera/libcameraservice/CameraService.cpp

在调用 cameraService.connectDevice(callbacks, cameraId, mContext.getOpPackageName(), uid) 时,最终将会调用到 CameraService.cpp 中的 connectDevice() 函数。而在 connectDevice() 函数中,connectHelper() 才是真正的逻辑实现,此时设定的模板类型是 <hardware::camera2::ICameraDeviceCallbacks,CameraDeviceClient>

Status CameraService::connectDevice(
        const sp<hardware::camera2::ICameraDeviceCallbacks>& cameraCb,
        const String16& cameraId,
        const String16& clientPackageName,
        int clientUid,
        /*out*/
        sp<hardware::camera2::ICameraDeviceUser>* device) {

    ATRACE_CALL();
    Status ret = Status::ok();
    String8 id = String8(cameraId);
    sp<CameraDeviceClient> client = nullptr;
    ret = connectHelper<hardware::camera2::ICameraDeviceCallbacks,CameraDeviceClient>(cameraCb, id,
            /*api1CameraId*/-1,
            CAMERA_HAL_API_VERSION_UNSPECIFIED, clientPackageName,
            clientUid, USE_CALLING_PID, API_2,
            /*legacyMode*/ false, /*shimUpdateOnly*/ false,
            /*out*/client);

    ...

    *device = client;
    return ret;
}

template<class CALLBACK, class CLIENT>
Status CameraService::connectHelper(const sp<CALLBACK>& cameraCb, const String8& cameraId,
        int api1CameraId, int halVersion, const String16& clientPackageName, int clientUid,
        int clientPid, apiLevel effectiveApiLevel, bool legacyMode, bool shimUpdateOnly,
        /*out*/sp<CLIENT>& device) {
    binder::Status ret = binder::Status::ok();

    String8 clientName8(clientPackageName);

    ...

    sp<CLIENT> client = nullptr;
    {
        ...

        sp<BasicClient> tmp = nullptr;
        /* makeClient() 将会根据API版本以及HAL版本选择生成具体的 Client 实例 */
        if(!(ret = makeClient(this, cameraCb, clientPackageName,
                cameraId, api1CameraId, facing,
                clientPid, clientUid, getpid(), legacyMode,
                halVersion, deviceVersion, effectiveApiLevel,
                /*out*/&tmp)).isOk()) {
            return ret;
        }
        client = static_cast<CLIENT*>(tmp.get());

        ...

        /* client 进行初始化工作
         * 注意这里的参数,mCameraProviderManager 是 CameraService 运行起来时,
         * 第一次强指针引用从而调用 onFirstRef() 函数创建的 CameraProviderManager
         * 实例对象,它保存了 CameraProvider 信息,待会再详细分析
         */    
        err = client->initialize(mCameraProviderManager, mMonitorTags);
        ...
    } // lock is destroyed, allow further connect calls

    // Important: release the mutex here so the client can call back into the service from its
    // destructor (can be at the end of the call)
    device = client;
    return ret;
}

Status CameraService::makeClient(const sp<CameraService>& cameraService,
        const sp<IInterface>& cameraCb, const String16& packageName, const String8& cameraId,
        int api1CameraId, int facing, int clientPid, uid_t clientUid, int servicePid,
        bool legacyMode, int halVersion, int deviceVersion, apiLevel effectiveApiLevel,
        /*out*/sp<BasicClient>* client) {

    if (halVersion < 0 || halVersion == deviceVersion) {
        // Default path: HAL version is unspecified by caller, create CameraClient
        // based on device version reported by the HAL.
        switch(deviceVersion) {
          /* CAMERA_DEVICE_API_VERSION_1_0 只能使用 API1 */
          case CAMERA_DEVICE_API_VERSION_1_0:
            ...
            break;
          case CAMERA_DEVICE_API_VERSION_3_0:
          case CAMERA_DEVICE_API_VERSION_3_1:
          case CAMERA_DEVICE_API_VERSION_3_2:
          case CAMERA_DEVICE_API_VERSION_3_3:
          case CAMERA_DEVICE_API_VERSION_3_4:
            if (effectiveApiLevel == API_1) { // Camera1 API route
                sp<ICameraClient> tmp = static_cast<ICameraClient*>(cameraCb.get());
                *client = new Camera2Client(cameraService, tmp, packageName,
                        cameraId, api1CameraId,
                        facing, clientPid, clientUid,
                        servicePid, legacyMode);
            } else { // Camera2 API route
                sp<hardware::camera2::ICameraDeviceCallbacks> tmp =
                        static_cast<hardware::camera2::ICameraDeviceCallbacks*>(cameraCb.get());
                /* API2 HAL3 将会通过 CameraDeviceClient() 创建 client,
                 * 这一 client 最终返回到 CameraDeviceImpl 实例,被保存在 mRemoteDevice
                 */
                *client = new CameraDeviceClient(cameraService, tmp, packageName, cameraId,
                        facing, clientPid, clientUid, servicePid);
            }
            break;
          default:
            // Should not be reachable
            ...
        }
    } else {
        // A particular HAL version is requested by caller. Create CameraClient
        // based on the requested HAL version.
        ...
    }
    return Status::ok();
}

至此,打开相机流程中,从 App 到 CameraService 的调用逻辑基本上就算走完了。

简图总结

根据上面的流程追踪,我们可以描绘一个比较简单直观的连路框架图,如下(图片来源 StoneDemo CSDN博客)。
其中黑色虚线表示下行(控制)路线,红色虚线表明上行(状态、数据)路线。
图片来源 StoneDemo CSDN博客

从CameraService到CameraProvider

由于 Android O 中加入了 Treble 机制,它带来的一个巨大变化就是将原本的 CameraServer 进程分隔成 CameraServer 与 Provider service 两个进程,它们之间通过 HIDL(一个类似 Binder 的机制)进行通信。
在这种情况下,CameraServer 一端主体为 CameraService,它将会寻找现存的 Provider service,将其加入到内部的 CameraProviderManager 中进行管理,相关操作都是通过远端调用进行的。
而 Provider service 一端的主体为 CameraProvider,它在初始化时就已经连接到 libhardware 的 Camera HAL 实现层,并以 CameraModule 来进行管理。这两个进程的启动以及初始化在系统启动时就已经进行,所以当相机应用运行时,它们的初始化工作已经完成。

CS

CameraDeviceClient

文件路径:android/frameworks/av/services/camera/libcameraservice/api2/CameraDeviceClient.cpp

Camera2ClientBase 的设置模板为 CameraService::BasicClient 。

// Interface used by CameraService

CameraDeviceClient::CameraDeviceClient(const sp<CameraService>& cameraService,
        const sp<hardware::camera2::ICameraDeviceCallbacks>& remoteCallback,
        const String16& clientPackageName,
        const String8& cameraId,
        int cameraFacing,
        int clientPid,
        uid_t clientUid,
        int servicePid) :
    /* 通过构造函数初始化 Camera2ClientBase,在它的
     * 构造函数中,将会创建一个 Camera3Device 实例对象
     */
    Camera2ClientBase(cameraService, remoteCallback, clientPackageName,
                cameraId, /*API1 camera ID*/ -1,
                cameraFacing, clientPid, clientUid, servicePid),
    mInputStream(),
    mStreamingRequestId(REQUEST_ID_NONE),
    mRequestIdCounter(0) {

    ATRACE_CALL();
    ALOGI("CameraDeviceClient %s: Opened", cameraId.string());
}
client->initialize()

回到 CameraService::connectHelper() 函数,在调用 makeClient() 函数之后,将得到相应的client,接着,在 CameraService::connectHelper() 函数进行 client->initialize(mCameraProviderManager, mMonitorTags),mCameraProviderManager 为 CameraService 保存 CameraProvider 信息的成员变量。

// Camera2ClientBase 的设置模板为 CameraService::BasicClient 
template<typename TProviderPtr>
status_t CameraDeviceClient::initializeImpl(TProviderPtr providerPtr, const String8& monitorTags) {
    ATRACE_CALL();
    status_t res;

    res = Camera2ClientBase::initialize(providerPtr, monitorTags);
    if (res != OK) {
        return res;
    }

    String8 threadName;
    /* 创建 FrameProcessorBase 实例对象,它将负责帧处理 */
    mFrameProcessor = new FrameProcessorBase(mDevice);
    threadName = String8::format("CDU-%s-FrameProc", mCameraIdStr.string());
    /* 创建线程进行获取图像数据 */
    mFrameProcessor->run(threadName.string());

	/* 将 CameraDeviceClient 注册登记到 FrameProcessorBase
	 * 的 mRangeListeners,当帧处理完成时,将会通过它来回调通知APP */
    mFrameProcessor->registerListener(FRAME_PROCESSOR_LISTENER_MIN_ID,
                                      FRAME_PROCESSOR_LISTENER_MAX_ID,
                                      /*listener*/this,
                                      /*sendPartials*/true);
    ...

    return OK;
}

template <typename TClientBase>
template <typename TProviderPtr>
status_t Camera2ClientBase<TClientBase>::initializeImpl(TProviderPtr providerPtr,
        const String8& monitorTags) {
    ...

    // Verify ops permissions
    /* 将调用到 CameraService::BasicClient::startCameraOps() 函数 */
    res = TClientBase::startCameraOps();
    ...

    /* 将调用 Camera3Device::initialize() 函数 */
    res = mDevice->initialize(providerPtr, monitorTags);
    ...

    return OK;
}


TClientBase::startCameraOps()
        |
        |
        |
       \|/
status_t CameraService::BasicClient::startCameraOps() {
    ...
    // Transition device state to OPEN
    /* 这里,将会通过Binder代理,将设备状态更新到远端,这个时候,
     * CameraProvider 管理的device 状态将更新,设备进入 OPEN 状态
     */
    sCameraService->updateProxyDeviceState(ICameraServiceProxy::CAMERA_STATE_OPEN,
            mCameraIdStr, mCameraFacing, mClientPackageName, apiLevel);

    return OK;
}


res = mDevice->initialize(providerPtr, monitorTags)
        |
        |
        |
       \|/
status_t Camera3Device::initialize(sp<CameraProviderManager> manager, const String8& monitorTags) {
    ...

    sp<ICameraDeviceSession> session;
    ATRACE_BEGIN("CameraHal::openSession");
    /* 从 CameraProviderManager 的 mProviders 查找对应 id 的 mDevices,
     * 而后调用相对应的 deviceInfo3->mInterface->open(),其中,mInterface
     * 是在CameraProvider adddevice 时,通过DeviceInfo3的构造函数传递进来的,
     * mInterface 最终为
     * android::hardware::camera::device::V3_4::implementation::CameraDevice
     */
    status_t res = manager->openSession(mId.string(), this,
            /*out*/ &session);
    ATRACE_END();
    ...

    /* 获取请求队列 */
    std::shared_ptr<RequestMetadataQueue> queue;
    auto requestQueueRet = session->getCaptureRequestMetadataQueue(
        [&queue](const auto& descriptor) {
            queue = std::make_shared<RequestMetadataQueue>(descriptor);
            if (!queue->isValid() || queue->availableToWrite() <= 0) {
                ALOGE("HAL returns empty request metadata fmq, not use it");
                queue = nullptr;
                // don't use the queue onwards.
            }
        });
    ...

	/* 获取请求结果返回队列 */
    std::unique_ptr<ResultMetadataQueue>& resQueue = mResultMetadataQueue;
    auto resultQueueRet = session->getCaptureResultMetadataQueue(
        [&resQueue](const auto& descriptor) {
            resQueue = std::make_unique<ResultMetadataQueue>(descriptor);
            if (!resQueue->isValid() || resQueue->availableToWrite() <= 0) {
                ALOGE("HAL returns empty result metadata fmq, not use it");
                resQueue = nullptr;
                // Don't use the resQueue onwards.
            }
        });
    ....

    /* 运行相应的队列线程 */
    return initializeCommonLocked();
}

status_t Camera3Device::initializeCommonLocked() {
    /** Start up status tracker thread */
    mStatusTracker = new StatusTracker(this);
    status_t res = mStatusTracker->run(String8::format("C3Dev-%s-Status", mId.string()).string());

	/** Register in-flight map to the status tracker */
    mInFlightStatusId = mStatusTracker->addComponent();
    
    /** Start up request queue thread */
    mRequestThread = new RequestThread(this, mStatusTracker, mInterface, sessionParamKeys);
    res = mRequestThread->run(String8::format("C3Dev-%s-ReqQueue", mId.string()).string());
    
    mPreparerThread = new PreparerThread();
    ...
}

CameraProviderService

CameraDevice

文件路径:android/hardware/interfaces/camera/device/3.4/default/CameraDevice.cpp

CameraDevice 实例对象在初始化 CameraProviderService 之后就存在了,前面说到通过 CameraProviderManager 中的 deviceInfo 接口,调用远端 CameraDevice 实例的 open() 函数。(3.4版本的open函数,将会调用到3.2版本的open函数)

Return<void> CameraDevice::open(const sp<ICameraDeviceCallback>& callback, open_cb _hidl_cb)  {
    Status status = initStatus();
    sp<CameraDeviceSession> session = nullptr;

    ...

    if (status != Status::OK) {
        ...
    } else {
        ...

        /** Open HAL device */
        status_t res;
        camera3_device_t *device;

        ATRACE_BEGIN("camera3->open");
        /* 调用HAL层的 open() 函数 */
        res = mModule->open(mCameraId.c_str(),
                reinterpret_cast<hw_device_t**>(&device));
        ...

        struct camera_info info;
        res = mModule->getCameraInfo(mCameraIdInt, &info);
        ...

        /* 创建session实例对象 */
        session = createSession(
                device, info.static_camera_characteristics, callback);
        ...
        mSession = session;

        ...
        mLock.unlock();
    }
    _hidl_cb(status, session->getInterface());
    return Void();
}

sp<CameraDeviceSession> CameraDevice::createSession(camera3_device_t* device,
        const camera_metadata_t* deviceInfo,
        const sp<ICameraDeviceCallback>& callback) {
    return new CameraDeviceSession(device, deviceInfo, callback);
}

至此,Android open camera的流程操作到相应SOC厂商的camera HAL层,各个平台将会有一些差异,就不再详细的分析。

参考的结构图如下:
在这里插入图片描述
由于自身能力有限,对这一方面的理解还不够透彻,所以很多地方没有说清楚,后续理解了再详细更新,也欢迎各路大佬指点迷津,感谢!

参考文章

在了解学习android camera的过程中,参考了网上很多的优秀文章,感谢!

StoneDemo [Android O] HAL3 之 Open Camera2 流程(一)—— 从 App 到 CameraService

StoneDemo [Android O] HAL3 之 Open Camera2 流程(二)—— 从 CameraService 到 HAL Service

StoneDemo [Android O] HAL3 之 Open Camera2 流程(三,完结)—— 从 HAL Service 到 Camera HAL

  • 7
    点赞
  • 46
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

chengwei_peng

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值