Android Camera SDK NDK NDK_vendor介绍

最新推荐文章于 2024-06-14 00:30:00 发布

chinamaoge

最新推荐文章于 2024-06-14 00:30:00 发布

阅读量1.3k

点赞数

分类专栏： Android Flow Introduction 文章标签： android c++ java

本文链接：https://blog.csdn.net/chinamaoge/article/details/129224096

版权

Android Flow Introduction 专栏收录该内容

5 篇文章 0 订阅

订阅专栏

Android Camera JNI NDK NDK_vendor介绍

前言
主要有哪几种interface？
Android SDK
Android NDK
Camera VTS Testcase
总结
Reference

前言

本篇博客是想介绍Android camera从application layer到camera service layer整个框架中，能够使用的所有接口方式。

主要有哪几种interface？

分为两大类：Android SDK，Android NDK。
Android NDK分为NDK和NDK vendor。对于camera module Android SDK可以分为camera API 1和camera API 2。最后有一种特殊的方式是用过HIDL interface来实现camera APP，给出总体架构图：
Android camera overall architecture diagram

Android SDK

Camera API 1

API 1作为最原始的一套camera API现在已经被Google启用，已经不在有功能上的升级，可以提供基础capture pictures and videos in your applications。API 1流程非常的特殊，他是通过JNI layer调用到libcameraservice，如果想了解JNI移步：链接: Camera Java Native Interface(JNI)介绍。
接下来通过camera open代码分析JNI调用过程

frameworks/base/core/java/android/hardware/Camera.java
public static Camera open() {
    int numberOfCameras = getNumberOfCameras();
    CameraInfo cameraInfo = new CameraInfo();
    for (int i = 0; i < numberOfCameras; i++) {
        getCameraInfo(i, cameraInfo);
        if (cameraInfo.facing == CameraInfo.CAMERA_FACING_BACK) {
            return new Camera(i);
        }
    }
    return null;
}

如果在camera APP中调用API 1的open function去打开默认后置camera，一开始会新建一个camera device object，直接调用camera构造函数。

frameworks/base/core/java/android/hardware/Camera.java
Camera(int cameraId) {
    if(cameraId >= getNumberOfCameras()){
            throw new RuntimeException("Unknown camera ID");
    }
    int err = cameraInit(cameraId);
	…
    initAppOps();
}

在camera 构造函数中调用cameraInit。

frameworks/base/core/java/android/hardware/Camera.java
private int cameraInit(int cameraId) {
	…
	//调用到JNI中android_hardware_Camera_native_setup
    return native_setup(new WeakReference<Camera>(this), cameraId,
            ActivityThread.currentOpPackageName());
}

在camera Init中通过native_setup interface就会调用到JNI中android_hardware_Camera_native_setup function。

frameworks/base/core/jni/android_hardware_Camera.cpp
static jint android_hardware_Camera_native_setup(JNIEnv *env, jobject thiz,
    jobject weak_this, jint cameraId, jstring clientPackageName)
{
    const char16_t *rawClientName = reinterpret_cast<const char16_t*>(
        env->GetStringChars(clientPackageName, NULL));
    jsize rawClientNameLen = env->GetStringLength(clientPackageName);
    String16 clientName(rawClientName, rawClientNameLen);
    env->ReleaseStringChars(clientPackageName,
                            reinterpret_cast<const jchar*>(rawClientName));
    int targetSdkVersion = android_get_application_target_sdk_version();
    sp<Camera> camera = Camera::connect(cameraId, clientName, Camera::USE_CALLING_UID,
                                        Camera::USE_CALLING_PID, targetSdkVersion);
	…
    sp<JNICameraContext> context = new JNICameraContext(env, weak_this, clazz, camera);
    context->incStrong((void*)android_hardware_Camera_native_setup);
    camera->setListener(context);
    return NO_ERROR;
}

android_hardware_Camera_native_setup 中connect是Camera C/S架构的客户端，调用connect函数向服务器发送连接请求。
JNICameraContext这个类是一个监听类，用于处理底层Camera回调函数传来的数据和消息。

frameworks/av/camera/Camera.cpp
sp<Camera> Camera::connect(int cameraId, const String16& clientPackageName,
        int clientUid, int clientPid, int targetSdkVersion)
{
    return CameraBaseT::connect(cameraId, clientPackageName, clientUid,
            clientPid, targetSdkVersion);
}

JNI调用进入C++ framework camera file connect function。

frameworks/av/camera/CameraBase.cpp
template <typename TCam, typename TCamTraits>
sp<TCam> CameraBase<TCam, TCamTraits>::connect(int cameraId,
                                               const String16& clientPackageName,
                                               int clientUid, int clientPid, int targetSdkVersion)
{
    ALOGV("%s: connect", __FUNCTION__);
	//创建一个camera对象，调用Camera和CameraBase构造函数
    sp<TCam> c = new TCam(cameraId);
    sp<TCamCallbacks> cl = c;
	//获取camera service对象
    const sp<::android::hardware::ICameraService> cs = getCameraService();
    binder::Status ret;
    if (cs != nullptr) {
        TCamConnectService fnConnectService = TCamTraits::fnConnectService;
        ret = (cs.get()->*fnConnectService)(cl, cameraId, clientPackageName, clientUid,
                                               clientPid, targetSdkVersion, /*out*/ &c->mCamera);
    }
	…
    return c;
}

CameraBase中通过获得到server端（camera service）代理对象ICameraService。
然后ICameraService通过fnConnectService链接camera service。

frameworks/av/camera/CameraBase.cpp
// establish binder interface to camera service
template <typename TCam, typename TCamTraits>
const sp<::android::hardware::ICameraService> CameraBase<TCam, TCamTraits>::getCameraService()
{
    Mutex::Autolock _l(gLock);
    if (gCameraService.get() == 0) {
        if (CameraUtils::isCameraServiceDisabled()) {
            return gCameraService;
        }
		//通过binder获取camera service
        sp<IServiceManager> sm = defaultServiceManager();
        sp<IBinder> binder;
        do {
            binder = sm->getService(String16(kCameraServiceName));
            if (binder != 0) {
                break;
            }
            ALOGW("CameraService not published, waiting...");
            usleep(kCameraServicePollDelay);
        } while(true);
        if (gDeathNotifier == NULL) {
            gDeathNotifier = new DeathNotifier();
        }
        binder->linkToDeath(gDeathNotifier);
        gCameraService = interface_cast<::android::hardware::ICameraService>(binder);
    }
    ALOGE_IF(gCameraService == 0, "no CameraService!?");
    return gCameraService;
}

getCameraService function中主要是展示通过binder的方法获取到camera service代理对象。
为什么fnConnectService可以链接camera device呢？因为fnConnectService就等于connect，如下代码：

frameworks/av/camera/Camera.cpp
CameraTraits<Camera>::TCamConnectService CameraTraits<Camera>::fnConnectService =
        &::android::hardware::ICameraService::connect;

而connect本身定义为AIDL interface，就是通过binder实现跨进程调用到camera service，在camera service中进一步处理。

frameworks/av/camera/aidl/android/hardware/ICameraService.aidl
AIDL 文件中定义的connect interface，目的为跨进程调用
ICamera connect(ICameraClient client,
        int cameraId,
        String opPackageName,
        int clientUid, int clientPid,
        int targetSdkVersion);

到此camera API 1的流程就梳理完毕，JNI在其中做一个桥梁的作用，API 1通过JNI连接到C++ framework，也展示JNI的目的就是为让java field可以调用到C++ field。

Camera API 2

为更好的控制camera device，Google抛弃camera API 1重新设计camera API 2接口。
Camera2 provides in-depth controls for complex use cases, but requires you to manage device-specific configurations。
Camera API2 框架为应用提供更接近底层的相机控件，包括高效的零复制连拍/视频流以及曝光、增益、白平衡增益、颜色转换、去噪、锐化等方面的每帧控件。
camera API 2调用流程相对简单，直接从API 2 interface通过AIDL调用到camera service，接下来也是举例openCamera流程进行分析。

frameworks/base/core/java/android/hardware/camera2/CameraManager.java
public void openCamera(@NonNull String cameraId,
        @NonNull final CameraDevice.StateCallback callback, @Nullable Handler handler)
        throws CameraAccessException {
    openCameraForUid(cameraId, callback, CameraDeviceImpl.checkAndWrapHandler(handler),
            USE_CALLING_UID);
}

public void openCameraForUid(@NonNull String cameraId,
        @NonNull final CameraDevice.StateCallback callback, @NonNull Executor executor,
        int clientUid, int oomScoreOffset) throws CameraAccessException {
	…
    openCameraDeviceUserAsync(cameraId, callback, executor, clientUid, oomScoreOffset);
}

APP通过API 2 open camera操作可以调用到camera manager中的openCamera function，进而调用到openCameraForUid、openCameraDeviceUserAsync。

frameworks/base/core/java/android/hardware/camera2/CameraManager.java
private CameraDevice openCameraDeviceUserAsync(String cameraId,
        CameraDevice.StateCallback callback, Executor executor, final int uid,
        final int oomScoreOffset) throws CameraAccessException {
    CameraCharacteristics characteristics = getCameraCharacteristics(cameraId);
    CameraDevice device = null;
    Map<String, CameraCharacteristics> physicalIdsToChars =
            getPhysicalIdToCharsMap(characteristics);
    synchronized (mLock) {
        ICameraDeviceUser cameraUser = null;
        android.hardware.camera2.impl.CameraDeviceImpl deviceImpl =
                new android.hardware.camera2.impl.CameraDeviceImpl(
                    cameraId,
                    callback,
                    executor,
                    characteristics,
                    physicalIdsToChars,
                    mContext.getApplicationInfo().targetSdkVersion,
                    mContext);
        ICameraDeviceCallbacks callbacks = deviceImpl.getCallbacks();
        try {
            ICameraService cameraService = CameraManagerGlobal.get().getCameraService();
            if (cameraService == null) {
                throw new ServiceSpecificException(
                    ICameraService.ERROR_DISCONNECTED,
                    "Camera service is currently unavailable");
            }
            cameraUser = cameraService.connectDevice(callbacks, cameraId,
                mContext.getOpPackageName(),  mContext.getAttributionTag(), uid,
                oomScoreOffset, mContext.getApplicationInfo().targetSdkVersion);
        } catch (ServiceSpecificException e) {
		…
     }
    return device;
}

openCameraDeviceUserAsync function中通过getCameraService获取到camera service代理ICameraService，这部分操作是在class CameraManagerGlobal中实现。

frameworks/base/core/java/android/hardware/camera2/CameraManager.java
class CameraManagerGlobal {
	public ICameraService getCameraService() {
	    synchronized(mLock) {
	        connectCameraServiceLocked();
	        if (mCameraService == null && !sCameraServiceDisabled) {
	            Log.e(TAG, "Camera service is unavailable");
	        }
	        return mCameraService;
	    }
	}

	private void connectCameraServiceLocked() {
	    // Only reconnect if necessary
	    if (mCameraService != null || sCameraServiceDisabled) return;
	    Log.i(TAG, "Connecting to camera service");
	    IBinder cameraServiceBinder = ServiceManager.getService(CAMERA_SERVICE_BINDER_NAME);
	    if (cameraServiceBinder == null) {
	        // Camera service is now down, leave mCameraService as null
	        return;
	    }
	    try {
	        cameraServiceBinder.linkToDeath(this, /*flags*/ 0);
	    } catch (RemoteException e) {
	        // Camera service is now down, leave mCameraService as null
	        return;
	    }
		//直接通过AIDL 获取camera service
	    ICameraService cameraService = ICameraService.Stub.asInterface(cameraServiceBinder);
		…
	}
}

global manger camera class主要作为保持和camera service的链接，并且distribute camera device callback notice。
getCameraService调用到connectCameraServiceLocked，可以很清楚的看到通过binder获取到cameraService。
因为connectDevice interface本身定义为AIDL，跨进程的方式从java framework调用到camera service中。

frameworks/av/camera/aidl/android/hardware/ICameraService.aidl
 ICameraDeviceUser connectDevice(ICameraDeviceCallbacks callbacks,
         String cameraId,
         String opPackageName,
         @nullable String featureId,
         int clientUid, int oomScoreOffset,
         int targetSdkVersion);

以上就讲解完通过 API 2方式从APP调用到camera service。

小结

虽然Android SDK中的API 1和API 2都是通过AIDL调用到camera service，但是分别是在C++ code和java code中调用，这里就体现AIDL这种接口描述性语言的优势，不需要考虑binder service和client两端的编程语言，只需要按照要求编写好AIDL interface，编译之后自动生成代码，在使用的时候按接口标准就可以实现。

Android NDK

NDK Interface

NDK的全拼是：Native Develop Kit，Android NDK 是一套允许您使用原生代码语言(例如C和C++)实现部分应用的工具集。在开发某些类型应用时，这有助于您重复使用以这些语言编写的代码库。
我们来分析要是使用NDK的方式来编写camera APP进程的代码，open camera的流程是怎么样的。

frameworks/av/camera/ndk/NdkCameraManager.cpp
camera_status_t ACameraManager_openCamera(
        ACameraManager* mgr, const char* cameraId,
        ACameraDevice_StateCallbacks* callback,
        /*out*/ACameraDevice** device) {
    return mgr->openCamera(cameraId, callback, device);
}

在system分区中通过使用C++代码编写的camera APP，open camera调用的是NDK中的ACameraManager_openCamera，进一步进入camera manager openCamera function。

frameworks/av/camera/ndk/NdkCameraManager.cpp
camera_status_t
ACameraManager::openCamera(
        const char* cameraId,
        ACameraDevice_StateCallbacks* callback,
        /*out*/ACameraDevice** outDevice)
{
    sp<hardware::ICameraService> cs = CameraManagerGlobal::getInstance().getCameraService();
    sp<hardware::camera2::ICameraDeviceCallbacks> callbacks = device->getServiceCallback();
    sp<hardware::camera2::ICameraDeviceUser> deviceRemote;
    int targetSdkVersion = android_get_application_target_sdk_version();
	//调用CameraService::connectDevice
    binder::Status serviceRet = cs->connectDevice(
            callbacks, String16(cameraId), String16(""), {},
            hardware::ICameraService::USE_CALLING_UID, /*oomScoreOffset*/0,
            targetSdkVersion, /*out*/&deviceRemote);
	...
}

openCamera function中先通过getCameraService获取到camera service代理对象（其中代码流程同上，就不分析了），再通过connectDevice AIDL interface 跨进程连接到camera service。

NDK Vendor Interface

VNDK的全称是Vendor Native Development Kit，是Android 8.0引入的一种新技术。它表现一系列库的合集，用于让供应商开发自己的HALs。VNDK 包含在 system.img 中，并在运行时与供应商代码动态关联。
使用vndk的原因是自Android8.0以来，Google引入了Treble架构，希望对vendor和system分区进行解耦处理，期待实现：framwork进程不加载vendor共享库，vendor进程仅加载vendor共享库（和部分framework共享库），而framework进程和vendor进程之间通过HIDL和hwbinder来通信
这里也是通过open camera function，来分析在vendor分区中编写C++ 代码的camera APP 调用到camera service。

frameworks/av/camera/ndk/ndk_vendor/impl/ACameraManager.cpp
camera_status_t
ACameraManager::openCamera(
        const char* cameraId,
        ACameraDevice_StateCallbacks* callback,
        /*out*/ACameraDevice** outDevice) {
    sp<ACameraMetadata> rawChars;
    camera_status_t ret = getCameraCharacteristics(cameraId, &rawChars);
    Mutex::Autolock _l(mLock);
    if (ret != ACAMERA_OK) {
        ALOGE("%s: cannot get camera characteristics for camera %s. err %d",
                __FUNCTION__, cameraId, ret);
        return ACAMERA_ERROR_INVALID_PARAMETER;
    }
    ACameraDevice* device = new ACameraDevice(cameraId, callback, std::move(rawChars));
    sp<ICameraService> cs = CameraManagerGlobal::getInstance().getCameraService();
    if (cs == nullptr) {
        ALOGE("%s: Cannot reach camera service!", __FUNCTION__);
        delete device;
        return ACAMERA_ERROR_CAMERA_DISCONNECTED;
    }
    sp<ICameraDeviceCallback> callbacks = device->getServiceCallback();
    sp<ICameraDeviceUser_2_0> deviceRemote_2_0;
	//通过调用HidlCameraService中connectDevice，hidl方式
    Status status = Status::NO_ERROR;
    auto serviceRet = cs->connectDevice(
            callbacks, cameraId, [&status, &deviceRemote_2_0](auto s, auto &device) {
                                     status = s;
                                     deviceRemote_2_0 = device;
                                 });
	…
    sp<ICameraDeviceUser> deviceRemote = castResult;
    device->setRemoteDevice(deviceRemote);
    device->setDeviceMetadataQueues();
    *outDevice = device;
    return ACAMERA_OK;
}

这里直接给出的是NDK Vendor中的openCamera function，但是这个函数也是被ACameraManager_openCamera调用。
其中mgr->openCamera是调用NDK还是NDK Vendor中的interface呢？需要通过mCameraManager来确定。

ACameraManager_openCamera(mCameraManager, mCameraId, &mDeviceCb, &mDevice);
mCameraManager = ACameraManager_create();
ACameraManager* ACameraManager_create() {
    ATRACE_CALL();
    return new ACameraManager();
}

ACameraManager_openCamera调用时会传入mCameraManager，mCameraManager是由ACameraManager_create创建出来。ACameraManager_create 创建过程直接new ACameraManager对象。

NDK ACameraManager：
struct ACameraManager {
    ACameraManager() :
            mGlobalManager(&(android::acam::CameraManagerGlobal::getInstance())) {}
    ~ACameraManager();
    camera_status_t getCameraIdList(ACameraIdList** cameraIdList);
    static void     deleteCameraIdList(ACameraIdList* cameraIdList);

    camera_status_t getCameraCharacteristics(
            const char* cameraId, android::sp<ACameraMetadata>* characteristics);
    camera_status_t openCamera(const char* cameraId,
                               ACameraDevice_StateCallbacks* callback,
                               /*out*/ACameraDevice** device);

  private:
    enum {
        kCameraIdListNotInit = -1
    };
    android::Mutex         mLock;
    android::sp<android::acam::CameraManagerGlobal> mGlobalManager;
};

NDK Vendor ACameraManager：
struct ACameraManager {
    ACameraManager() :
            mGlobalManager(&(android::acam::CameraManagerGlobal::getInstance())) {}
    ~ACameraManager();
    camera_status_t getCameraIdList(ACameraIdList** cameraIdList);
    static void     deleteCameraIdList(ACameraIdList* cameraIdList);

    camera_status_t getCameraCharacteristics(
            const char* cameraId, android::sp<ACameraMetadata>* characteristics);

    camera_status_t openCamera(const char* cameraId,
                               ACameraDevice_StateCallbacks* callback,
                               /*out*/ACameraDevice** device);
    camera_status_t getTagFromName(const char *cameraId, const char *name, uint32_t *tag);

  private:
    enum {
        kCameraIdListNotInit = -1
    };
    android::Mutex         mLock;
    android::sp<android::acam::CameraManagerGlobal> mGlobalManager;
};

new ACameraManager需要区分是调用NDK还是NDK Vendor创建对象，不同的对象进而实现不同的调用接口逻辑。
再回来看openCamera function中的connectDevice interface，其中传递的参数和NDK代码中明显不同，因为NDK Vendor调用的是HIDL 定义connectDevice interface。

frameworks/hardware/interfaces/cameraservice/service/2.0/ICameraService.hal
connectDevice(ICameraDeviceCallback callback, string cameraId)
    generates (Status status, ICameraDeviceUser device);

ICameraService.hal文件中定义的HIDL connectDevice interface。
cs->connectDevice通过binder跨进程可以调用到HidlCameraService::connectDevice中。

小结

以上分析了通过NDK和NDK Vendor方式，connect device连接到camera service的代码流程，区别在于NDK使用AIDL interface而NDK Vendor使用 HIDL interface，并且在libcameraservice中对接的接口也不一样。
System分区中new ACameraManager实例调用NDK interface，Vendor 分区中new ACameraManager实例调用NDK Vendor interface。
HidlCameraService对上连接NDK Vendor，但是最终实现功能还是依赖于CameraService。

Camera VTS Testcase

HIDL interface提供的接口是可以直接实现camera device所有基础的功能，但是由于没有 camera service的介入对于管理camera device是非常困难的。使用这种方式构建camera APP非常的特殊，没有经过整个Android framework，可以直接对HAL进行测试。
Camera VTS testcase 是基于HDIL interface来实现功能的测试程序，毕竟是Google提供给HAL 厂商稳定的接口，肯定需要测试保证其稳定性。
对于如果是camera service process没有启动的情况下，可以使用HDIL interface来写camera APP，感兴趣可以移步: Camera HIDL接口实现camera preview功能

总结

这篇博客主要是对Android framework接口方式进行介绍，其中包括Android SDK和Android NDK，Android SDK可以分为API 1和API 2，其中API 1通过JNI的方式调用实现。
JNI调用的cameraBase是和NDK、NDK vendor处于同一目录下，framework中的audio和video服务层（av目录名字含义）。
CameraBase、API 2和NDK都是通过AIDL跨进程调用到libcameraservice，而只有NDK Vendor独特的使用HIDL跨进程调用到libcameraservice。

Reference

链接: Android JNI(一)——NDK与JNI基础

chinamaoge

关注

0
点赞
踩
10

收藏

觉得还不错? 一键收藏
0
评论
Android Camera SDK NDK NDK_vendor介绍

本篇博客是想介绍Android camera从application layer到camera service layer整个框架中，能够使用的所有接口方式。
复制链接

扫一扫

专栏目录