Camera Framework 分析

最新推荐文章于 2023-06-04 14:00:09 发布
最新推荐文章于 2023-06-04 14:00:09 发布
阅读量1.4k 收藏 17
点赞数 3
分类专栏： camera 文章标签： android
原文链接：https://www.dazhuanlan.com/2019/10/16/5da68019bdbe3/?__yjs_captcha_trigger__=1
版权

Camera Framework 分析，本文主要介绍 Camera API2 相关。

类文件速查表

类文件目录

1
2
3
4
5
6

1. Framework Java API1:frameworks/base/core/java/android/hardware/Camera.java
2. Framework Java API2:frameworks/base/core/java/android/hardware/camera2
3. Framework JNI: frameworks/base/core/jni/
4. AIDL: frameworks/av/camera/aidl
5. Framework Native: frameworks/av/camera 
6. Framework Service: frameworks/av/services/camera/libcameraservice

JNI 相关

1
2
3
4
5
6
7
8

// frameworks/base/core/jni
./android_hardware_camera2_legacy_LegacyCameraDevice.cpp
./android_hardware_Camera.cpp
./android/graphics/Camera.cpp
./include/android_runtime/android_hardware_camera2_CameraMetadata.h
./android_hardware_camera2_DngCreator.cpp
./android_hardware_camera2_CameraMetadata.cpp
./android_hardware_camera2_legacy_PerfMeasurement.cpp

API 1 中，使用 jni 通过 Binder 机制和 CameraService 通信。
API 2 中，直接在 CameraManager.java 中通过 Binder 机制和 CameraService 通信。

AIDL 相关

Framework Camere AIDL 是 Camera 中客户端和服务端跨进程通信时使用的 AIDL 文件，代码都在 frameworks/av/camera/ 目录下，其中 aidl 文件一共有 16 个：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

xmt@server005:~/frameworks/av/camera/aidl/android/hardware$ tree
.
├── camera2
│   ├── CaptureRequest.aidl
│   ├── ICameraDeviceCallbacks.aidl
│   ├── ICameraDeviceUser.aidl
│   ├── impl
│   │   ├── CameraMetadataNative.aidl
│   │   └── CaptureResultExtras.aidl
│   ├── params
│   │   ├── OutputConfiguration.aidl
│   │   ├── VendorTagDescriptor.aidl
│   │   └── VendorTagDescriptorCache.aidl
│   └── utils
│       └── SubmitInfo.aidl
├── CameraInfo.aidl
├── CameraStatus.aidl
├── ICamera.aidl
├── ICameraClient.aidl
├── ICameraService.aidl
├── ICameraServiceListener.aidl
└── ICameraServiceProxy.aidl

4 directories, 16 files

frameworks/av/camera/aidl/ 目录下的 aidl 文件有两种类型：

  • 作为 Binder 中的 IInterface 跨进程通信中能提供的方法
  • 作为 Binder 中的 parcelable 跨进程通信数据传输的数据结构

很容易从名字上区分这两种类型的文件，IInterface 类型的文件都是以 I 开头的，比如：ICameraService.aidl, ICameraDeviceUser.aidl 等。不管是哪种类型的 aidl 文件，它们都会生成对应的 .java, .h, .cpp 文件，分别供 Java 层和 CPP 层调用。

IInterface 类型文件

IInterface 类型文件一共有 7 个，它们的 .java, .h, .cpp 文件，绝大部分都是自动生成的。

Java 文件是在 frameworks/base/Android.mk 中定义规则，在编译时自动生成：

1
2
3
4
5
6
7
8
9
10
11

// frameworks/base/Android.mk
LOCAL_SRC_FILES += 
    ...
    ../av/camera/aidl/android/hardware/ICameraService.aidl 
    ../av/camera/aidl/android/hardware/ICameraServiceListener.aidl 
    ../av/camera/aidl/android/hardware/ICameraServiceProxy.aidl 
    ../av/camera/aidl/android/hardware/ICamera.aidl 
    ../av/camera/aidl/android/hardware/ICameraClient.aidl 
    ../av/camera/aidl/android/hardware/camera2/ICameraDeviceUser.aidl 
    ../av/camera/aidl/android/hardware/camera2/ICameraDeviceCallbacks.aidl 
    ...

在 out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/dotdot/ 目录下生成对应的 Java 文件：

1
2
3
4
5
6
7
8

// out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/dotdot/
av/camera/aidl/android/hardware/ICameraService.java
av/camera/aidl/android/hardware/ICameraServiceListener.java
av/camera/aidl/android/hardware/ICameraServiceProxy.java
av/camera/aidl/android/hardware/ICamera.java
av/camera/aidl/android/hardware/ICameraClient.java
av/camera/aidl/android/hardware/camera2/ICameraDeviceUser.java
av/camera/aidl/android/hardware/camera2/ICameraDeviceCallbacks.java

.h, .cpp 文件中，ICamera.aidl, ICameraClient.aidl 两个文件是直接以代码形式手动实现的：

1
2
3
4
5
6
7
8
9

// 1. ICameraClient.aidl
frameworks/av/camera/aidl/android/hardware/ICameraClient.aidl
frameworks/av/camera/include/camera/android/hardware/ICameraClient.h
frameworks/av/camera/ICameraClient.cpp

// 2. ICamera.aidl
frameworks/av/camera/aidl/android/hardware/ICamera.aidl
frameworks/av/camera/include/camera/android/hardware/ICamera.h
frameworks/av/camera/ICamera.cpp

其他 5 个 aidl 文件是在 frameworks/av/camera/Android.bp 中定义规则，编译时自动生成对应的 .h, .cpp 文件：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

// frameworks/av/camera/Android.bp
cc_library_shared {
    name: "libcamera_client",

    aidl: {
        export_aidl_headers: true,
        local_include_dirs: ["aidl"],
        include_dirs: [
            "frameworks/native/aidl/gui",
        ],
    },

    srcs: [
        // AIDL files for camera interfaces
        // The headers for these interfaces will be 
        // available to any modules that
        // include libcamera_client, at the path "aidl/package/path/BnFoo.h"
        "aidl/android/hardware/ICameraService.aidl",
        "aidl/android/hardware/ICameraServiceListener.aidl",
        "aidl/android/hardware/ICameraServiceProxy.aidl",
        "aidl/android/hardware/camera2/ICameraDeviceCallbacks.aidl",
        "aidl/android/hardware/camera2/ICameraDeviceUser.aidl",


        // Source for camera interface parcelables, 
        // and manually-written interfaces
        "Camera.cpp",
        "CameraMetadata.cpp",
        "CameraParameters.cpp",
        ...
}

在 out/soong/.intermediates/frameworks/av/camera/libcamera_client/ 目录下生成对应的 .h, .cpp 文件，通常在该目录下会同时生成 32 和 64 位两套代码，但实际两份代码是一样的，这里选取 64 位的：

  • 64 位：android_arm64_armv8-a_shared_core
  • 32 位：android_arm_armv7-a-neon_cortex-a53_shared_core
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

// 目录 out/soong/.intermediates/frameworks/av/camera/libcamera_client
// 64 位 android_arm64_armv8-a_shared_core/gen/aidl/
android/hardware/ICameraService.h
android/hardware/BnCameraService.h
frameworks/av/camera/aidl/android/hardware/ICameraService.cpp

android/hardware/ICameraServiceListener.h
android/hardware/BnCameraServiceListener.h
frameworks/av/camera/aidl/android/hardware/ICameraServiceListener.cpp

android/hardware/ICameraServiceProxy.h
android/hardware/BnCameraServiceProxy.h
frameworks/av/camera/aidl/android/hardware/ICameraServiceProxy.cpp

android/hardware/camera2/ICameraDeviceUser.h
android/hardware/camera2/BnCameraDeviceUser.h
frameworks/av/camera/aidl/android/hardware/camera2/ICameraDeviceUser.cpp

android/hardware/camera2/ICameraDeviceCallbacks.h
android/hardware/camera2/BnCameraDeviceCallbacks.h
frameworks/av/camera/aidl/android/hardware/camera2/ICameraDeviceCallbacks.cpp

parcelable 类型文件

parcelable 类型文件一共有 9 个，它们都是手动编写的代码。

Java 文件目录为 frameworks/base/core/java/android/hardware/ ：

1
2
3
4
5
6
7
8
9
10

// frameworks/base/core/java/android/hardware/
camera2/CaptureRequest.java
camera2/impl/CameraMetadataNative.java
camera2/impl/CaptureResultExtras.java
camera2/params/OutputConfiguration.java
camera2/params/VendorTagDescriptor.java
camera2/params/VendorTagDescriptorCache.java
camera2/utils/SubmitInfo.java
CameraInfo.java
CameraStatus.java

.h, .cpp 文件并不一定是和 aidl 文件名称一一对应的，而是在 aidl 文件中定义的，比如 CameraStatus.aidl 定义如下：

1
2
3
4

package android.hardware;

/** @hide */
parcelable CameraStatus cpp_header "camera/CameraBase.h";

parcelable 类型的 aidl 文件对应的 .h, .cpp 文件目录为 frameworks/av/camera ，对应关系整理如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

// .h, .cpp 文件目录 frameworks/av/camera
// CaptureRequest.aidl
include/camera/camera2/CaptureRequest.h
camera2/CaptureRequest.cpp

// CameraMetadataNative.aidl
include/camera/CameraMetadata.h
CameraMetadata.cpp

// CaptureResultExtras.aidl
include/camera/CaptureResult.h
CaptureResult.cpp

// OutputConfiguration.aidl
include/camera/camera2/OutputConfiguration.h
camera2/OutputConfiguration.cpp

// VendorTagDescriptor.aidl 和 VendorTagDescriptorCache.aidl
include/camera/VendorTagDescriptor.h
VendorTagDescriptor.cpp

// SubmitInfo.aidl
include/camera/camera2/SubmitInfo.h
camera2/SubmitInfo.cpp

// CameraInfo.aidl 和 CameraStatus.aidl
include/camera/CameraBase.h
CameraBase.cpp

ICameraService 相关

分为客户端向服务端的请求 ICameraService.aidl 和客户端监听服务端的变化 ICameraServiceListener.aidl 。这两个 AIDL 是在 CameraService.cpp 中实现对应功能的。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

interface 
{
    ...
    const int CAMERA_TYPE_BACKWARD_COMPATIBLE = 0;
    const int CAMERA_TYPE_ALL = 1;

    // 返回指定类型的相机设备数量
    int getNumberOfCameras(int type);
    // 根据 id 返回当前相机设备信息
    CameraInfo getCameraInfo(int cameraId);

    ...
    const int CAMERA_HAL_API_VERSION_UNSPECIFIED = -1;
    // api1 + hal1
    ICamera connect(ICameraClient client,
            int cameraId,
            String opPackageName,
            int clientUid, int clientPid);

    // api2 + hal3
    ICameraDeviceUser connectDevice(ICameraDeviceCallbacks callbacks,
            String cameraId,
            String opPackageName,
            int clientUid);

    // api1 + 指定 hal 版本（通常为 hal1）
    ICamera connectLegacy(ICameraClient client,
            int cameraId,
            int halVersion,
            String opPackageName,
            int clientUid);

    // 添加和移除 ICameraServiceListener 监听
    CameraStatus[] addListener(ICameraServiceListener listener);
    void removeListener(ICameraServiceListener listener);

    // 根据 id 返回相机支持的属性
    CameraMetadataNative getCameraCharacteristics(String cameraId);

    // 获取 vendor tag 
    VendorTagDescriptor getCameraVendorTagDescriptor();
    VendorTagDescriptorCache getCameraVendorTagCache();

    // camera api 1 获取参数信息
    String getLegacyParameters(int cameraId);

    const int API_VERSION_1 = 1;
    const int API_VERSION_2 = 2;
    // 指定 id 支持的 API 版本
    boolean supportsCameraApi(String cameraId, int apiVersion);
    // 指定 id 设置手电筒模式
    void setTorchMode(String cameraId, boolean enabled, 
        IBinder clientBinder);

    // 服务端向系统打印系统消息
    const int EVENT_NONE = 0;
    const int EVENT_USER_SWITCHED = 1;
    oneway void notifySystemEvent(int eventId, in int[] args);
}

// 2. ICameraServiceListener.aidl
interface ICameraServiceListener
{
    const int STATUS_NOT_PRESENT      = 0;
    const int STATUS_PRESENT          = 1;
    const int STATUS_ENUMERATING      = 2;
    const int STATUS_NOT_AVAILABLE    = -2;
    const int STATUS_UNKNOWN          = -1;
    // 相机设备状态变化事件
    oneway void onStatusChanged(int status, String cameraId);

    const int TORCH_STATUS_NOT_AVAILABLE = 0;
    const int TORCH_STATUS_AVAILABLE_OFF = 1;
    const int TORCH_STATUS_AVAILABLE_ON  = 2;
    const int TORCH_STATUS_UNKNOWN = -1;
    // 手电筒状态变化事件
    oneway void onTorchStatusChanged(int status, String cameraId);
}

ICameraServiceProxy.aidl 文件

CameraServiceProxy 服务是在 Java 层注册的：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

interface ICameraServiceProxy
{
    // CameraService 向代理服务发送消息，通知用户更新
    oneway void pingForUserUpdate();

    const int CAMERA_STATE_OPEN = 0;
    const int CAMERA_STATE_ACTIVE = 1;
    const int CAMERA_STATE_IDLE = 2;
    const int CAMERA_STATE_CLOSED = 3;
    const int CAMERA_FACING_BACK = 0;
    const int CAMERA_FACING_FRONT = 1;
    const int CAMERA_FACING_EXTERNAL = 2;

    // CameraService 向代理服务发送消息，通知相机设备状态更新
    oneway void notifyCameraState(String cameraId, int facing, 
            int newCameraState, String clientName);
}

ICamera 相关

Camera API1 才会使用到，分为 ICamera.aidl, ICameraClient.aidl
它们的代码是手动实现的，参考：CameraClient.h/cpp, Camera.h/cpp

ICameraDevice 相关

Camera API2 才会使用到，分为客户端向服务端的请求 ICameraDeviceUser.aidl 和服务端发给客户端的回调 ICameraDeviceCallbacks.aidl 。
表示相机设备具备的能力，能够提供的函数；这两个 AIDL 是在 CameraDeviceClient 中实现对应功能的。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

// 1. ICameraDeviceUser.aidl 
interface ICameraDeviceUser
{
    void disconnect();

    const int NO_IN_FLIGHT_REPEATING_FRAMES = -1;
    // 向设备提交捕获请求
    SubmitInfo submitRequest(in CaptureRequest request, boolean streaming);
    SubmitInfo submitRequestList(in CaptureRequest[] requestList, 
        boolean streaming);
    // 取消置顶 id 的重复请求，并返回上次请求的帧 id
    long cancelRequest(int requestId);

    const int NORMAL_MODE = 0;
    const int CONSTRAINED_HIGH_SPEED_MODE = 1;
    const int VENDOR_MODE_START = 0x8000;

    // 在流处理前执行配置请求
    void beginConfigure();
    // 根据指定输出配置，创建流
    int createStream(in OutputConfiguration outputConfiguration);
    void endConfigure(int operatingMode);
    void deleteStream(int streamId);

    // 创建输入流，返回流 id
    int createInputStream(int width, int height, int format);
    // 返回输入流的 Surface
    Surface getInputSurface();

    // Keep in sync with public API in
    // frameworks/base/core/java/android/hardware/camera2/CameraDevice.java
    const int TEMPLATE_PREVIEW = 1;
    const int TEMPLATE_STILL_CAPTURE = 2;
    const int TEMPLATE_RECORD = 3;
    const int TEMPLATE_VIDEO_SNAPSHOT = 4;
    const int TEMPLATE_ZERO_SHUTTER_LAG = 5;
    const int TEMPLATE_MANUAL = 6;
    // 根据模板创建默认请求，返回相机参数信息
    CameraMetadataNative createDefaultRequest(int templateId);
    // 获取相机参数信息
    CameraMetadataNative getCameraInfo();
    void waitUntilIdle();
    long flush();
    void prepare(int streamId);
    void tearDown(int streamId);
    void prepare2(int maxCount, int streamId);
    void finalizeOutputConfigurations(int streamId, 
        in OutputConfiguration outputConfiguration);
}

// 2. ICameraDeviceCallbacks.aidl
interface ICameraDeviceCallbacks
{
    ...
    oneway void onDeviceError(int errorCode, 
        in CaptureResultExtras resultExtras);
    oneway void onDeviceIdle();
    oneway void onCaptureStarted(in CaptureResultExtras resultExtras, 
        long timestamp);
    oneway void onResultReceived(in CameraMetadataNative result,
                                 in CaptureResultExtras resultExtras);
    oneway void onPrepared(int streamId);
    // 重复请求引起的错误回调
    oneway void onRepeatingRequestError(in long lastFrameNumber,
                                        in int repeatingRequestId);
    oneway void onRequestQueueEmpty();
}

Services 目录下的文件介绍

frameworks/av/services/camera/libcameraservice
AOSP 中这个目录下是 87 个文件，而 Qcom 的基线中增加了 27 个文件，分别为 api1/qticlient2 目录下的 25 个文件，以及 QTICamera2Client.cpp, QTICamera2Client.h 两个文件。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

.
├── Android.mk
├── api1
│   ├── client2
│   └── qticlient2
├── api2
├── CameraFlashlight.cpp
├── CameraFlashlight.h
├── CameraService.cpp
├── CameraService.h
├── common
├── device1
├── device3
├── gui
├── MODULE_LICENSE_APACHE2
├── NOTICE
├── tests
└── utils

从目录结构上可以看出，API1/2 和 HAL1/3 就是在这一层体现的。

API1/API2

APP Java 客户端调用服务端方法时，Camera API1/2 接口对应功能都是在 CameraService 中实现的，而这里的 API1/2 目录对应的就是对上层不同版本接口的处理。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

api1
├── Camera2Client.cpp
├── Camera2Client.h
├── CameraClient.cpp
├── CameraClient.h
├── client2
│   ├── CallbackProcessor.cpp
│   ├── CallbackProcessor.h
│   ├── Camera2Heap.h
│   ├── CaptureSequencer.cpp
│   ├── CaptureSequencer.h
│   ├── FrameProcessor.cpp
│   ├── FrameProcessor.h
│   ├── JpegCompressor.cpp
│   ├── JpegCompressor.h
│   ├── JpegProcessor.cpp
│   ├── JpegProcessor.h
│   ├── Parameters.cpp
│   ├── Parameters.h
│   ├── StreamingProcessor.cpp
│   ├── StreamingProcessor.h
│   ├── ZslProcessor.cpp
│   └── ZslProcessor.h
├── QTICamera2Client.cpp
├── QTICamera2Client.h
└── qticlient2
    ├── CallbackProcessor.cpp
    ├── CallbackProcessor.h
    ├── Camera2Heap.h
    ├── CaptureSequencer.cpp
    ├── CaptureSequencer.h
    ├── FrameProcessor.cpp
    ├── FrameProcessor.h
    ├── JpegCompressor.cpp
    ├── JpegCompressor.h
    ├── JpegProcessor.cpp
    ├── JpegProcessor.h
    ├── Parameters.cpp
    ├── Parameters.h
    ├── QTICaptureSequencer.cpp
    ├── QTICaptureSequencer.h
    ├── QTIFrameProcessor.cpp
    ├── QTIFrameProcessor.h
    ├── QTIParameters.cpp
    ├── QTIParameters.h
    ├── RawProcessor.cpp
    ├── RawProcessor.h
    ├── StreamingProcessor.cpp
    ├── StreamingProcessor.h
    ├── ZslProcessor.cpp
    └── ZslProcessor.h
api2
├── CameraDeviceClient.cpp
└── CameraDeviceClient.h

BasicClient 有三个重要的子类：

  • CameraClient
    如果平台仅支持 HAL 1，即 CAMERA_DEVICE_API_VERSION_1_0 ；使用 API 1/2 + HAL 1 都会对应该客户端。
  • Camera2Client
    如果平台支持 HAL 3 ，即 CAMERA_DEVICE_API_VERSION_3_0 及以上版本；使用 API 1 + HAL 3 对应的客户端。Camera2Client 会将 API1 中的接口转换为 API2 中对应的功能。
  • CameraDeviceClient
    如果平台支持 HAL 3 ，使用 API 2 + HAL 3 对应的客户端。

平台仅支持 HAL 1 时，API 2 在 openCamera 时，通过 CameraDeviceUserShim 将 API 2 转换为 API 1 ，即 HAL 1 + API 1 向下发起请求。
LegacyCameraDevice 会将 CAMERA API2 转换为 CAMERA API1 ，而 CameraDeviceUserShim 封装了 LegacyCameraDevice 。

QTICamera2Client

Qcom 的基线中增加了 27 个文件，分别为 api1/qticlient2 目录下的 25 个文件，以及 QTICamera2Client.cpp, QTICamera2Client.h 两个文件。
而 QTICamera2Client 是高通针对 API1 做的优化？在什么情况下会转换为 QTICamera2Client 呢？看如下源码：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

// 1. Camera2Client.h
class Camera2Client :
        public Camera2ClientBase<CameraService::Client>
{

friend class QTICamera2Client;
#endif
...

    sp<camera2::RawProcessor> mRawProcessor;
#endif
...

    sp<QTICamera2Client> mQTICamera2Client;
#endif
...
}

// 2. Camera2Client.cpp
template<typename TProviderPtr>
status_t Camera2Client::initializeImpl(TProviderPtr providerPtr)
{
...

    mQTICamera2Client = new QTICamera2Client(this);
#endif
...

    mRawProcessor = new RawProcessor(this, mCaptureSequencer);
    threadName = String8::format("C2-%d-RawProc", mCameraId);
    mRawProcessor->run(threadName.string());
#endif
...
}

QTICamera2Client 是高通对 API 1 中 Camera2Client 做的一层封装，添加了部分功能，主要是向上提供 raw 数据。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

// 1. QTICamera2Client.h
class QTICamera2Client: public virtual RefBase{
private:
    wp<Camera2Client> mParentClient;
    status_t stopPreviewExtn();

public:
    QTICamera2Client(sp<Camera2Client> client);
    ~QTICamera2Client();
    ...
}

// 2. QTICamera2Client.cpp
QTICamera2Client::QTICamera2Client(sp<Camera2Client> client):
        mParentClient(client) {
}

device1/device3

device1/device3 可以理解为 Framework 层对应 HAL 层的 HAL 1/3 。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

device1
├── CameraHardwareInterface.cpp
└── CameraHardwareInterface.h
device3
├── Camera3BufferManager.cpp
├── Camera3BufferManager.h
├── Camera3Device.cpp
├── Camera3Device.h
├── Camera3DummyStream.cpp
├── Camera3DummyStream.h
├── Camera3InputStream.cpp
├── Camera3InputStream.h
├── Camera3IOStreamBase.cpp
├── Camera3IOStreamBase.h
├── Camera3OutputStream.cpp
├── Camera3OutputStream.h
├── Camera3OutputStreamInterface.h
├── Camera3SharedOutputStream.cpp
├── Camera3SharedOutputStream.h
├── Camera3StreamBufferFreedListener.h
├── Camera3StreamBufferListener.h
├── Camera3Stream.cpp
├── Camera3Stream.h
├── Camera3StreamInterface.h
├── Camera3StreamSplitter.cpp
├── Camera3StreamSplitter.h
├── StatusTracker.cpp
└── StatusTracker.h
  • API1/device1/HAL1 的连接过程
1
2
3
4
5

// API1: CameraClient.h
sp<CameraHardwareInterface>     mHardware;
// device1: CameraHardwareInterface.h
sp<hardware::camera::device::V1_0::ICameraDevice> mHidlDevice;
// 这里的 ICameraDevice 即为 HAL1

API1 的客户端 CameraClient 对应的 device1: CameraHardwareInterface，而它直接包含了 HAL1 中 ICameraDevice 。

  • API1/3/device3/HAL3 的连接过程 
    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

    // API1: Camera2Client.h
class Camera2Client :
    public Camera2ClientBase<CameraService::Client>{...}

// API2: CameraDeviceClient.h
class CameraDeviceClient :
    public Camera2ClientBase<CameraDeviceClientBase>,
    public camera2::FrameProcessorBase::FilteredListener{...}

// Camera2ClientBase.h
sp<CameraDeviceBase>  mDevice;

// Camera2ClientBase.cpp
template <typename TClientBase>
Camera2ClientBase<TClientBase>::Camera2ClientBase(
    const sp<CameraService>& cameraService,
    const sp<TCamCallbacks>& remoteCallback,
    const String16& clientPackageName,
    const String8& cameraId,
    int cameraFacing,
    int clientPid,
    uid_t clientUid,
    int servicePid):
    TClientBase(cameraService, remoteCallback, clientPackageName,
            cameraId, cameraFacing, clientPid, clientUid, servicePid),
    mSharedCameraCallbacks(remoteCallback),
    mDeviceVersion(cameraService->getDeviceVersion(
            TClientBase::mCameraIdStr)),
    mDeviceActive(false)
{
    ...
    mInitialClientPid = clientPid;
    // 只要是 HAL3 ，则 device 都是对应的 Camera3Device
    mDevice = new Camera3Device(cameraId);
    ...
}

从源码可以看出，不管是 API1/2 ，只要是 HAL 3 ，Camera2Client, CameraDeviceClient 两个客户端对应的都是 device3: Camera3Device 。

Camera3Device::HalInterface 内部类，用于和 HAL 层通信，实现了 HAL 层 ICameraDeviceSession.hal 部分代码：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

// Camera3Device.h
class Camera3Device :
    ...
    class HalInterface : public camera3::Camera3StreamBufferFreedListener {
      public:
        ...
        // Calls into the HAL interface

        // Caller takes ownership of requestTemplate
        status_t constructDefaultRequestSettings(
                camera3_request_template_t templateId,
                /*out*/ camera_metadata_t **requestTemplate);
        status_t configureStreams(
            /*inout*/ camera3_stream_configuration *config);
        status_t processCaptureRequest(camera3_capture_request_t *request);
        status_t flush();
        status_t close();
        ...
    }
    ...
}

cameraserver 进程

cameraserver 进程的源码在 frameworks/av/camera/cameraserver 目录下，该目录只有三个文件：

1
2
3
4

.
├── Android.mk              
├── cameraserver.rc         // rc 文件
└── main_cameraserver.cpp   // 主进程

cameraserver 进程在启动时，做了三件事：

  • 设置 Socket 通信时，对端关闭读取时进程不退出，返回错误信息（Socket 用在了哪？）
  • HIDL 通信初始化
  • Native Binder 初始化，CameraService 向 service_manager 注册服务
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

// frameworks/av/camera/cameraserver/main_cameraserver.cpp
int main(int argc __unused, char** argv __unused)
{
    // 1. Socket 通信时，对端关闭读取时进程不退出，返回错误信息
    signal(SIGPIPE, SIG_IGN);

    // 2. HIDL 通信初始化
    // Set 3 threads for HIDL calls
    hardware::configureRpcThreadpool(3, /*willjoin*/ false);

    // 3. Native Binder 初始化，CameraService 是具体的服务
    sp<ProcessState> proc(ProcessState::self());
    sp<IServiceManager> sm = defaultServiceManager();
    ALOGI("ServiceManager: %p", sm.get());
    CameraService::instantiate();
    ProcessState::self()->startThreadPool();
    IPCThreadState::self()->joinThreadPool();
}

1
2
3
4
5
6
7
8
9
10

// init 进程启动名字为 cameraserver 的进程及对应路径
service cameraserver /system/bin/cameraserver
    // class 表示类别，同一类别的进程同时启动
    class main
    // 用户名
    user cameraserver
    // 分组
    group audio camera input drmrpc
    ioprio rt 4
    writepid /dev/cpuset/camera-daemon/tasks /dev/stune/top-app/tasks

CameraService 启动服务注册流程图：
0109-android-camera-4-framework-CameraService-addService.png

CameraService 服务

CameraService 服务的名称为：media.camera ，主要有两个功能：

  • 作为服务端
    实现 AIDL 对应功能，当 API1/2 客户端发出请求后，作为服务端响应并处理这些功能。
  • 作为客户端
    实现 HIDL 回调，用于响应 HAL 层发回的回调。并且通过 CameraProviderManager 和 HAL 层实现双向通信。

服务名称

CameraService 继承了 BinderService<CameraService> ，将 CameraService::instantiate(); 代码展开：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

// BinderService.h
template<typename SERVICE>
class BinderService
{
public:
    static status_t publish(bool allowIsolated = false) {
        sp<IServiceManager> sm(defaultServiceManager());
        return sm->addService(
                String16(SERVICE::getServiceName()),
                new SERVICE(), allowIsolated);
    }
    ...
    static void instantiate() { publish(); }
    ...
}

// IServiceManager.h
class IServiceManager : public IInterface
{
public:
    ...
    virtual status_t addService( const String16& name,
            const sp<IBinder>& service,
            bool allowIsolated = false) = 0;
    ...
}

// CameraService.h
class CameraService :
    public BinderService<CameraService>,
    public virtual ::android::hardware::BnCameraService,
    public virtual IBinder::DeathRecipient,
    public camera_module_callbacks_t,
    public virtual CameraProviderManager::StatusListener
{
    ...
    // Implementation of BinderService<T>
    static char const* getServiceName() { return "media.camera"; }
    ...
}

从继承关系及 CameraService.h 源码，getServiceName 设置了 CameraService 服务的名称为 media.camera 。

注册流程图

CameraService 注册流程，查看原图

0109-android-camera-4-framework-CameraService-init.jpg

源码分析

先来看 CameraService.h 头文件相关定义：

1
2
3
4
5
6
7
8
9
10
11

// CameraService.h
class CameraService :
    public BinderService<CameraService>,
    public virtual ::android::hardware::BnCameraService,
    public virtual IBinder::DeathRecipient,
    public camera_module_callbacks_t,
    public virtual CameraProviderManager::StatusListener
{

    static char const* getServiceName() { return "media.camera"; }
}
  • BinderService
    继承了 BinderService ，用于注册服务。服务名称为 media.camera 。
  • camera_module_callbacks_t
    继承了 camera_module_callbacks_t ，它是在 HAL 中定义的，用于 HAL 向 Framework 发送通知。
  • StatusListener
    继承了 StatusListener ，它是在 CameraProviderManager.h 中定义的，用于 CameraProviderManager 向 CameraService 发送通知。

现在查看 CameraService 的构造方法，因为在注册服务时 BinderService 会对 CameraService 强指针引用，所以会调用对应函数 onFirstRef ：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

// CameraService.cpp
CameraService::CameraService() :
        mEventLog(DEFAULT_EVENT_LOG_LENGTH),
        mNumberOfCameras(0), mNumberOfNormalCameras(0),
        mSoundRef(0), mInitialized(false) {
    // camera_module_callbacks_t 结构体的函数指针赋值
    this->camera_device_status_change = 
        android::camera_device_status_change;
    this->torch_mode_status_change = android::torch_mode_status_change;
    ...
}

void CameraService::onFirstRef()
{
    ...
    BatteryNotifier& notifier(BatteryNotifier::getInstance());
    notifier.noteResetCamera();
    notifier.noteResetFlashlight();

    status_t res = INVALID_OPERATION;

    // 实例化 CameraProviderManager ，并连接 Hardware
    res = enumerateProviders();
    if (res == OK) {
        mInitialized = true;
    }

    // CameraServiceProxy 服务是 Java 代码注册的
    // 但是 CameraService 启动时间很早，CameraServiceProxy 可能还并没有注册
    // 实际调试结果也是，这段代码实际不会调用 CameraServiceProxy 对应方法
    CameraService::pingCameraServiceProxy();
}

构造函数中非常简单，仅仅是将 camera_module_callbacks_t 结构体的函数指针赋值；在 onFirstRef 中，主要通过 enumerateProviders 来实例化对应的 CameraProviderManager 并连接 HAL ，最后去 ping 一次 CameraServiceProxy 代理服务，实际上是 ping 不通的，因为 CameraService.cpp 一定是比 CameraServiceProxy.java 启动的早。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

// CameraService.cpp
status_t CameraService::enumerateProviders() {
    ...
    // 实例化 CameraProviderManager
    if (nullptr == mCameraProviderManager.get()) {
        mCameraProviderManager = new CameraProviderManager();
        res = mCameraProviderManager->initialize(this);
        ...
    }

    mNumberOfCameras = mCameraProviderManager->getCameraCount();
    mNumberOfNormalCameras =
            mCameraProviderManager->getAPI1CompatibleCameraCount();
    mCameraProviderManager->setUpVendorTags();

    if (nullptr == mFlashlight.get()) {
        mFlashlight = new CameraFlashlight(mCameraProviderManager, this);
    }
    res = mFlashlight->findFlashUnits();
    ...
    for (auto& cameraId : mCameraProviderManager->getCameraDeviceIds()){
        ...
        onDeviceStatusChanged(id8, CameraDeviceStatus::PRESENT);
        ...
    }

    return OK;
}

如果 mCameraProviderManager 为空，则实例化并调用 initialize ；接着实例化 CameraFlashlight ；先看头文件 CameraProviderManager.h 中定义的几个重要数据结构和函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

// CameraProviderManager.h
class CameraProviderManager : 
    virtual public hidl::manager::V1_0::IServiceNotification {
public:
    ...
    // 定义纯虚函数
    struct ServiceInteractionProxy {
        virtual bool registerForNotifications(
                const std::string &serviceName,
                const sp<hidl::manager::V1_0::IServiceNotification>
                &notification) = 0;
        virtual sp<hardware::camera::provider::V2_4::ICameraProvider> 
                getService(const std::string &serviceName) = 0;
        virtual ~ServiceInteractionProxy() {}
    };

    // 调用 ICameraProvider 实现这些方法
    struct HardwareServiceInteractionProxy : 
            public ServiceInteractionProxy {
        virtual bool registerForNotifications(
                const std::string &serviceName,
                const sp<hidl::manager::V1_0::IServiceNotification>
                &notification) override {
            return hardware::camera::provider::V2_4::
                    ICameraProvider::registerForNotifications(
                    serviceName, notification);
        }
        virtual sp<hardware::camera::provider::V2_4::ICameraProvider>
                getService(const std::string &serviceName) override {
            return hardware::camera::provider::V2_4::
                    ICameraProvider::getService(serviceName);
        }
    };

    struct StatusListener : virtual public RefBase {
        ~StatusListener() {}
        virtual void onDeviceStatusChanged(const String8 &cameraId,
                hardware::camera::common::V1_0::CameraDeviceStatus 
                newStatus) = 0;
        virtual void onTorchStatusChanged(const String8 &cameraId,
                hardware::camera::common::V1_0::TorchModeStatus 
                newStatus) = 0;
        virtual void onNewProviderRegistered() = 0;
    };

    virtual hardware::Return<void> onRegistration(
            const hardware::hidl_string& fqName,
            const hardware::hidl_string& name,
            bool preexisting) override;

    status_t initialize(wp<StatusListener> listener,
            ServiceInteractionProxy *proxy 
            = &sHardwareServiceInteractionProxy);

private:
  static HardwareServiceInteractionProxy sHardwareServiceInteractionProxy;
  • ServiceInteractionProxy
    定义了几个纯虚函数，用于向 HAL 系统服务中注册 registerForNotifications 监听 ICameraProvider.hal 的消息；getService 返回 ICameraProvider 的实例。
  • HardwareServiceInteractionProxy
    ServiceInteractionProxy 的实现结构体，具体调用 ICameraProvider 对应的 registerForNotifications, getService ；也就是 CameraProviderManager 持有 ICameraProvider 的远程实例。
  • onRegistration
    registerForNotifications 的回调函数，注册成功后回调。
  • StatusListener
    状态监听接口，这些接口是在 CameraService 中实现的；用于 CameraProviderManager 回调 CameraService 。
  • sHardwareServiceInteractionProxy
    静态变量，是初始化 initialize 函数形参 ServiceInteractionProxy 的默认值。

从 CameraService 中调用 CameraProviderManager::initialize 时，传入的是 CameraService 的实例，仅仅一个参数，所以 ServiceInteractionProxy 使用的是默认的 sHardwareServiceInteractionProxy 实例。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

// CameraProviderManager.cpp
// 实例化 HAL 代理
CameraProviderManager::HardwareServiceInteractionProxy
CameraProviderManager::sHardwareServiceInteractionProxy{};

status_t CameraProviderManager::initialize(
        wp<CameraProviderManager::StatusListener> listener,
        ServiceInteractionProxy* proxy) {
    ...
    mListener = listener;
    mServiceProxy = proxy;

    bool success = mServiceProxy->registerForNotifications(
        /* instance name, empty means no filter */ "",
        this);
    ...

    addProviderLocked(kLegacyProviderName, /*expected*/ false);
    return OK;
}

CameraProviderManager::initialize 中主要是初始化赋值 mListener, mServiceProxy ，并通过 sHardwareServiceInteractionProxy->registerForNotifications 向 HIDL 服务管理注册了自己，最后调用 addProviderLocked 。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

// CameraProviderManager.cpp
status_t CameraProviderManager::addProviderLocked(
        const std::string& newProvider, bool expected) {
    for (const auto& providerInfo : mProviders) {
        if (providerInfo->mProviderName == newProvider) {
            ALOGW(...);
            return ALREADY_EXISTS;
        }
    }

    // HIDL 通信，通过 ICameraProvider 和 HAL 层通信
    sp<provider::V2_4::ICameraProvider> interface;
    interface = mServiceProxy->getService(newProvider);

    if (interface == nullptr) {
        ...
    }

    sp<ProviderInfo> providerInfo =
            new ProviderInfo(newProvider, interface, this);
    status_t res = providerInfo->initialize();
    if (res != OK) {
        return res;
    }

    mProviders.push_back(providerInfo);

    return OK;
}

addProviderLocked 中有如下信息：

  • 通过代理获取 ICameraProvider 实例，用于和 HAL 通信
  • 新建 ProviderInfo 并初始化，保存 ICameraProvider 实例
  • mProviders 保存所有的 ProviderInfo （实测只有一个实例元素，名称为 legacy/0）
1
2
3
4
5
6
7
8
9
10
11
12

// CameraProviderManager.h
struct ProviderInfo :
    virtual public 
        hardware::camera::provider::V2_4::ICameraProviderCallback,
    virtual public hardware::hidl_death_recipient
{
    const std::string mProviderName;
    const sp<hardware::camera::provider::V2_4::ICameraProvider> 
        mInterface;
    const metadata_vendor_id_t mProviderTagid;
    ...
}

ProviderInfo 继承了 ICameraProviderCallback, hidl_death_recipient ，它会处理来着 ICameraProvider 的回调。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

// CameraProviderManager.cpp
CameraProviderManager::ProviderInfo::ProviderInfo(
        const std::string &providerName,
        sp<provider::V2_4::ICameraProvider>& interface,
        CameraProviderManager *manager) :
        mProviderName(providerName),
        mInterface(interface),
        mProviderTagid(generateVendorTagId(providerName)),
        mUniqueDeviceCount(0),
        mManager(manager) {
    (void) mManager;
}

status_t CameraProviderManager::ProviderInfo::initialize() {
    status_t res = parseProviderName(mProviderName, &mType, &mId);
    ...
    // 设置回调
    hardware::Return<Status> status = mInterface->setCallback(this);
    ...
    // HIDL 连接
    hardware::Return<bool> linked = 
        mInterface->linkToDeath(this, /*cookie*/ mId);
    ...
    // Get initial list of camera devices, if any
    std::vector<std::string> devices;
    // 获取 CameraIdList ，实际是获取的一组设备名
    hardware::Return<void> ret = 
        mInterface->getCameraIdList([&status, &devices](
            Status idStatus,
            const hardware::hidl_vec<hardware::hidl_string>& 
                cameraDeviceNames) {
        status = idStatus;
        if (status == Status::OK) {
            for (size_t i = 0; i < cameraDeviceNames.size(); i++) {
                devices.push_back(cameraDeviceNames[i]);
            }
        } });
    ...
    for (auto& device : devices) {
        std::string id;
        // 添加从 HAL 返回的每个设备名
        status_t res = addDevice(device,
            hardware::camera::common::V1_0::CameraDeviceStatus::PRESENT,
            &id);
        ...
    }
    ...
    return OK;
}

ProviderInfo::initialize 初始化，主要是从 HAL 获取设备名后，添加具体的设备信息。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

// CameraProviderManager.h
struct DeviceInfo {
    const std::string mName;  // Full instance name
    const std::string mId;    // ID section of full name
    const hardware::hidl_version mVersion;
    const metadata_vendor_id_t mProviderTagid;
    ...
protected:
    bool mHasFlashUnit;
    template<class InterfaceT>
    static status_t setTorchMode(InterfaceT& interface, bool enabled);
};

// HALv1-specific camera fields, including the actual device interface
struct DeviceInfo1 : public DeviceInfo {
    typedef hardware::camera::device::V1_0::ICameraDevice InterfaceT;
    const sp<InterfaceT> mInterface;
    ...
private:
    CameraParameters2 mDefaultParameters;
};

// HALv3-specific camera fields, including the actual device interface
struct DeviceInfo3 : public DeviceInfo {
    typedef hardware::camera::device::V3_2::ICameraDevice InterfaceT;
    const sp<InterfaceT> mInterface;
    ...
private:
    CameraMetadata mCameraCharacteristics;
};

头文件中可以看出，DeviceInfo 有两个子类，分别对应 HAL 1 和 HAL 3 ，并将具体的 ICameraDevice 版本保存到 mInterface 中；所以设备添加时也会根据不同版本分别添加：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

// CameraProviderManager.cpp
status_t CameraProviderManager::ProviderInfo::addDevice(
        const std::string& name,
        CameraDeviceStatus initialStatus, 
        /*out*/ std::string* parsedId) {
    ...
    status_t res = parseDeviceName(name, &major, &minor, &type, &id);
    ...
    std::unique_ptr<DeviceInfo> deviceInfo;
    switch (major) {
        case 1:
            deviceInfo = initializeDeviceInfo<DeviceInfo1>(
                name, mProviderTagid, id, minor);
            break;
        case 3:
            deviceInfo = initializeDeviceInfo<DeviceInfo3>(
                name, mProviderTagid, id, minor);
            break;
        default:
            ALOGE(...);
            return BAD_VALUE;
    }
    ...
    return OK;
}

template<class DeviceInfoT>
std::unique_ptr<CameraProviderManager::ProviderInfo::DeviceInfo>
    CameraProviderManager::ProviderInfo::initializeDeviceInfo(
        const std::string &name, const metadata_vendor_id_t tagId,
        const std::string &id, uint16_t minorVersion) const {
    Status status;

    auto cameraInterface =
            getDeviceInterface<typename DeviceInfoT::InterfaceT>(name);
    if (cameraInterface == nullptr) return nullptr;

    CameraResourceCost resourceCost;
    cameraInterface->getResourceCost([&status, &resourceCost](
        Status s, CameraResourceCost cost) {
                status = s;
                resourceCost = cost;
            });
    ...
    return std::unique_ptr<DeviceInfo>(
        new DeviceInfoT(name, tagId, id, minorVersion, resourceCost,
                cameraInterface));
}

根据传入的 deviceName 解析版本号、类型、设备 Id （前后摄），并根据 major 版本号（表示 HAL 1 或者 HAL 3） 分别初始化对应的 DeviceInfo ；在 initializeDeviceInfo 中通过 getDeviceInterface 获取对应的 ICameraDevice 版本，在对应版本 DeviceInfo 实例化时保存；也就是将 DeviceInfo 和 HAL 层的 ICameraDevice 绑定。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

// CameraProviderManager.cpp
CameraProviderManager::ProviderInfo::DeviceInfo3::DeviceInfo3(
        const std::string& name,
        const metadata_vendor_id_t tagId, const std::string &id,
        uint16_t minorVersion,
        const CameraResourceCost& resourceCost,
        sp<InterfaceT> interface) :
        DeviceInfo(name, tagId, id, 
            hardware::hidl_version{3, minorVersion}, resourceCost),
        mInterface(interface) {
    // Get camera characteristics and initialize flash unit availability
    Status status;
    hardware::Return<void> ret;
    // 获取 Camera 设备配置信息
    ret = mInterface->getCameraCharacteristics([&status, this](Status s,
                device::V3_2::CameraMetadata metadata) {
            status = s;
            if (s == Status::OK) {
                camera_metadata_t *buffer =
                    reinterpret_cast<camera_metadata_t*>(metadata.data());
                size_t expectedSize = metadata.size();
                int res = validate_camera_metadata_structure(buffer, 
                    &expectedSize);
                if (res==OK||res==CAMERA_METADATA_VALIDATION_SHIFTED) {
                    set_camera_metadata_vendor_id(buffer, mProviderTagid);
                    mCameraCharacteristics = buffer;
                } else {
                    ALOGE(...);
                    status = Status::INTERNAL_ERROR;
                }
            }
        });
    ...
    camera_metadata_entry flashAvailable =
            mCameraCharacteristics.find(ANDROID_FLASH_INFO_AVAILABLE);
    if (flashAvailable.count == 1 &&
            flashAvailable.data.u8[0] == ANDROID_FLASH_INFO_AVAILABLE_TRUE) {
        mHasFlashUnit = true;
    } else {
        mHasFlashUnit = false;
    }
}

这里分析的是 DeviceInfo3 的构造函数，它会向 HAL 层请求当前设备的配置信息，并保存 mCameraCharacteristics ，后续查看属性时都会通过这个变量查询。
CameraService::enumerateProviders 中，首先新建并初始化 CameraProviderManager ，其持有和 HAL 通信的实例；接着新建并初始化 CameraFlashlight ，用于控制闪光灯。先看头文件：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

// CameraFlashlight.h
class FlashControlBase : public virtual VirtualLightRefBase {
public:
    ...
    virtual status_t hasFlashUnit(const String8& cameraId,
                bool *hasFlash) = 0;
    virtual status_t setTorchMode(const String8& cameraId,
                bool enabled) = 0;
};

// HAL 3 闪光灯控制
class ProviderFlashControl : public FlashControlBase {
public:
    ...
    // FlashControlBase
    status_t hasFlashUnit(const String8& cameraId, bool *hasFlash);
    status_t setTorchMode(const String8& cameraId, bool enabled);
private:
    sp<CameraProviderManager> mProviderManager;
...
};

// HAL 1 闪光灯控制，通过 CameraHardwareInterface 向下调用
class CameraHardwareInterfaceFlashControl : public FlashControlBase {
public:
    ...
    // FlashControlBase
    status_t setTorchMode(const String8& cameraId, bool enabled);
    status_t hasFlashUnit(const String8& cameraId, bool *hasFlash);
private:
    sp<CameraProviderManager> mProviderManager;
    const camera_module_callbacks_t *mCallbacks;
    sp<CameraHardwareInterface> mDevice;
    String8 mCameraId;
    CameraParameters mParameters;
    ...
}

class CameraFlashlight : public virtual VirtualLightRefBase {
public:
    ...
    bool hasFlashUnit(const String8& cameraId);
    status_t setTorchMode(const String8& cameraId, bool enabled);
private:
    sp<FlashControlBase> mFlashControl;
    sp<CameraProviderManager> mProviderManager;
    const camera_module_callbacks_t *mCallbacks;

头文件定义的几个信息：

  • CameraHardwareInterfaceFlashControl
    HAL 1 闪光灯控制类，通过 CameraHardwareInterface 向下调用。
  • ProviderFlashControl
    HAL 3 闪光灯控制类。
  • FlashControlBase
    基类。
  • CameraProviderManager
    主要用于 ProviderFlashControl 向下发送信息。
  • camera_module_callbacks_t
    HAL 层的回调。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

// CameraFlashlight.cpp
CameraFlashlight::CameraFlashlight(
        sp<CameraProviderManager> providerManager,
        camera_module_callbacks_t* callbacks) :
        mProviderManager(providerManager),
        mCallbacks(callbacks),
        mFlashlightMapInitialized(false) {
}

status_t CameraFlashlight::findFlashUnits() {
    ...
    mFlashControl.clear();
    for (auto &id : cameraIds) {
        ssize_t index = mHasFlashlightMap.indexOfKey(id);
        if (0 <= index) {
            continue;
        }

        bool hasFlash = false;
        res = createFlashlightControl(id);
        ...
    }
    ...
    return OK;
}

status_t CameraFlashlight::createFlashlightControl(
        const String8& cameraId) {
    ...
    if (mProviderManager->supportSetTorchMode(cameraId.string())) {
        mFlashControl = new ProviderFlashControl(mProviderManager);
    } else {
        // Only HAL1 devices do not support setTorchMode
        mFlashControl =
            new CameraHardwareInterfaceFlashControl(mProviderManager,
            *mCallbacks);
    }
    return OK;
}

CameraFlashlight 的构造函数仅仅初始化了几个本地变量，CameraService 中调用 CameraFlashlight::findFlashUnits 时，会根据 HAL 1/3 分别来创建对应的闪光灯控制类。至此整个 CameraService 注册流程结束。

小结

CameraService 初始化和注册流程中，实例化了两个对象：

  • CameraProviderManager mCameraProviderManager 对象
  • Flashlight mFlashlight 对象

CameraProviderManager 初始化完后：

  • mProviders 保存了 ProviderInfo 对象；并关联了 ICameraProvider ，用于和 HAL 通信
  • ProviderInfo 中 mDevices 保存了所有的 DeviceInfo1, DeviceInfo3 设备信息，并关联 ICameraDevice 实例，用于直接通信
  • DeviceInfo1 中保存了 CameraParameters2 mDefaultParameters 参数信息
  • DeviceInfo3 中保存了 CameraMetadata mCameraCharacteristics 参数信息

CameraFlashlight 新建和初始化后：

  • 如果是 HAL 1 会实例化控制类 CameraHardwareInterfaceFlashControl
  • 如果是 HAL 3 会实例化控制类 ProviderFlashControl

Camera Open 流程

API

Camera API 2 开启摄像头设备时，通过 CameraManager.openCamera 来打开：

1
2
3
4
5
6
7
8
9

// CameraManager.java
@RequiresPermission(android.Manifest.permission.CAMERA)
public void openCamera(@NonNull String cameraId,
        @NonNull final CameraDevice.StateCallback callback,
        @Nullable Handler handler)
        throws CameraAccessException {

    openCameraForUid(cameraId, callback, handler, USE_CALLING_UID);
}
  • String cameraId
    表示前后摄的 ID ，通常 0 表示后摄。
  • CameraDevice.StateCallback callback
    打开设备时，状态回调接口。
  • Handler handler
    表示回调接口在哪个线程执行。

示例

打开一个设备，在回调中保存 CameraDevice ：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

CameraDevice.StateCallback mCameraDeviceStateCallback 
        = new CameraDevice.StateCallback() {
    @Override
    public void onOpened(@NonNull CameraDevice camera) {
        mCameraDevice = camera;
        //createCameraCaptureSession();
    }

    @Override
    public void onDisconnected(@NonNull CameraDevice camera) {
        camera.close();
        mCameraDevice = null;
    }

    @Override
    public void onError(@NonNull CameraDevice camera, int error) {
        camera.close();
        mCameraDevice = null;
    }
};

try {
    mCameraManager.openCamera(mCameraId, mCameraDeviceStateCallback,
            mBackHandler);
} catch (CameraAccessException e) {
    e.printStackTrace();
}

CameraDevice.StateCallback 接口

在打开设备时，会传入 StateCallback 回调接口，它有四个方法，都是在 CameraDeviceImpl 中回调的：

  • onOpened
    在 CameraManager.openCameraDeviceUserAsync 方法中，CameraDeviceImpl.setRemoteDevice(cameraUser); 会触发 StateCallback.onOpened 回调。
  • onClosed
    CameraDevice.close 是在 CameraDeviceImpl.close 中实现的，同时会触发 StateCallback.onClosed 回调。
  • onDisconnected
    CameraDeviceImpl.setRemoteDevice(cameraUser); 中如果远程连接断开，或者 ICameraDeviceCallbacks.onDeviceError 返回了 ERROR_CAMERA_DISCONNECTED 错误码，都会触发 StateCallback.onDisconnected 回调。
  • onError
    在 Binder 通信中绑定失败 binderDied ，setRemoteFailure 以及 ICameraDeviceCallbacks.onDeviceError 返回了 ERROR_CAMERA_DEVICE/ERROR_CAMERA_SERVICE 错误码，都会触发 StateCallback.onError 回调。

在设备打开时，会通过 StateCallback 回调返回打开状态，从代码可以看出，只要 ICameraService.connectDevice 成功后，直接调用 CameraDeviceImpl.setRemoteDevice(cameraUser); 来触发 StateCallback.onOpened ，表示设备打开成功。
StateCallback 是 Java 接口，它的 onDisconnected, onError 两个回调方法，需要真实的与物理设备交互；所以需要通过 ICameraDeviceCallbacks.aidl 从 Framework Service 中获取真实的信息回调。

流程图

Camera API 2 开启相机设备流程图：

0109-android-camera-4-framework-Camera2-open.png

0109-android-camera-4-framework-Camera2-open-CameraService_connectDevice.jpg

源码分析

通过 CameraManager.openCamera 打开设备，我们重点分析如下代码，代码执行路径为 ：
openCamera -> openCameraForUid -> openCameraDeviceUserAsync

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

// CameraManager.java
@RequiresPermission(android.Manifest.permission.CAMERA)
public void openCamera(@NonNull String cameraId,
        @NonNull final CameraDevice.StateCallback callback, 
        @Nullable Handler handler)
        throws CameraAccessException {

    openCameraForUid(cameraId, callback, handler, USE_CALLING_UID);
}

private CameraDevice openCameraDeviceUserAsync(String cameraId,
        CameraDevice.StateCallback callback, 
        Handler handler, final int uid)
        throws CameraAccessException {
    CameraCharacteristics characteristics = 
        getCameraCharacteristics(cameraId);
    CameraDevice device = null;

    synchronized (mLock) {
        ICameraDeviceUser cameraUser = null;
        // 新建 CameraDeviceImpl 实例
        android.hardware.camera2.impl.CameraDeviceImpl deviceImpl =
                new android.hardware.camera2.impl.CameraDeviceImpl(
                    cameraId,
                    callback,
                    handler,
                    characteristics,
                    mContext.getApplicationInfo().targetSdkVersion);
        // 获取 ICameraDeviceCallbacks 回调
        ICameraDeviceCallbacks callbacks = deviceImpl.getCallbacks();

        try {
            if (supportsCamera2ApiLocked(cameraId)) {
                // Use cameraservice's cameradeviceclient 
                // implementation for HAL3.2+ devices
                ICameraService cameraService = 
                    CameraManagerGlobal.get().getCameraService();
                if (cameraService == null) {
                    throw new ServiceSpecificException(...);
                }
                // 连接设备，并获取 ICameraDeviceUser
                cameraUser = cameraService.connectDevice(callbacks,
                        cameraId, mContext.getOpPackageName(), uid);
            } else {
                // Use legacy camera implementation for HAL1 devices
                int id;
                try {
                    id = Integer.parseInt(cameraId);
                } catch (NumberFormatException e) {
                    throw new IllegalArgumentException(...);
                }

                Log.i(TAG, "Using legacy camera HAL.");
                cameraUser = 
                    CameraDeviceUserShim.connectBinderShim(callbacks, id);
            }
        } catch (ServiceSpecificException e) {
            ...
        } catch (RemoteException e) {
            ...
        }
        // 关联 CameraDeviceImpl 和 ICameraDeviceUser ，方便直接通信
        deviceImpl.setRemoteDevice(cameraUser);
        device = deviceImpl;
    }

    return device;
}

// CameraDeviceImpl.java
public void setRemoteDevice(ICameraDeviceUser remoteDevice) 
    throws CameraAccessException {
    synchronized(mInterfaceLock) {
        ...
        // 新建包装类，包装接口并处理对应访问异常
        mRemoteDevice = new ICameraDeviceUserWrapper(remoteDevice);
        ...
    }
}

从上面展示的 API 部分代码中可以看出：

  • 支持 API 2
    如果系统开启了 HAL 3 ，则支持 API 2 ；此时通过 ICameraService 访问服务。
  • 不支持 API 2
    如果系统仅支持 HAL 1 ，则 API 2 需要通过 CameraDeviceUserShim 转换为对应的 API 1 + HAL 1 来实现对应功能。CameraDeviceUserShim 是 ICameraDeviceUser 的实现类；整个 frameworks/base/core/java/android/hardware/camera2/legacy 目录下的代码都是为了实现这个转换功能。

整个打开设备的动作有如下功能：

  • 新建了 CameraDeviceImpl 实例，它是 CameraDevice 的实现类
  • CameraManager 通过 CameraService.connectDevice 连接设备，获取到 ICameraDeviceUser, ICameraDeviceCallbacks 对象，它们用于后续 CameraDeviceImpl.java 和 CameraDeviceClient.cpp 绑定通信
  • 新建 ICameraDeviceUserWrapper 实例，它是对 ICameraDeviceUser 的包装类，捕获并处理远程访问异常等

这里需要重点分析 connectDevice ：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

// CameraService.cpp
Status CameraService::connectDevice(
        const sp<hardware::camera2::ICameraDeviceCallbacks>& cameraCb,
        const String16& cameraId,
        const String16& clientPackageName,
        int clientUid,
        /*out*/
        sp<hardware::camera2::ICameraDeviceUser>* device) {

    ATRACE_CALL();
    Status ret = Status::ok();
    String8 id = String8(cameraId);
    sp<CameraDeviceClient> client = nullptr;
    ret = connectHelper<hardware::camera2::ICameraDeviceCallbacks,
            CameraDeviceClient>(cameraCb, id,
            CAMERA_HAL_API_VERSION_UNSPECIFIED, clientPackageName,
            clientUid, USE_CALLING_PID, API_2,
            /*legacyMode*/ false, /*shimUpdateOnly*/ false,
            /*out*/client);

    ...
    *device = client;
    return ret;
}

template<class CALLBACK, class CLIENT>
Status CameraService::connectHelper(const sp<CALLBACK>& cameraCb,
        const String8& cameraId, int halVersion,
        const String16& clientPackageName, int clientUid, int clientPid,
        apiLevel effectiveApiLevel, bool legacyMode, bool shimUpdateOnly,
        /*out*/sp<CLIENT>& device) {
    ...
    {
        sp<BasicClient> tmp = nullptr;
        if(!(ret = makeClient(this, cameraCb, clientPackageName, 
                cameraId, facing, clientPid, clientUid, getpid(),
                legacyMode, halVersion, deviceVersion, effectiveApiLevel,
                /*out*/&tmp)).isOk()) {
            return ret;
        }
        ...
        err = client->initialize(mCameraProviderManager);
        ...
        if (shimUpdateOnly) {
            mServiceLock.unlock();
            client->disconnect();
            mServiceLock.lock();
        } else {
            // Otherwise, add client to active clients list
            finishConnectLocked(client, partial);
        }
    } // lock is destroyed, allow further connect calls

    device = client;
    return ret;
}

CameraService::connectDevice 函数调用了模板函数 connectHelper ，而该模板主要的两个功能就是：makeClient 新建客户端，initialize 初始化客户端。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

Status CameraService::makeClient(const sp<CameraService>& cameraService,
        const sp<IInterface>& cameraCb, const String16& packageName,
        const String8& cameraId, int facing, int clientPid, 
        uid_t clientUid, int servicePid, bool legacyMode,
        int halVersion, int deviceVersion, apiLevel effectiveApiLevel,
        /*out*/sp<BasicClient>* client) {

    if (halVersion < 0 || halVersion == deviceVersion) {
        switch(deviceVersion) {
          case CAMERA_DEVICE_API_VERSION_1_0:
            if (effectiveApiLevel == API_1) {  // Camera1 API route
                sp<ICameraClient> tmp = 
                    static_cast<ICameraClient*>(cameraCb.get());
                *client = new CameraClient(cameraService, tmp, 
                    packageName, cameraIdToInt(cameraId), facing, 
                    clientPid, clientUid, getpid(), legacyMode);
            } else { // Camera2 API route
                ALOGW("Camera using old HAL version: %d", deviceVersion);
                return STATUS_ERROR_FMT(...);
            }
            break;
          case CAMERA_DEVICE_API_VERSION_3_0:
          case CAMERA_DEVICE_API_VERSION_3_1:
          case CAMERA_DEVICE_API_VERSION_3_2:
          case CAMERA_DEVICE_API_VERSION_3_3:
          case CAMERA_DEVICE_API_VERSION_3_4:
            if (effectiveApiLevel == API_1) { // Camera1 API route
                sp<ICameraClient> tmp = 
                    static_cast<ICameraClient*>(cameraCb.get());
                *client = new Camera2Client(cameraService, tmp, 
                    packageName, cameraIdToInt(cameraId), facing,
                    clientPid, clientUid, servicePid, legacyMode);
            } else { // Camera2 API route
                sp<hardware::camera2::ICameraDeviceCallbacks> tmp =
                        static_cast<hardware::camera2::
                            ICameraDeviceCallbacks*>(cameraCb.get());
                *client = new CameraDeviceClient(cameraService, tmp, 
                    packageName, cameraId, facing, clientPid,
                    clientUid, servicePid);
            }
            break;
          default:
            // Should not be reachable
            ALOGE("Unknown camera device HAL version:%d", deviceVersion);
            return STATUS_ERROR_FMT(...);
        }
    } else {
        if (deviceVersion > CAMERA_DEVICE_API_VERSION_1_0 &&
            halVersion == CAMERA_DEVICE_API_VERSION_1_0) {
            sp<ICameraClient> tmp = 
                static_cast<ICameraClient*>(cameraCb.get());
            *client = new CameraClient(cameraService, tmp, packageName,
                cameraIdToInt(cameraId), facing, clientPid, 
                clientUid, servicePid, legacyMode);
        } else {
            ALOGE("Invalid camera HAL version ..");
            return STATUS_ERROR_FMT(...;
        }
    }
    return Status::ok();
}

makeClient 主要是根据 device, HAL 版本和调用 API 的版本来创建对应的客户端：

  • HAL 1 + API 1 ：新建 CameraClient
  • HAL 1 + API 2 ：不支持
  • HAL 3 + API 1 ：新建 Camera2Client
  • HAL 3 + API 2 ：新建 CameraDeviceClient

这里的三个变量 effectiveApiLevel, legacyMode=0, halVersion ，主要是有三个连接函数决定： connect, connectLegacy, connectDevice ，其中 connectLegacy 可以指定 HAL 版本（来决定到底使用哪个 client）：

  • 使用系统自带相机
    effectiveApiLevel=1, legacyMode=1, halVersion=256(HAL 1) ，系统自带应用使用的是 connectLegacy 。
  • 使用标准 API2 接口
    effectiveApiLevel=2, legacyMode=0, halVersion=-1 ，其中 -1 表示 CAMERA_HAL_API_VERSION_UNSPECIFIED 。

所谓的 HAL 版本，实际指的就是 Device 的版本：其中 HAL 1 对应 CAMERA_DEVICE_API_VERSION_1_0 ；HAL 3 对应的是 CAMERA_DEVICE_API_VERSION_3_0 及以上版本。而 HAL 2 和 CAMERA_DEVICE_API_VERSION_2_0 已经废弃。
因为手机平台使用 HAL 3 时，为了满足部分应用中使用了 API 1 的接口，常常需要兼容 HAL 1，所以支持 HAL 3 即意味着同时会支持 HAL 1 。

这里流程跟踪的是新建 CameraDeviceClient ，先看头文件：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

// CameraService.h
class BasicClient : public virtual RefBase {
public:
    virtual status_t initialize(sp<CameraProviderManager> manager) = 0;
    ...
private:
    ...
    class OpsCallback : public BnAppOpsCallback {
    public:
        explicit OpsCallback(wp<BasicClient> client);
        virtual void opChanged(int32_t op, const String16& packageName);

    private:
        wp<BasicClient> mClient;

    }; // class OpsCallback

    sp<OpsCallback> mOpsCallback;
    ...
}

// Camera3Device.h
class Camera3Device :
    public CameraDeviceBase,
    virtual public hardware::camera::device::V3_2::ICameraDeviceCallback,
    private camera3_callback_ops 
{...}

// Camera2ClientBase.h
template <typename TClientBase>
class Camera2ClientBase :
        public TClientBase,
        public CameraDeviceBase::NotificationListener
{
public:
    typedef typename TClientBase::TCamCallbacks TCamCallbacks;
    ...
protected:
    // 实例为 Camera3Device
    sp<CameraDeviceBase>  mDevice;
    ...
private:
    ...
    template<typename TProviderPtr>
    status_t              initializeImpl(TProviderPtr providerPtr);
};

// CameraDeviceClient.h
struct CameraDeviceClientBase :
         public CameraService::BasicClient,
         public hardware::camera2::BnCameraDeviceUser
{
    typedef hardware::camera2::ICameraDeviceCallbacks TCamCallbacks;
    ...

protected:
    CameraDeviceClientBase(const sp<CameraService>& cameraService,
        const sp<hardware::camera2::ICameraDeviceCallbacks>& 
            remoteCallback,
        const String16& clientPackageName,
        const String8& cameraId,
        int cameraFacing,
        int clientPid,
        uid_t clientUid,
        int servicePid);

    sp<hardware::camera2::ICameraDeviceCallbacks> mRemoteCallback;
};

class CameraDeviceClient :
        public Camera2ClientBase<CameraDeviceClientBase>,
        public camera2::FrameProcessorBase::FilteredListener
{...}

从类图结构来看：BasicClient 是三个客户端 CameraClient, Camera2Client, CameraDeviceClient 的基类；而 Camera2ClientBase 中的变量 CameraDeviceBase 实际的子类是 Camera3Device 。来看构造函数的流程：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80

// CameraDeviceClient.cpp
CameraDeviceClient::CameraDeviceClient(
    const sp<CameraService>& cameraService,
    const sp<hardware::camera2::ICameraDeviceCallbacks>& remoteCallback,
    const String16& clientPackageName,
    const String8& cameraId,
    int cameraFacing,
    int clientPid,
    uid_t clientUid,
    int servicePid) :
    Camera2ClientBase(cameraService, remoteCallback, clientPackageName,
            cameraId, cameraFacing, clientPid, clientUid, servicePid),
    mInputStream(),
    mStreamingRequestId(REQUEST_ID_NONE),
    mRequestIdCounter(0),
    mPrivilegedClient(false) {
    ...
}

// Camera2ClientBase.cpp
template <typename TClientBase>
Camera2ClientBase<TClientBase>::Camera2ClientBase(
    const sp<CameraService>& cameraService,
    const sp<TCamCallbacks>& remoteCallback,
    const String16& clientPackageName,
    const String8& cameraId,
    int cameraFacing,
    int clientPid,
    uid_t clientUid,
    int servicePid):
    TClientBase(cameraService, remoteCallback, clientPackageName,
            cameraId, cameraFacing, clientPid, clientUid, servicePid),
    mSharedCameraCallbacks(remoteCallback),
    mDeviceVersion(cameraService->getDeviceVersion(
        TClientBase::mCameraIdStr)),
    mDeviceActive(false)
{
    ...
    // 实例化 Camera3Device
    mDevice = new Camera3Device(cameraId);
}

// CameraDeviceClient.cpp
CameraDeviceClientBase::CameraDeviceClientBase(
    const sp<CameraService>& cameraService,
    const sp<hardware::camera2::ICameraDeviceCallbacks>& remoteCallback,
    const String16& clientPackageName,
    const String8& cameraId,
    int cameraFacing,
    int clientPid,
    uid_t clientUid,
    int servicePid) :
    BasicClient(cameraService,
            IInterface::asBinder(remoteCallback),
            clientPackageName,
            cameraId,
            cameraFacing,
            clientPid,
            clientUid,
            servicePid),
    mRemoteCallback(remoteCallback) {
}

// CameraService.cpp
CameraService::BasicClient::BasicClient(
    const sp<CameraService>& cameraService,
    const sp<IBinder>& remoteCallback,
    const String16& clientPackageName,
    const String8& cameraIdStr, int cameraFacing,
    int clientPid, uid_t clientUid,
    int servicePid):
    mCameraIdStr(cameraIdStr), mCameraFacing(cameraFacing),
    mClientPackageName(clientPackageName), 
    mClientPid(clientPid), mClientUid(clientUid),
    mServicePid(servicePid),
    mDisconnected(false),
    mRemoteBinder(remoteCallback)
{
    ...
}

根据类继承关系，一条链路实例化各个子类，最终会新建 Camera3Device 实例。makeClient 新建完客户端后，调用客户端的初始化：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

// CameraDeviceClient.cpp
template<typename TProviderPtr>
status_t CameraDeviceClient::initializeImpl(TProviderPtr providerPtr) {
    ...
    res = Camera2ClientBase::initialize(providerPtr);
    ...
    String8 threadName;
    mFrameProcessor = new FrameProcessorBase(mDevice);
    threadName = String8::format("CDU-%s-FrameProc", 
        mCameraIdStr.string());
    mFrameProcessor->run(threadName.string());

    mFrameProcessor->registerListener(FRAME_PROCESSOR_LISTENER_MIN_ID,
                                      FRAME_PROCESSOR_LISTENER_MAX_ID,
                                      /*listener*/this,
                                      /*sendPartials*/true);

    return OK;
}

CameraDeviceClient::initializeImpl 是一个模板函数，主要有两个功能：调用 Camera2ClientBase 及其父类初始化；新建 FrameProcessorBase 实例，它主要功能是在发出预览、拍照、录像请求后，HAL 层向 Framework 层返回结果的回调类，后面讲预览流程时会详细分析。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

// Camera2ClientBase.cpp
template <typename TClientBase>
status_t Camera2ClientBase<TClientBase>::initialize(
    sp<CameraProviderManager> manager) {
    return initializeImpl(manager);
}

template <typename TClientBase>
template <typename TProviderPtr>
status_t Camera2ClientBase<TClientBase>::initializeImpl(
    TProviderPtr providerPtr) {
    ...
    res = mDevice->initialize(providerPtr);
    ...
    return OK;
}

Camera2ClientBase::initialize 也是一个模板函数，最终会调用 Camera3Device 的初始化：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

// Camera3Device.cpp
status_t Camera3Device::initialize(sp<CameraProviderManager> manager) {
    ...
    sp<ICameraDeviceSession> session;
    // 打开会话，获取 ICameraDeviceSession
    status_t res = manager->openSession(mId.string(), this,
            /*out*/ &session);
    ...
    // 获取当前设备的配置信息，并保存到 CameraMetadata mDeviceInfo
    res = manager->getCameraCharacteristics(mId.string(), &mDeviceInfo);
    ...
    // 通过 ICameraDeviceSession 获取请求队列
    std::shared_ptr<RequestMetadataQueue> queue;
    auto requestQueueRet = session->getCaptureRequestMetadataQueue(
        [&queue](const auto& descriptor) {
            queue = std::make_shared<RequestMetadataQueue>(descriptor);
            ...
        });
    ...
    // 通过 ICameraDeviceSession 获取结果队列
    std::unique_ptr<ResultMetadataQueue>& resQueue=mResultMetadataQueue;
    auto resultQueueRet = session->getCaptureResultMetadataQueue(
        [&resQueue](const auto& descriptor) {
            resQueue=std::make_unique<ResultMetadataQueue>(descriptor);
            ...
        });
    ...
    // 新建 HalInterface 实例，并绑定 ICameraDeviceSession 以及请求队列
    mInterface = new HalInterface(session, queue);
    ...
    return initializeCommonLocked();
}

Camera3Device::initialize 初始化中，重点实现的功能为打开物理设备，并获取 ICameraDeviceSession 用于后续直接和 HAL 通信，并通过它从 HAL 获取请求队列和结果队列；最后新建 HalInterface 实例，并将 ICameraDeviceSession 保存并绑定。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

// CameraProviderManager.cpp
status_t CameraProviderManager::openSession(const std::string &id,
    const sp<hardware::camera::device::V3_2::ICameraDeviceCallback>&
    callback, /*out*/
    sp<hardware::camera::device::V3_2::ICameraDeviceSession> *session){

    std::lock_guard<std::mutex> lock(mInterfaceMutex);

    auto deviceInfo = findDeviceInfoLocked(id,
            /*minVersion*/ {3,0}, /*maxVersion*/ {4,0});
    if (deviceInfo == nullptr) return NAME_NOT_FOUND;

   auto *deviceInfo3=static_cast<ProviderInfo::DeviceInfo3*>(deviceInfo);

    Status status;
    hardware::Return<void> ret;
    // 向 HAL 打开设备，并返回 ICameraDeviceSession
    ret = deviceInfo3->mInterface->open(callback, 
        [&status, &session] (Status s, 
        const sp<device::V3_2::ICameraDeviceSession>& cameraSession) {
            status = s;
            if (status == Status::OK) {
                *session = cameraSession;
            }
        });
    ...
}

CameraProviderManager::openSession 打开设备时，会向 HAL 打开设备，将 ICameraDeviceCallback 传入 HAL 并获取 ICameraDeviceSession 实例。接着看 Camera3Device::initializeCommonLocked ：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

// Camera3Device.cpp
status_t Camera3Device::initializeCommonLocked() {

    /** Start up status tracker thread */
    mStatusTracker = new StatusTracker(this);
    status_t res = mStatusTracker->run(
        String8::format("C3Dev-%s-Status", mId.string()).string());
    ...

    /** Register in-flight map to the status tracker */
    mInFlightStatusId = mStatusTracker->addComponent();

    /** Create buffer manager */
    mBufferManager = new Camera3BufferManager();

    mTagMonitor.initialize(mVendorTagId);

    /** Start up request queue thread */
    mRequestThread = new RequestThread(this, mStatusTracker, mInterface);
    res = mRequestThread->run(
        String8::format("C3Dev-%s-ReqQueue", mId.string()).string());
    ...
    mPreparerThread = new PreparerThread();
    ...
    return OK;
}

在 initializeCommonLocked 中新建了很多实例：

  • StatusTracker ：状态跟踪线程
  • Camera3BufferManager ：输出流的图形缓冲区管理，比如 Camera3OutputStream 的管理
  • TagMonitor ：相机元数据 metadata 的监视器，比如 3A 信息等
  • RequestThread ：请求线程，比如拍照、录像、预览的数据请求
  • PreparerThread ：监测数据已经准备好流的线程

小结

以上流程图都是基于 API 2 + HAL 3，当 Camera Open 流程结束后：

  • 客户端调用 API 时，得到了 CameraDevice 的实例，并将 ICameraDeviceUser 和 CameraDeviceImpl 绑定
  • 根据 HAL 1/3 生成了对应的 Device 客户端，当前生成的是 CameraDeviceClient 实例
  • Camera3Device 在初始化时，调用 CameraProviderManager.openSession ，它会通过 HIDL 通知 HAL 层打开摄像头物理设备；打开成功会 Camera3Device::HalInterface 和 ICameraDeviceSession 实例绑定
  • 新建 RequestThread 对象，后台运行线程，用于监听 API 发起的请求 CaptureRequest ：预览、拍照、录像等
  • 新建 FrameProcessorBase 对象，后台运行线程，用于监听 HAL 返回的请求结果 CaptureResult

打开设备时，实际上 Framework, HAL 已经创建好会话 ICameraDeviceSession ；而下面分析的 API 创建会话流程，实际是根据不同需求（预览、拍照、录像）来创建和配置输出流。

创建会话流程

API

在打开设备后，获取到了 CameraDevice 的实例，通过它来创建会话 Session ：

1
2
3
4
5

// CameraDevice.java
public abstract void createCaptureSession(@NonNull List<Surface> outputs,
        @NonNull CameraCaptureSession.StateCallback callback, 
        @Nullable Handler handler)
        throws CameraAccessException;
  • List<Surface> outputs
    表示有多少个输出流，通常为预览流和拍照/录像，两个输出流。
  • CameraCaptureSession.StateCallback callback
    创建会话状态回调。
  • Handler handler
    回调方法使用哪个线程响应，如果为 null 表示当前线程。

API 创建会话过程源码分析：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

// CameraDeviceImpl.java
@Override
public void createCaptureSession(List<Surface> outputs,
        CameraCaptureSession.StateCallback callback, Handler handler)
        throws CameraAccessException {
    // 将 Surface 转换为 OutputConfiguration
    List<OutputConfiguration> outConfigurations = 
        new ArrayList<>(outputs.size());
    for (Surface surface : outputs) {
        outConfigurations.add(new OutputConfiguration(surface));
    }
    createCaptureSessionInternal(null, outConfigurations, callback, 
            handler, ICameraDeviceUser.NORMAL_MODE);
}

private void createCaptureSessionInternal(InputConfiguration inputConfig,
        List<OutputConfiguration> outputConfigurations,
        CameraCaptureSession.StateCallback callback, Handler handler,
        int operatingMode) throws CameraAccessException {
    ...
    // 创建会话时，输入 Surface 为空
    Surface input = null;
    try {
        // configure streams and then block until IDLE
        // 向 Framework, HAL 发送信息，配置设备
        configureSuccess = configureStreamsChecked(inputConfig, 
            outputConfigurations, operatingMode);
        if (configureSuccess == true && inputConfig != null) {
            input = mRemoteDevice.getInputSurface();
        }
    } catch (CameraAccessException e) {
        ...
    }
    ...
    CameraCaptureSessionCore newSession = null;
    // 根据模式来实例化对应的 Session
    if (isConstrainedHighSpeed) {
        newSession = new CameraConstrainedHighSpeedCaptureSessionImpl(
            mNextSessionId++, callback, handler, this, 
            mDeviceHandler, configureSuccess, mCharacteristics);
    } else {
        // 假设实例化 CameraCaptureSessionImpl
        newSession = new CameraCaptureSessionImpl(mNextSessionId++, 
            input, callback, handler, this, 
            mDeviceHandler, configureSuccess);
    }

    mCurrentSession = newSession;
    ...
}
  • 将 List<Surface> 转换为 List<OutputConfiguration>
  • createCaptureSession 创建会话时，输入 Surface, InputConfiguration 都为空，即只有输出流
  • 根据 isConstrainedHighSpeed 来创建 CameraCaptureSession 实例；如果支持高速模式，则创建 CameraConstrainedHighSpeedCaptureSessionImpl 实例；否则创建普通 CameraCaptureSessionImpl 实例

示例

创建预览 mTextureSurface 和拍照 ImageReader.getSurface 两个输出流的会话，使用当前线程处理回调接口：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

mCameraDevice.createCaptureSession(
    Arrays.asList(mTextureSurface, mImageReader.getSurface()), 
    new CameraCaptureSession.StateCallback() {
        @Override
        public void onConfigured(@NonNull CameraCaptureSession session) {
            Log.d(TAG, "onConfigured: ");
            mCameraCaptureSession = session;
            preview();
        }

        @Override
        public void onConfigureFailed(
            @NonNull CameraCaptureSession session) {
            Log.e(TAG, "onConfigureFailed: ");
        }
    }, 
    null);
} catch (CameraAccessException e) {
    e.printStackTrace();
}

CameraCaptureSession.StateCallback 回调

CameraCaptureSession.StateCallback 回调用来处理 createCaptureSession 创建会话过程中出现的各种状态，比如创建成功、失败等，这些回调处理直接在 API Java 层实现的；回调接口中会获取到 CameraCaptureSession 实例。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

public static abstract class StateCallback {
    public abstract void onConfigured(
        @NonNull CameraCaptureSession session);
    public abstract void onConfigureFailed(
        @NonNull CameraCaptureSession session);
    public void onReady(@NonNull CameraCaptureSession session) {
        // default empty implementation
    }
    public void onActive(@NonNull CameraCaptureSession session) {
        // default empty implementation
    }
    public void onCaptureQueueEmpty(
        @NonNull CameraCaptureSession session) {
        // default empty implementation
    }
    public void onClosed(@NonNull CameraCaptureSession session) {
        // default empty implementation
    }
    public void onSurfacePrepared(@NonNull CameraCaptureSession session,
        @NonNull Surface surface) {
        // default empty implementation
    }
}

createCaptureSession 创建会话时，会创建 CameraCaptureSessionImpl 实例，而 CameraCaptureSession.StateCallback 接口的回调都是在 CameraCaptureSessionImpl 中实现的：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

// CameraCaptureSessionImpl.java
public class CameraCaptureSessionImpl extends CameraCaptureSession
        implements CameraCaptureSessionCore {
    CameraCaptureSessionImpl(int id, Surface input,
            CameraCaptureSession.StateCallback callback,
            Handler stateHandler,
            android.hardware.camera2.impl.CameraDeviceImpl deviceImpl,
            Handler deviceStateHandler, boolean configureSuccess) {
        ...
        mStateCallback = 
            createUserStateCallbackProxy(mStateHandler, callback);
        ...
        // 根据传入的参数，响应 CameraCaptureSession.StateCallback 回调
        if (configureSuccess) {
            mStateCallback.onConfigured(this);
            if (DEBUG) Log.v(...);
            mConfigureSuccess = true;
        } else {
            mStateCallback.onConfigureFailed(this);
            mClosed = true; 
            Log.e(...);
            mConfigureSuccess = false;
        }
    }
    ...
    
    // 用户指定并传入的回调实现及对应线程
    private final CameraCaptureSession.StateCallback mStateCallback;
    private final Handler mStateHandler;
    
    private StateCallback createUserStateCallbackProxy(Handler handler,
        StateCallback callback) {
        InvokeDispatcher<StateCallback> userCallbackSink 
            = new InvokeDispatcher<>(callback);
        HandlerDispatcher<StateCallback> handlerPassthrough =
                new HandlerDispatcher<>(userCallbackSink, handler);
        // 创建代理类
        return new CallbackProxies.SessionStateCallbackProxy(
            handlerPassthrough);
    }
}

用户指定的 StateCallback 传入后，在方法 createUserStateCallbackProxy 中，通过 CallbackProxies 重新生成一个代理 mStateCallback 对象，通过反射的方式，完成所有回调响应过程。

  • 如果 configureStreamsChecked 创建 Stream 成功，则响应回调 mStateCallback.onConfigured
  • 如果失败则响应 mStateCallback.onConfigureFailed ，其他场景会产生剩余的回调

动态代理类 CallbackProxies 源码注释（JDK 中的动态代理只支持接口 interface，对于抽象类只能自己实现了）：

1
2
3
4
5
6
7
8
9
10

/**
 * Proxy out invocations to the camera2 API callbacks into 
 * a {@link Dispatchable}.
 *
 * <p>Since abstract classes do not support Java's dynamic {@code Proxy},
 * we have to to use our own proxy mechanism.</p>
 */
public class CallbackProxies {

}

流程图

创建会话流程，查看原图

0109-android-camera-4-framework-CameraService-addService-CameraDevice_createCaptureSession.jpg

源码分析

API 中创建捕获会话 createCaptureSession 时，CameraDeviceImpl.configureStreamsChecked 源码中可以看到；CameraDeviceImpl 是通过 ICameraDeviceUser 来向 Framework, HAL 层发送配置信息的：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

// CameraDeviceImpl.java
public boolean configureStreamsChecked(InputConfiguration inputConfig,
        List<OutputConfiguration> outputs, int operatingMode)
                throws CameraAccessException {
    ...
    // createCaptureSession 时，imputConfig 为空
    checkInputConfiguration(inputConfig);
    boolean success = false;
    synchronized(mInterfaceLock) {
        ...
        mDeviceHandler.post(mCallOnBusy);
        stopRepeating();

        try {
            waitUntilIdle();
            mRemoteDevice.beginConfigure();
            ...
            // Delete all streams first (to free up HW resources)
            for (Integer streamId : deleteList) {
                mRemoteDevice.deleteStream(streamId);
                mConfiguredOutputs.delete(streamId);
            }

            // Add all new streams
            for (OutputConfiguration outConfig : outputs) {
                if (addSet.contains(outConfig)) {
                    int streamId = mRemoteDevice.createStream(outConfig);
                    mConfiguredOutputs.put(streamId, outConfig);
                }
            }
            operatingMode = (operatingMode | (customOpMode << 16));
            mRemoteDevice.endConfigure(operatingMode);

            success = true;
        } catch (...)
    }

    return success;
}

configureStreamsChecked 配置流有三个主要过程：beginConfigure, createStream, endConfigure ，都是通过 ICameraDeviceUser 向下发送信息。 native 代码中由 CameraDeviceClient.cpp 实现了 ICameraDeviceUser 中的所有功能，这里重点分析 createStream 函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

// CameraDeviceClient.cpp
binder::Status CameraDeviceClient::createStream(
        const hardware::camera2::params::OutputConfiguration 
        &outputConfiguration,
        /*out*/int32_t* newStreamId) {
    ...
    // 获取 IGraphicBufferProducer 的个数
    const std::vector<sp<IGraphicBufferProducer>>& bufferProducers =
            outputConfiguration.getGraphicBufferProducers();
    size_t numBufferProducers = bufferProducers.size();
    ...
    std::vector<sp<Surface>> surfaces;
    std::vector<sp<IBinder>> binders;
    ...
    OutputStreamInfo streamInfo;
    bool isStreamInfoValid = false;
    for (auto& bufferProducer : bufferProducers) {
        ...
        // 创建 Native Surface
        sp<Surface> surface;
        res = createSurfaceFromGbp(streamInfo, isStreamInfoValid, 
            surface, bufferProducer);
        if (!res.isOk())
            return res;
        ...
        binders.push_back(IInterface::asBinder(bufferProducer));
        surfaces.push_back(surface);
    }

    int streamId = camera3::CAMERA3_STREAM_ID_INVALID;
    // 调用 Camera3Device 创建流
    err = mDevice->createStream(surfaces, deferredConsumer, 
            streamInfo.width, streamInfo.height, 
            streamInfo.format, streamInfo.dataSpace,
            static_cast<camera3_stream_rotation_t>(
                outputConfiguration.getRotation()),
            &streamId, outputConfiguration.getSurfaceSetID(), isShared);
            
    if (err != OK) {
        ...
    } else {
        ...
        mStreamInfoMap[streamId] = streamInfo;
        ...
        // Set transform flags to ensure preview to be rotated correctly.
        res = setStreamTransformLocked(streamId);
        *newStreamId = streamId;
    }

    return res;
}

CameraDeviceClient.createStream 中，将 API 传入的 OutputConfiguration 数据，转换成 native Surface, OutputStreamInfo ；根据 OutputConfiguration 中 IGraphicBufferProducer 的个数创建对应的 native Surface ，并最终通过设备来创建流。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

binder::Status CameraDeviceClient::createSurfaceFromGbp(
        OutputStreamInfo& streamInfo, bool isStreamInfoValid,
        sp<Surface>& surface, const sp<IGraphicBufferProducer>& gbp) {

    ...
    // 根据 IGraphicBufferProducer 创建 Surface
    surface = new Surface(gbp, useAsync);
    ANativeWindow *anw = surface.get();

    int width, height, format;
    android_dataspace dataSpace;
    // 查询对应的长、宽、格式、数据空间
    if ((err = anw->query(anw, NATIVE_WINDOW_WIDTH, &width)) != OK) {
        ...
    }
    if ((err = anw->query(anw, NATIVE_WINDOW_HEIGHT, &height)) != OK) {
        ...
    }
    if ((err = anw->query(anw, NATIVE_WINDOW_FORMAT, &format)) != OK) {
        ...
    }
    if ((err = anw->query(anw, NATIVE_WINDOW_DEFAULT_DATASPACE,
            reinterpret_cast<int*>(&dataSpace))) != OK) {
        ...
    }

    ...
    // 赋值给输出流
    if (!isStreamInfoValid) {
        streamInfo.width = width;
        streamInfo.height = height;
        streamInfo.format = format;
        streamInfo.dataSpace = dataSpace;
        streamInfo.consumerUsage = consumerUsage;
        return binder::Status::ok();
    }
    ...
}

这里的 NATIVE_WINDOW_FORMAT 格式代表着不同流的类型，在 systemcorelibsystemincludesystemgraphics-base.h 文件中定义：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

typedef enum {
    HAL_PIXEL_FORMAT_RGBA_8888 = 1,
    HAL_PIXEL_FORMAT_RGBX_8888 = 2,
    HAL_PIXEL_FORMAT_RGB_888 = 3,
    HAL_PIXEL_FORMAT_RGB_565 = 4,
    HAL_PIXEL_FORMAT_BGRA_8888 = 5,
    HAL_PIXEL_FORMAT_RGBA_1010102 = 43, // 0x2B
    HAL_PIXEL_FORMAT_RGBA_FP16 = 22, // 0x16
    HAL_PIXEL_FORMAT_YV12 = 842094169, // 0x32315659
    HAL_PIXEL_FORMAT_Y8 = 538982489, // 0x20203859
    HAL_PIXEL_FORMAT_Y16 = 540422489, // 0x20363159
    HAL_PIXEL_FORMAT_RAW16 = 32, // 0x20
    HAL_PIXEL_FORMAT_RAW10 = 37, // 0x25
    HAL_PIXEL_FORMAT_RAW12 = 38, // 0x26
    HAL_PIXEL_FORMAT_RAW_OPAQUE = 36, // 0x24
    HAL_PIXEL_FORMAT_BLOB = 33, // 0x21
    HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED = 34, // 0x22
    HAL_PIXEL_FORMAT_YCBCR_420_888 = 35, // 0x23
    HAL_PIXEL_FORMAT_YCBCR_422_888 = 39, // 0x27
    HAL_PIXEL_FORMAT_YCBCR_444_888 = 40, // 0x28
    HAL_PIXEL_FORMAT_FLEX_RGB_888 = 41, // 0x29
    HAL_PIXEL_FORMAT_FLEX_RGBA_8888 = 42, // 0x2A
    HAL_PIXEL_FORMAT_YCBCR_422_SP = 16, // 0x10
    HAL_PIXEL_FORMAT_YCRCB_420_SP = 17, // 0x11
    HAL_PIXEL_FORMAT_YCBCR_422_I = 20, // 0x14
    HAL_PIXEL_FORMAT_JPEG = 256, // 0x100
} android_pixel_format_t;
  • HAL_PIXEL_FORMAT_BLOB 拍照流
    值为 33 ，通常对应 mImageReader.getSurface()
  • HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED 预览和录像流
    值为 34 ，通常对应预览 new Surface(mTextureView.getSurfaceTexture()) 和录像 mMediaRecorder.getSurface 。

CameraDeviceClient::createStream 最终会调用 Camera3Device::createStream ，它会根据 NATIVE_WINDOW_FORMAT 格式创建不同配置的流：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

// Camera3Device.cpp
status_t Camera3Device::createStream(
        const std::vector<sp<Surface>>& consumers,
        bool hasDeferredConsumer, uint32_t width, uint32_t height, 
        int format, android_dataspace dataSpace, 
        camera3_stream_rotation_t rotation, int *id,
        int streamSetId, bool isShared, uint64_t consumerUsage) {
    ...
    // 拍照流
    if (format == HAL_PIXEL_FORMAT_BLOB) {
        ssize_t blobBufferSize;
        if (dataSpace != HAL_DATASPACE_DEPTH) {
            blobBufferSize = getJpegBufferSize(width, height);
            if (blobBufferSize <= 0) {
                SET_ERR_L(...);
                return BAD_VALUE;
            }
        } else {
            blobBufferSize = getPointCloudBufferSize();
            if (blobBufferSize <= 0) {
                SET_ERR_L(...);
                return BAD_VALUE;
            }
        }
        newStream = new Camera3OutputStream(mNextStreamId, consumers[0],
                width, height, blobBufferSize, format, dataSpace, 
                rotation, mTimestampOffset, streamSetId);
    } else if (format == HAL_PIXEL_FORMAT_RAW_OPAQUE) {
        ssize_t rawOpaqueBufferSize = 
            getRawOpaqueBufferSize(width, height);
        if (rawOpaqueBufferSize <= 0) {
            SET_ERR_L(...);
            return BAD_VALUE;
        }
        newStream = new Camera3OutputStream(mNextStreamId, consumers[0],
                width, height, rawOpaqueBufferSize, format, dataSpace, 
                rotation, mTimestampOffset, streamSetId);
    }
    ...
    else {
        newStream = new Camera3OutputStream(mNextStreamId, consumers[0],
                width, height, format, dataSpace, rotation,
                mTimestampOffset, streamSetId);
    }
    newStream->setStatusTracker(mStatusTracker);
    newStream->setBufferManager(mBufferManager);

    res = mOutputStreams.add(mNextStreamId, newStream);
    if (res < 0) {
        SET_ERR_L(...);
        return res;
    }

    *id = mNextStreamId++;
    mNeedConfig = true;

    ...
    ALOGV("Camera %s: Created new stream", mId.string());
    return OK;
}

注意：每配置一个输出 Surface ，都会创建对应的输出流 Camera3OutputStream ，这是一个 for 循环过程。
在 API 调用过程中，CameraDeviceImpl.configureStreamsChecked 的第三步为 endConfigure，而 CameraDeviceClient::endConfigure 代码流程如下：

1
2
3
4
5
6
7

// CameraDeviceClient.cpp
binder::Status CameraDeviceClient::endConfigure(int operatingMode) {
    ...
    status_t err = mDevice->configureStreams(operatingMode);
    ...
    return res;
}

它的主要作用就是通过 Camera3Device 来配置流，configureStreamsLocked 配置流主要有三个过程：startConfiguration, configureStreams, endConfigure ：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76

// Camera3Device.cpp
status_t Camera3Device::configureStreams(int operatingMode) {
    ...
    return configureStreamsLocked(operatingMode);
}

status_t Camera3Device::configureStreamsLocked(int operatingMode) {
    ...
    camera3_stream_configuration config;
    config.operation_mode = mOperatingMode;
    config.num_streams = (mInputStream != NULL) + mOutputStreams.size();

    Vector<camera3_stream_t*> streams;
    streams.setCapacity(config.num_streams);

    if (mInputStream != NULL) {
        camera3_stream_t *inputStream;
        inputStream = mInputStream->startConfiguration();
        if (inputStream == NULL) {
            CLOGE("Can't start input stream configuration");
            cancelStreamsConfigurationLocked();
            return INVALID_OPERATION;
        }
        streams.add(inputStream);
    }

    for (size_t i = 0; i < mOutputStreams.size(); i++) {
        if (mOutputStreams[i].get() ==
            static_cast<Camera3StreamInterface*>(mInputStream.get())) {

            config.num_streams--;
            continue;
        }

        camera3_stream_t *outputStream;
      outputStream = mOutputStreams.editValueAt(i)->startConfiguration();
        if (outputStream == NULL) {
            CLOGE("Can't start output stream configuration");
            cancelStreamsConfigurationLocked();
            return INVALID_OPERATION;
        }
        streams.add(outputStream);
    }

    config.streams = streams.editArray();

    // Do the HAL configuration; will potentially touch stream
    // max_buffers, usage, priv fields.
    // 向 HAL 层发送配置信息
    res = mInterface->configureStreams(&config);
    ...
    if (mInputStream != NULL && mInputStream->isConfiguring()) {
        res = mInputStream->finishConfiguration();
        if (res != OK) {
            CLOGE("Can't finish configuring input stream %d: %s (%d)",
                    mInputStream->getId(), strerror(-res), res);
            cancelStreamsConfigurationLocked();
            return BAD_VALUE;
        }
    }

    for (size_t i = 0; i < mOutputStreams.size(); i++) {
        sp<Camera3OutputStreamInterface> outputStream =
            mOutputStreams.editValueAt(i);
        if (outputStream->isConfiguring() && 
            !outputStream->isConsumerConfigurationDeferred()) {
            res = outputStream->finishConfiguration();
            if (res != OK) {
                CLOGE(...);
                cancelStreamsConfigurationLocked();
                return BAD_VALUE;
            }
        }
    }
    ...
}

最终通过 Camera3Device::HalInterface::configureStreams 向 HAL 层发起配置信息：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

// Camera3Device.cpp
status_t Camera3Device::HalInterface::configureStreams(
    camera3_stream_configuration *config) {
    ...
    // Invoke configureStreams

    device::V3_3::HalStreamConfiguration finalConfiguration;
    common::V1_0::Status status;

    // See if we have v3.3 HAL
    sp<device::V3_3::ICameraDeviceSession> hidlSession_3_3;
    auto castResult = 
        device::V3_3::ICameraDeviceSession::castFrom(mHidlSession);
    if (castResult.isOk()) {
        hidlSession_3_3 = castResult;
    } else {
        ALOGE(...);
    }
    if (hidlSession_3_3 != nullptr) {
        // We do; use v3.3 for the call
        ALOGV("%s: v3.3 device found", __FUNCTION__);
        auto err = hidlSession_3_3->configureStreams_3_3(
            requestedConfiguration,
            [&status, &finalConfiguration]
            (common::V1_0::Status s, const 
                device::V3_3::HalStreamConfiguration& halConfiguration) {
                finalConfiguration = halConfiguration;
                status = s;
            });
        if (!err.isOk()) {
            ALOGE(...);
            return DEAD_OBJECT;
        }
    } else {
        // We don't; use v3.2 call and 
        // construct a v3.3 HalStreamConfiguration
        ALOGV("%s: v3.2 device found", __FUNCTION__);
        HalStreamConfiguration finalConfiguration_3_2;
        auto err = mHidlSession->configureStreams(requestedConfiguration,
                [&status, &finalConfiguration_3_2]
                (common::V1_0::Status s, const 
                    HalStreamConfiguration& halConfiguration) {
                    finalConfiguration_3_2 = halConfiguration;
                    status = s;
                });
        if (!err.isOk()) {
            ALOGE(...);
            return DEAD_OBJECT;
        }
        finalConfiguration.streams.resize(
            finalConfiguration_3_2.streams.size());
        for (size_t i = 0; i<finalConfiguration_3_2.streams.size(); i++){
            finalConfiguration.streams[i].v3_2 = 
                finalConfiguration_3_2.streams[i];
            finalConfiguration.streams[i].overrideDataSpace =
                    requestedConfiguration.streams[i].dataSpace;
        }
    }
    ...
}

这里需要注意的是 HAL 3.3, 3.2 的配置是有区别的；执行完配置后，Camera3Stream::finishConfiguration 结束配置：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

// Camera3Stream.cpp
status_t Camera3Stream::finishConfiguration() {
    ....
    res = configureQueueLocked();
    ...
}

// Camera3OutputStream.cpp
status_t Camera3OutputStream::configureConsumerQueueLocked() {
    ...
    // Configure consumer-side ANativeWindow interface. 
    // to notify buffer manager (if it is used) of the returned buffers.
    res = mConsumer->connect(NATIVE_WINDOW_API_CAMERA,
            /*listener*/mBufferReleasedListener,
            /*reportBufferRemoval*/true);
    if (res != OK) {
        ALOGE(...);
        return res;
    }

    mConsumerName = mConsumer->getConsumerName();

    res = native_window_set_usage(mConsumer.get(), mUsage);
    if (res != OK) {
        ALOGE(...);
        return res;
    }

    res = native_window_set_scaling_mode(mConsumer.get(),
            NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW);
    if (res != OK) {
        ALOGE(...);
        return res;
    }
    ...
}

这里 mConsumer 是 native Surface，也就是将生产者-消费者模型连接起来；同时 configureConsumerQueueLocked 有非常多的 native window 配置。

小结

创建会话 createCaptureSession 过程中，小结如下：

  • API 调用时，最终通过 CameraCaptureSession.StateCallback 获取到 CameraCaptureSessionImpl 实例
  • ICameraDeviceUser.createStream 由输入的 Surface 信息，根据不同的 format 创建对应输出流 Camera3OutputStream
  • ICameraDeviceUser.endConfigure 最终通过 CameraDeviceSession.configureStream_3_3 会向 HAL 层发送配置信息

相机预览过程中，如果 session 创建成功，会出现正常的预览界面；如果 session 创建失败，则预览会出现黑屏。

预览/拍照/录像流程

API

创建会话 createCaptureSession 成功后，通过拿到的 CameraCaptureSession 来预览、拍照、录像：

1
2
3
4
5
6
7
8
9
10

// CameraCaptureSession.java
// 预览和录像使用同一个 API
public abstract int setRepeatingRequest(@NonNull CaptureRequest request,
        @Nullable CaptureCallback listener, 
        @Nullable Handler handler)
        throws CameraAccessException;
// 拍照
public abstract int capture(@NonNull CaptureRequest request,
        @Nullable CaptureCallback listener, @Nullable Handler handler)
        throws CameraAccessException;
  • CaptureRequest request
    捕获请求，比如创建一个预览模板的请求 CameraDevice.TEMPLATE_PREVIEW ；拍照模板的请求 CameraDevice.TEMPLATE_STILL_CAPTURE ；录像模板的请求 CameraDevice.TEMPLATE_RECORD 。
  • CameraCaptureSession.CaptureCallback listener
    捕获状态的回调接口。
  • Handler handler
    回调接口使用哪个线程响应，如果是 null 表示当前线程。

CameraDevice 请求模板是一组常量：

1
2
3
4
5
6
7

// CameraDevice.java
public static final int TEMPLATE_PREVIEW = 1;
public static final int TEMPLATE_STILL_CAPTURE = 2;
public static final int TEMPLATE_RECORD  = 3;
public static final int TEMPLATE_VIDEO_SNAPSHOT = 4;
public static final int TEMPLATE_ZERO_SHUTTER_LAG = 5;
public static final int TEMPLATE_MANUAL = 6;

各模板对应的含义：

  • TEMPLATE_PREVIEW
    创建适合相机预览的窗口，高帧率优于高质量的后期处理。
  • TEMPLATE_STILL_CAPTURE
    创建适合拍照的请求，优先考虑帧速率的图像质量。
  • TEMPLATE_RECORD
    创建适合录像的请求，使用稳定的帧率。
  • TEMPLATE_VIDEO_SNAPSHOT
    创建录像时快照的请求，在不中断录像的前提下最大化图像质量。
  • TEMPLATE_ZERO_SHUTTER_LAG
    创建 ZSL 零延时拍照请求，也就是连拍功能，在不影响帧率的前提下最大化图像质量，并开启 3A 算法。
  • TEMPLATE_MANUAL
    手动控制模板，禁用所有的自动控制 3A 算法。

示例

给 mTextureSurface 创建预览请求 TEMPLATE_PREVIEW ，使用后台线程处理回调接口：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

CaptureRequest.Builder previewRequestBuilder = 
    mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
previewRequestBuilder.addTarget(mTextureSurface);
CameraCaptureSession.CaptureCallback captureCallback 
    = new CameraCaptureSession.CaptureCallback() {
        @Override
        public void onCaptureCompleted(
            @NonNull CameraCaptureSession session, 
            @NonNull CaptureRequest request, 
            @NonNull TotalCaptureResult result) {
            //Log.d(TAG, "preview, onCaptureCompleted: ");
        }
};
mCameraCaptureSession.setRepeatingRequest(
    previewRequestBuilder.build(), captureCallback, mBackHandler);

给 ImageReader 创建拍照请求 TEMPLATE_STILL_CAPTURE ，使用后台线程处理回调：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

CaptureRequest.Builder captureRequestBuild = mCameraDevice
    .createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);
captureRequestBuild.addTarget(mImageReader.getSurface());
captureRequestBuild.set(CaptureRequest.CONTROL_AF_MODE,
    CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
captureRequestBuild.set(CaptureRequest.JPEG_ORIENTATION, 
    mImageOrientation);
CameraCaptureSession.CaptureCallback captureCallback = 
    new CameraCaptureSession.CaptureCallback() {
        @Override
        public void onCaptureCompleted(
            @NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request,
            @NonNull TotalCaptureResult result) {
            // Log.d(TAG, "takePicture, onCaptureCompleted: ");
        }
};
mCameraCaptureSession.stopRepeating();
mCameraCaptureSession.abortCaptures();
mCameraCaptureSession.capture(captureRequestBuild.build(),
    captureCallback, mBackHandler);

给 MediaRecorder 创建录像请求 TEMPLATE_RECORD ，不处理回调：

1
2
3
4
5
6
7

CaptureRequest.Builder recordCaptureBuild = mCameraDevice
    .createCaptureRequest(CameraDevice.TEMPLATE_RECORD);
recordCaptureBuild.addTarget(mMediaRecorder.getSurface());
recordCaptureBuild.set(CaptureRequest.CONTROL_MODE, 
    CameraMetadata.CONTROL_MODE_AUTO);
mCameraCaptureSession.setRepeatingRequest(
    recordCaptureBuild.build(), null, null);

CameraCaptureSession.CaptureCallback 回调

CameraCaptureSession 在请求预览、拍照、录像等功能时，出现的各种状态通过 CameraCaptureSession.CaptureCallback 回调来处理，回调是由 HAL 层发起向上传递的；回调接口中通常包含当前会话信息 CameraCaptureSession ，捕获请求 CaptureRequest ，捕获的结果 CaptureResult 等。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

// CameraCaptureSession.java
public static abstract class CaptureCallback {

    public static final int NO_FRAMES_CAPTURED = -1;

    // 设备开始拍照时
    public void onCaptureStarted(@NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request, long timestamp, 
            long frameNumber) {
        // default empty implementation
    }
    // 返回部分数据
    public void onCapturePartial(CameraCaptureSession session,
            CaptureRequest request, CaptureResult result) {
        // default empty implementation
    }
    // 返回部分数据时，表示正在捕获数据过程中
   public void onCaptureProgressed(@NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request, 
            @NonNull CaptureResult partialResult) {
        // default empty implementation
    }
    // 数据捕获已经完成，回调最终总的结果集
    public void onCaptureCompleted(@NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request, 
            @NonNull TotalCaptureResult result) {
        // default empty implementation
    }
    // 捕获失败
    public void onCaptureFailed(@NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request, 
            @NonNull CaptureFailure failure) {
        // default empty implementation
    }
    // 所有捕获结果已经发送完毕
    public void onCaptureSequenceCompleted(
            @NonNull CameraCaptureSession session,
            int sequenceId, long frameNumber) {
        // default empty implementation
    }
    // 请求捕获被中止
    public void onCaptureSequenceAborted(
            @NonNull CameraCaptureSession session,
            int sequenceId) {
        // default empty implementation
    }
    // 捕获的 buffer 不能成功显示
   public void onCaptureBufferLost(@NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request, 
            @NonNull Surface target, long frameNumber) {
        // default empty implementation
    }
}

API 在发起请求 CameraCaptureSession.setRepeatingRequest/capture 时，用户会创建 CaptureCallback 的实例，这些接口都在 CameraDeviceImpl 中实现回调。

  • onCaptureStarted
    该回调接口从 HAL 回调路径为：CameraDeviceClient::notifyShutter -> CameraDeviceImpl.onCaptureStarted -> onCaptureStarted 。
  • onCapturePartial
    该回调接口搜索整个 framework ，发现没有任何地方会回调它。
  • onCaptureProgressed, onCaptureCompleted, onCaptureSequenceCompleted
    三个接口都是在 CameraDeviceImpl.onResultReceived 中回调的。
  • onCaptureSequenceAborted
    CameraDeviceClient.cpp 中的 submitCaptureRequest, stopRepeating, flush 这三个函数会回调该接口。
  • onCaptureFailed, onCaptureBufferLost
    从 HAL 回调路径为 CameraDeviceClient::notifyError -> CameraDeviceImpl.onDeviceError ，而这两个接口在 CameraDeviceImpl 中的回调路径为 onDeviceError -> onCaptureErrorLocked -> onCaptureFailed/onCaptureBufferLost 。

流程图

创建捕获请求流程，查看原图

0109-android-camera-4-framework-CameraDevice_createCaptureRequest.jpg

预览/拍照/录像流程基本一致，这里仅给出预览的流程图：预览流程，查看原图

 

源码分析

在分析预览、拍照、录像流程前，先回顾下打开设备 openCamera 时，做的一些初始化：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

// CameraDeviceClient.cpp
status_t CameraDeviceClient::initialize(sp<CameraProviderManager> manager) {
    return initializeImpl(manager);
}

template<typename TProviderPtr>
status_t CameraDeviceClient::initializeImpl(TProviderPtr providerPtr) {
    ...
    res = Camera2ClientBase::initialize(providerPtr);
    if (res != OK) {
        return res;
    }

    String8 threadName;
    mFrameProcessor = new FrameProcessorBase(mDevice);
    threadName = String8::format("CDU-%s-FrameProc", 
        mCameraIdStr.string());
    mFrameProcessor->run(threadName.string());

    mFrameProcessor->registerListener(FRAME_PROCESSOR_LISTENER_MIN_ID,
                                      FRAME_PROCESSOR_LISTENER_MAX_ID,
                                      /*listener*/this,
                                      /*sendPartials*/true);

    return OK;
}

在 CameraDeviceClient::initializeImpl 中，调用了 Camera2ClientBase::initialize 的初始化，以及实例化一个 FrameProcessorBase 对象；

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// Camera2ClientBase.cpp
template <typename TClientBase>
status_t Camera2ClientBase<TClientBase>::initialize(
        sp<CameraProviderManager> manager) {
    return initializeImpl(manager);
}

template <typename TClientBase>
template <typename TProviderPtr>
status_t Camera2ClientBase<TClientBase>::initializeImpl(
        TProviderPtr providerPtr) {
    ...
    res = mDevice->initialize(providerPtr);
    ...
}

这里 Camera2ClientBase::initializeImpl 中主要是调用了 Camera3Device::initialize 函数，下面只关心和捕获请求有关的代码：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// Camera3Device.cpp
status_t Camera3Device::initialize(sp<CameraProviderManager> manager) {
    ...
    return initializeCommonLocked();
}

status_t Camera3Device::initializeCommonLocked() {
    ...
    /** Start up request queue thread */
    mRequestThread = new RequestThread(this, mStatusTracker, mInterface);
    res = mRequestThread->run(
        String8::format("C3Dev-%s-ReqQueue", mId.string()).string());
    ...
    return OK;
}

在 Camera3Device::initializeCommonLocked 中实例化了 RequestThread 对象。至此，捕获流程中的发起请求的对象 RequestThread 和响应回调的对象 FrameProcessorBase 都实例化完毕，并开始运行。他们两个都继承的是线程，参看 system 目录下的 Thread.h/Threads.cpp 源码，可以看到 threadLoop 是在一个 while 中被循环调用的。当 threadLoop 返回 true 时就会不停的循环；返回 false 时会退出循环：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

// Threads.cpp
int Thread::_threadLoop(void* user)
{
    Thread* const self = static_cast<Thread*>(user);
    ...
    bool first = true;
    do {
        bool result;
        if (first) {
            first = false;
            self->mStatus = self->readyToRun();
            result = (self->mStatus == NO_ERROR);

            if (result && !self->exitPending()) {
                result = self->threadLoop();
            }
        } else {
            result = self->threadLoop();
        }

        ...
        if (result == false || self->mExitPending) {
            ...
            break;
        }
        ...
    } while(strong != 0);

    return 0;
}

先来看发送捕获请求的线程 RequestThread ：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

// Camera3Device.h
class RequestThread : public Thread {
  ...
  protected:
    ...
    virtual bool threadLoop();
  private:
    ...
    // Used to prepare a batch of requests.
    struct NextRequest {
        sp<CaptureRequest>              captureRequest;
        camera3_capture_request_t       halRequest;
        Vector<camera3_stream_buffer_t> outputBuffers;
        bool                            submitted;
    };
    Vector<NextRequest> mNextRequests;
    ...
    Condition          mRequestSignal;
    RequestList        mRequestQueue;
    RequestList        mRepeatingRequests;
    ...
}

这里只关注 RequestThread 类中几个关键函数和变量，NextRequest 结构体包含了请求信息，逐个向 HAL 发送这些信息；类中定义了多个条件变量，重点关注 mRequestSignal 条件变量， threadLoop 运行时，会通过 mRequestSignal.waitRelative 阻塞等待 50 ms；直到等到捕获请求后 mRequestSignal.signal 发出通知，threadLoop 继续运行。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

// Camera3Device.cpp
bool Camera3Device::RequestThread::threadLoop() {
    ...
    status_t res;

    // Handle paused state.
    if (waitIfPaused()) {
        return true;
    }

    // Wait for the next batch of requests.
    waitForNextRequestBatch();
    if (mNextRequests.size() == 0) {
        return true;
    }
    ...
    // Prepare a batch of HAL requests and output buffers.
    res = prepareHalRequests();
    ...
}

void Camera3Device::RequestThread::waitForNextRequestBatch() {
    ...
    NextRequest nextRequest;
    nextRequest.captureRequest = waitForNextRequestLocked();
    if (nextRequest.captureRequest == nullptr) {
        return;
    }
    ...
}

sp<Camera3Device::CaptureRequest>
        Camera3Device::RequestThread::waitForNextRequestLocked() {
    status_t res;
    sp<CaptureRequest> nextRequest;

    while (mRequestQueue.empty()) {
        if (!mRepeatingRequests.empty()) {
            const RequestList &requests = mRepeatingRequests;
            RequestList::const_iterator firstRequest =
                    requests.begin();
            nextRequest = *firstRequest;
            mRequestQueue.insert(mRequestQueue.end(),
                    ++firstRequest,
                    requests.end());

            mRepeatingLastFrameNumber = mFrameNumber+requests.size()-1;

            break;
        }
        // 条件变量 mRequestSignal 阻塞等待 kRequestTimeout 
        res = mRequestSignal.waitRelative(mRequestLock, kRequestTimeout);
        ...
    }
}

RequestThread 在没有捕获请求时，会循环调用 threadLoop ，并阻塞等待 mRequestSignal 的通知。再看响应回调的线程 FrameProcessorBase ：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

// FrameProcessorBase.h
class FrameProcessorBase: public Thread {
  public:
    ...
    struct FilteredListener: virtual public RefBase {
        virtual void onResultAvailable(const CaptureResult &result) = 0;
    };
  protected:
    static const nsecs_t kWaitDuration = 10000000; // 10 ms
    wp<CameraDeviceBase> mDevice;
    virtual bool threadLoop();
  ...
}

// FrameProcessorBase.cpp
bool FrameProcessorBase::threadLoop() {
    status_t res;

    sp<CameraDeviceBase> device;
    {
        device = mDevice.promote();
        if (device == 0) return false;
    }

    res = device->waitForNextFrame(kWaitDuration);
    if (res == OK) {
        processNewFrames(device);
    } else if (res != TIMED_OUT) {
        ALOGE(...);
    }

    return true;
}

FrameProcessorBase::threadLoop 代码非常简单，device->waitForNextFrame 阻塞等待 10ms ，这里 CameraDeviceBase 实际类型为 Camera3Device ：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

// Camera3Device.h
class Camera3Device :
    public CameraDeviceBase,
    virtual public hardware::camera::device::V3_2::ICameraDeviceCallback,
    private camera3_callback_ops {
    ...
    private:
        List<CaptureResult>   mResultQueue;
        Condition              mResultSignal;
    ...
}

// Camera3Device.cpp
status_t Camera3Device::waitForNextFrame(nsecs_t timeout) {
    ...
    while (mResultQueue.empty()) {
        res = mResultSignal.waitRelative(mOutputLock, timeout);
        if (res == TIMED_OUT) {
            return res;
        } else if (res != OK) {
            ALOGW(...);
            return res;
        }
    }
    return OK;
}

Camera3Device::waitForNextFrame 代码也很简单，调用条件变量 mResultSignal.waitRelative 实现阻塞等待 10 ms。
至此初始化过程中，捕获请求线程 RequestThread 循环执行 threadLoop ，并会阻塞等待 mRequestSignal 的通知；回调响应线程 FrameProcessorBase 循环执行 threadLoop ，并会阻塞等待 mResultSignal 的通知。

当用户调用 API 创建捕获请求时，mRequestSignal 会发出通知；因为预览、拍照、录像流程基本一样，一起分析：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

// CameraCaptureSessionImpl.java
// 预览和录像
@Override
public int setRepeatingRequest(CaptureRequest request, 
        CaptureCallback callback,
        Handler handler) 
        throws CameraAccessException {
    ...
    synchronized (mDeviceImpl.mInterfaceLock) {
        ...
        return addPendingSequence(mDeviceImpl.setRepeatingRequest(
            request,
            createCaptureCallbackProxy(handler, callback), 
            mDeviceHandler));
    }
}

// 拍照
@Override
public int capture(CaptureRequest request, CaptureCallback callback,
        Handler handler) throws CameraAccessException {
    ...
    synchronized (mDeviceImpl.mInterfaceLock) {
        ...
        return addPendingSequence(mDeviceImpl.capture(request,
                createCaptureCallbackProxy(handler, callback), 
                mDeviceHandler));
    }
}

createCaptureCallbackProxy 创建了一个回调动态代理，通过 CameraDeviceImpl.setRepeatingRequest/capture 下发预览或者拍照的捕获请求：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

// CameraDeviceImpl.java
// 预览和录像
public int setRepeatingRequest(CaptureRequest request, 
        CaptureCallback callback,
        Handler handler) throws CameraAccessException {
    List<CaptureRequest> requestList = new ArrayList<CaptureRequest>();
    requestList.add(request);
    return submitCaptureRequest(requestList, callback, 
        handler, /*streaming*/true);
}
// 拍照
public int capture(CaptureRequest request, CaptureCallback callback, 
        Handler handler) throws CameraAccessException {
    List<CaptureRequest> requestList = new ArrayList<CaptureRequest>();
    requestList.add(request);
    return submitCaptureRequest(requestList, callback, 
        handler, /*streaming*/false);
}

private int submitCaptureRequest(List<CaptureRequest> requestList, 
        CaptureCallback callback,
        Handler handler, boolean repeating) 
        throws CameraAccessException {
    ...
    synchronized(mInterfaceLock) {
        checkIfCameraClosedOrInError();
        if (repeating) {
            stopRepeating();
        }
        SubmitInfo requestInfo;
        CaptureRequest[] requestArray = requestList.toArray(
            new CaptureRequest[requestList.size()]);
        requestInfo = mRemoteDevice.submitRequestList(requestArray,
            repeating);
        ...
        if (callback != null) {
            mCaptureCallbackMap.put(requestInfo.getRequestId(),
                    new CaptureCallbackHolder(
                        callback, requestList, handler, 
                        repeating, mNextSessionId - 1));
        } else {
            ...
        }

        if (repeating) {
            if (mRepeatingRequestId != REQUEST_ID_NONE) {
                checkEarlyTriggerSequenceComplete(mRepeatingRequestId,
                        requestInfo.getLastFrameNumber());
            }
            mRepeatingRequestId = requestInfo.getRequestId();
        } else {
            mRequestLastFrameNumbersList.add(
                new RequestLastFrameNumbersHolder(requestList,
                    requestInfo));
        }
        ...
    }
}

从代码流程来看，预览和录像使用同一个接口；预览和拍照的主要区别是 repeating 的值；当为 true 时，表示预览/录像；当为 false 时，表示为拍照。通过 ICameraDeviceUser.submitRequestList 向下发送请求：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

// CameraDeviceClient.cpp
binder::Status CameraDeviceClient::submitRequestList(
        const std::vector<hardware::camera2::CaptureRequest>& requests,
        bool streaming,
        /*out*/
        hardware::camera2::utils::SubmitInfo *submitInfo) {
    ...
    if (streaming) {
        err = mDevice->setStreamingRequestList(metadataRequestList, 
                surfaceMapList, &(submitInfo->mLastFrameNumber));
        ...
    } else {
        err = mDevice->captureList(metadataRequestList, surfaceMapList,
                &(submitInfo->mLastFrameNumber));
        ...
    }
    return res;
}

如果是预览/录像，则调用 Camera3Device->setStreamingRequestList ；如果是拍照，则调用 Camera3Device->captureList ：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

//Camera3Device.cpp
// 预览和录像
status_t Camera3Device::setStreamingRequestList(
        const List<const CameraMetadata> &requests,
        const std::list<const SurfaceMap> &surfaceMaps,
        int64_t *lastFrameNumber) {
    ...
    return submitRequestsHelper(requests, surfaceMaps, 
        /*repeating*/true, lastFrameNumber);
}
// 拍照
status_t Camera3Device::captureList(
        const List<const CameraMetadata> &requests,
        const std::list<const SurfaceMap> &surfaceMaps,
        int64_t *lastFrameNumber) {
    ...
    return submitRequestsHelper(requests, surfaceMaps, 
        /*repeating*/false, lastFrameNumber);
}

status_t Camera3Device::submitRequestsHelper(
        const List<const CameraMetadata> &requests,
        const std::list<const SurfaceMap> &surfaceMaps,
        bool repeating, /*out*/ int64_t *lastFrameNumber) {
    ...
    RequestList requestList;
    res = convertMetadataListToRequestListLocked(requests, surfaceMaps,
            repeating, /*out*/&requestList);
    ...

    if (repeating) {
        res = mRequestThread->setRepeatingRequests(requestList,
            lastFrameNumber);
    } else {
        res = mRequestThread->queueRequestList(requestList,
            lastFrameNumber);
    }
    ...
    return res;
}

同样，预览/录像和拍照请求在 Camera3Device 中的区别也主要是 repeating 的值，都会调用 Camera3Device::submitRequestsHelper ，并通过 RequestThread 发起捕获请求；当预览/录像时，调用 setRepeatingRequests ；当拍照时，调用 queueRequestList ：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

// Camera3Device.cpp
// 预览和录像
status_t Camera3Device::RequestThread::setRepeatingRequests(
        const RequestList &requests,
        /*out*/
        int64_t *lastFrameNumber) {
    ...
    if (lastFrameNumber != NULL) {
        *lastFrameNumber = mRepeatingLastFrameNumber;
    }
    mRepeatingRequests.clear();
    mRepeatingRequests.insert(mRepeatingRequests.begin(),
            requests.begin(), requests.end());

    unpauseForNewRequests();

    mRepeatingLastFrameNumber = 
        ...ICameraDeviceUser::NO_IN_FLIGHT_REPEATING_FRAMES;
    return OK;
}
// 拍照
status_t Camera3Device::RequestThread::queueRequestList(
        List<sp<CaptureRequest> > &requests,
        /*out*/
        int64_t *lastFrameNumber) {
    ...
    for (List<sp<CaptureRequest> >::iterator it = requests.begin();
        it != requests.end(); ++it) {
        mRequestQueue.push_back(*it);
    }
    if (lastFrameNumber != NULL) {
        *lastFrameNumber = mFrameNumber + mRequestQueue.size() - 1;
        ALOGV(...);
    }

    unpauseForNewRequests();
    return OK;
}

void Camera3Device::RequestThread::unpauseForNewRequests() {
    ...
    mRequestSignal.signal();
    ...
}

预览/录像时会将捕获请求存入 mRepeatingRequests 列表中；拍照时会将捕获请求存入 mRequestQueue 列表中；它们最终都会调用 unpauseForNewRequests ，而该函数的核心功能就是通过 mRequestSignal.signal 发出消息，通知在开启设备初始化过程中 waitForNextRequestLocked 的阻塞等待。我们重新进入 RequestThread::threadLoop 中，继续向下分析：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

// Camera3Device.cpp
bool Camera3Device::RequestThread::threadLoop() {
    ...
    status_t res;
    ...
    // Wait for the next batch of requests.
    // 阻塞等待
    waitForNextRequestBatch();
    if (mNextRequests.size() == 0) {
        return true;
    }
    ...
    bool submitRequestSuccess = false;
    nsecs_t tRequestStart = systemTime(SYSTEM_TIME_MONOTONIC);
    if (mInterface->supportBatchRequest()) {
        submitRequestSuccess = sendRequestsBatch();
    } else {
        submitRequestSuccess = sendRequestsOneByOne();
    }
    ...
    return submitRequestSuccess;
}

bool Camera3Device::RequestThread::sendRequestsBatch() {
    status_t res;
    size_t batchSize = mNextRequests.size();
    std::vector<camera3_capture_request_t*> requests(batchSize);
    uint32_t numRequestProcessed = 0;
    for (size_t i = 0; i < batchSize; i++) {
        requests[i] = &mNextRequests.editItemAt(i).halRequest;
    }

    ...
    res = mInterface->processBatchCaptureRequests(requests, 
        &numRequestProcessed);
    ...
}

status_t Camera3Device::HalInterface::processBatchCaptureRequests(
        std::vector<camera3_capture_request_t*>& requests,
        /*out*/uint32_t* numRequestProcessed) {
    ...
    hardware::hidl_vec<device::V3_2::CaptureRequest> captureRequests;
    size_t batchSize = requests.size();
    captureRequests.resize(batchSize);
    ...
    auto err = mHidlSession->processCaptureRequest(captureRequests, 
            cachesToRemove,
            [&status, &numRequestProcessed] (auto s, uint32_t n) {
                status = s;
                *numRequestProcessed = n;
            });
    ...
    return CameraProviderManager::mapToStatusT(status);
}

当 waitForNextRequestBatch 拿到请求通知后，会将捕获请求存入 mNextRequests 中，当前平台支持批量请求处理，sendRequestsBatch -> processBatchCaptureRequests 流程，向 HAL 层发送捕获请求 mHidlSession->processCaptureRequest ，至此捕获请求从 API 发送到 HAL 整个流程全部分析完毕。

当 HAL 拿到捕获的结果后，会从 ICameraDeviceSession.processCaptureResult 回调到 Framework 层：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

// Camera3Device.cpp
hardware::Return<void> Camera3Device::processCaptureResult(
    const hardware::hidl_vec<
            hardware::camera::device::V3_2::CaptureResult>& results) {
    ...
    for (const auto& result : results) {
        processOneCaptureResultLocked(result);
    }
    mProcessCaptureResultLock.unlock();
    return hardware::Void();
}

void Camera3Device::processOneCaptureResultLocked(
        const hardware::camera::device::V3_2::CaptureResult& result) {
    camera3_capture_result r;
    ...
    processCaptureResult(&r);
}

void Camera3Device::processCaptureResult(
        const camera3_capture_result *result) {
    ...
    if (result->partial_result != 0)
        request.resultExtras.partialResultCount = result->partial_result;

    // Check if this result carries only partial metadata
    if (mUsePartialResult && result->result != NULL) {
        ...
        if (isPartialResult && request.hasCallback) {
            // Send partial capture result
            sendPartialCaptureResult(result->result, request.resultExtras,
                    frameNumber);
        }
    }
    ...
    if (result->result != NULL && !isPartialResult) {
        if (shutterTimestamp == 0) {
            request.pendingMetadata = result->result;
            request.collectedPartialResult = collectedPartialResult;
        } else if (request.hasCallback) {
            CameraMetadata metadata;
            metadata = result->result;
            sendCaptureResult(metadata, request.resultExtras,
                collectedPartialResult, frameNumber,
                hasInputBufferInRequest);
        }
    }
    ...
}

void Camera3Device::sendPartialCaptureResult(
        const camera_metadata_t * partialResult,
        const CaptureResultExtras &resultExtras, uint32_t frameNumber) {
    ...
    CaptureResult captureResult;
    captureResult.mResultExtras = resultExtras;
    captureResult.mMetadata = partialResult;
    insertResultLocked(&captureResult, frameNumber);
}


void Camera3Device::sendCaptureResult(CameraMetadata &pendingMetadata,
        CaptureResultExtras &resultExtras,
        CameraMetadata &collectedPartialResult,
        uint32_t frameNumber,
        bool reprocess) {
    ...
    CaptureResult captureResult;
    captureResult.mResultExtras = resultExtras;
    captureResult.mMetadata = pendingMetadata;
    ...
    insertResultLocked(&captureResult, frameNumber);
}

void Camera3Device::insertResultLocked(CaptureResult *result,
        uint32_t frameNumber) {
    ...
    camera_metadata_t *meta = const_cast<camera_metadata_t *>(
            result->mMetadata.getAndLock());
    set_camera_metadata_vendor_id(meta, mVendorTagId);
    result->mMetadata.unlock(meta);
    ...
    // Valid result, insert into queue
    List<CaptureResult>::iterator queuedResult =
            mResultQueue.insert(mResultQueue.end(), 
                CaptureResult(*result));
    ...
    mResultSignal.signal();
}

从代码流程来看，从 HAL 传过来的捕获结果，不管是发回部分结果 sendPartialCaptureResult 还是最终结果 sendCaptureResult ，最终都会调用 insertResultLocked ，它的主要功能就是将捕获结果放入 mResultQueue 队列，并由 mResultSignal.signal 发出消息，通知在开启设备初始化过程中 waitForNextFrame 的阻塞等待。一旦 FrameProcessorBase::threadLoop 获取到捕获结果后，逐个处理：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

// FrameProcessorBase.cpp
void FrameProcessorBase::processNewFrames(
        const sp<CameraDeviceBase> &device) {
    status_t res;
    ...
    while ( (res = device->getNextResult(&result)) == OK) {
        camera_metadata_entry_t entry;
        entry = result.mMetadata.find(ANDROID_REQUEST_FRAME_COUNT);
        ...
        if (!processSingleFrame(result, device)) {
            break;
        }
        ...
    }
    ...
    return;
}

bool FrameProcessorBase::processSingleFrame(CaptureResult &result,
         const sp<CameraDeviceBase> &device) {
    ...
    return processListeners(result, device) == OK;
}

status_t FrameProcessorBase::processListeners(
        const CaptureResult &result,
        const sp<CameraDeviceBase> &device) {
    ...
    List<sp<FilteredListener> >::iterator item = listeners.begin();
    for (; item != listeners.end(); item++) {
        (*item)->onResultAvailable(result);
    }
    return OK;
}

代理流程可以看出，逐个取出 CaptureResult 并处理，最终调用 CameraDeviceClient::onResultAvailable 向 API 发送捕获结果：

1
2
3
4
5
6
7
8
9
10

// CameraDeviceClient.cpp
void CameraDeviceClient::onResultAvailable(const CaptureResult& result) {
    ...
    sp<hardware::camera2::ICameraDeviceCallbacks> remoteCb 
        = mRemoteCallback;
    if (remoteCb != NULL) {
        remoteCb->onResultReceived(result.mMetadata, 
            result.mResultExtras);
    }
}

而 API 中的回调是在 CameraDeviceImpl.java 中实现的：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88

// CameraDeviceImpl.java
@Override
public void onResultReceived(CameraMetadataNative result,
        CaptureResultExtras resultExtras) throws RemoteException {
    int requestId = resultExtras.getRequestId();
    long frameNumber = resultExtras.getFrameNumber();
    ...
    synchronized(mInterfaceLock) {
      ...
      final CaptureCallbackHolder holder =
              CameraDeviceImpl.this.mCaptureCallbackMap.get(requestId);
      final CaptureRequest request = 
          holder.getRequest(resultExtras.getSubsequenceId());
      boolean isPartialResult = (resultExtras.getPartialResultCount() 
              < mTotalPartialCount);
      ...
      if (isPartialResult) {
        final CaptureResult resultAsCapture =
                new CaptureResult(result, request, resultExtras);
        // Partial result
        resultDispatch = new Runnable() {
        @Override
        public void run() {
          if (!CameraDeviceImpl.this.isClosed()) {
            if (holder.hasBatchedOutputs()) {
              for (int i = 0; i < holder.getRequestCount(); i++) {
                CameraMetadataNative resultLocal =
                        new CameraMetadataNative(resultCopy);
                CaptureResult resultInBatch = new CaptureResult(
                    resultLocal, holder.getRequest(i), resultExtras);

                holder.getCallback().onCaptureProgressed(
                    CameraDeviceImpl.this,
                    holder.getRequest(i),
                    resultInBatch);
              }
            } else {
              holder.getCallback().onCaptureProgressed(
                  CameraDeviceImpl.this,
                  request,
                  resultAsCapture);
            }
          }
        }
        };
        finalResult = resultAsCapture;
      } else {
        List<CaptureResult> partialResults =
                mFrameNumberTracker.popPartialResults(frameNumber);
        ...
        final TotalCaptureResult resultAsCapture = 
                new TotalCaptureResult(result, request, resultExtras,
                partialResults, holder.getSessionId());
        // Final capture result
        resultDispatch = new Runnable() {
        @Override
        public void run() {
          if (!CameraDeviceImpl.this.isClosed()){
            if (holder.hasBatchedOutputs()) {
              for (int i = 0; i < holder.getRequestCount(); i++) {
                ...
                TotalCaptureResult resultInBatch = new TotalCaptureResult(
                    resultLocal, holder.getRequest(i), resultExtras,
                    partialResults, holder.getSessionId());

                holder.getCallback().onCaptureCompleted(
                    CameraDeviceImpl.this,
                    holder.getRequest(i),
                    resultInBatch);
              }
            } else {
              holder.getCallback().onCaptureCompleted(
                  CameraDeviceImpl.this,
                  request,
                  resultAsCapture);
            }
          }
        }
        };
        finalResult = resultAsCapture;
      }
      ...
      // Fire onCaptureSequenceCompleted
      if (!isPartialResult) {
          checkAndFireSequenceComplete();
      }
    }
}

在 CameraDeviceImpl 中处理 CameraCaptureSession.CaptureCallback 各回调结果：如果返回的是部分结果，则回调 onCaptureProgressed ；如果返回最终结果，则回调 onCaptureCompleted 。整个预览、拍照、录像流程及回调分析完毕。

小结

  • Camera3Device::RequestThread
    这个类主要是处理预览、拍照、录像的请求 CaptureRequest 。
  • FrameProcessorBase.cpp
    这个类主要是处理请求后的回调函数，回调中会包含捕获的结果 CaptureResult 。
  • Condition
    不管是请求还是结果回调，因为是多线程处理，都涉及到条件变量的阻塞等待和通知机制。
  • CameraDeviceSession.CaptureCallback
    该回调接口都是在 CameraDeviceImpl 中实现的。

三者异同

预览、拍照、录像三者的流程基本一致，它们之间有如下异同：

  • 预览
    捕获请求模板为 CameraDevice.TEMPLATE_PREVIEW ；API 接口为 CameraCaptureSession.setRepeatingRequest ；repeating 值为 true 。
  • 拍照
    捕获请求模板为 CameraDevice.TEMPLATE_STILL_CAPTURE ；API 接口为 CameraCaptureSession.Capture ；repeating 值为 false 。
  • 录像
    捕获请求模板为 CameraDevice.TEMPLATE_RECORD ；API 接口为 CameraCaptureSession.setRepeatingRequest ；repeating 值为 true 。

也就是说，预览和录像仅仅是捕获请求模板不一样；而预览和拍照不管是模板，接口，repeating 值都不一样；但是它们三者最终在 Framework 中代码流程基本一致。

CameraServiceProxy 注册服务

AIDL

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

interface ICameraServiceProxy
{
    // CameraService 向代理服务发送消息，通知用户更新
    oneway void pingForUserUpdate();

    const int CAMERA_STATE_OPEN = 0;
    const int CAMERA_STATE_ACTIVE = 1;
    const int CAMERA_STATE_IDLE = 2;
    const int CAMERA_STATE_CLOSED = 3;
    const int CAMERA_FACING_BACK = 0;
    const int CAMERA_FACING_FRONT = 1;
    const int CAMERA_FACING_EXTERNAL = 2;

    // CameraService 向代理服务发送消息，通知相机设备状态更新
    oneway void notifyCameraState(String cameraId, int facing, 
            int newCameraState, String clientName);
}

从 AIDL 文件看出，CameraServiceProxy 主要是响应 CameraService 的请求，也就是向 Framework Java 发送消息。

流程图

CameraServiceProxy 服务名称：media.camera.proxy ；CameraServiceProxy 继承了 SystemService ，注册流程如下：

0109-android-camera-framework-CameraServiceProxy-addService.png

源码分析

先来看注册流程的源码，服务的标准注册流程：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

// CameraServiceProxy.java
public class CameraServiceProxy extends SystemService
        implements Handler.Callback, IBinder.DeathRecipient {
    ...
    private static final String CAMERA_SERVICE_BINDER_NAME 
        = "media.camera";
    public static final String CAMERA_SERVICE_PROXY_BINDER_NAME 
        = "media.camera.proxy";
    ...
    // 构造方法中，初始化线程相关
    public CameraServiceProxy(Context context) {
        super(context);
        mContext = context;
        mHandlerThread = new ServiceThread(TAG, 
            Process.THREAD_PRIORITY_DISPLAY, /*allowTo*/false);
        mHandlerThread.start();
        mHandler = new Handler(mHandlerThread.getLooper(), this);

        mNotifyNfc = SystemProperties.getInt(NFC_NOTIFICATION_PROP, 0)>0;
        if (DEBUG) Slog.v(...);
    }
    
    // onStart 主要是注册服务，并监听 User 相关广播
    @Override
    public void onStart() {
        mUserManager = UserManager.get(mContext);
        ...
        IntentFilter filter = new IntentFilter();
        filter.addAction(Intent.ACTION_USER_ADDED);
        filter.addAction(Intent.ACTION_USER_REMOVED);
        filter.addAction(Intent.ACTION_USER_INFO_CHANGED);
        filter.addAction(Intent.ACTION_MANAGED_PROFILE_ADDED);
        filter.addAction(Intent.ACTION_MANAGED_PROFILE_REMOVED);
        mContext.registerReceiver(mIntentReceiver, filter);

        // 注册 CameraServiceProxy 服务
        publishBinderService(CAMERA_SERVICE_PROXY_BINDER_NAME, 
            mCameraServiceProxy);
        publishLocalService(CameraServiceProxy.class, this);
        CameraStatsJobService.schedule(mContext);
    }
    ...
}

类中的 User 指的是 Android 多用户；再看回调接口的实现：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

// CameraServiceProxy.java
private final ICameraServiceProxy.Stub mCameraServiceProxy 
        = new ICameraServiceProxy.Stub() {
    @Override
    public void pingForUserUpdate() {
        notifySwitchWithRetries(30);
    }

    @Override
    public void notifyCameraState(String cameraId, int newCameraState,
            int facing, String clientName) {
        String state = cameraStateToString(newCameraState);
        String facingStr = cameraFacingToString(facing);
        if (DEBUG) Slog.v(...);
        updateActivityCount(cameraId, newCameraState,facing,clientName);
    }
};

小结

通常情况下 API 中，CameraManager 通过 ICameraService.aidl 向 CameraService 下发请求；而 CameraService 通过 ICameraServiceListener.aidl 发回回调。
而如果没有 API 请求的情况下，CameraService 无法向 Framework Java 发送信息，所以系统开机时注册了 CameraServiceProxy 服务，用于响应 CameraService 的回调。

其他

Camera 相关声音

1
2
3
4
5
6
7
8
9
10
11
12
13
14

void CameraService::loadSound() {
    ATRACE_CALL();

    Mutex::Autolock lock(mSoundLock);
    LOG1("CameraService::loadSound ref=%d", mSoundRef);
    if (mSoundRef++) return;

    mSoundPlayer[SOUND_SHUTTER] = newMediaPlayer(
        "/system/media/audio/ui/camera_click.ogg");
    mSoundPlayer[SOUND_RECORDING_START] = newMediaPlayer(
        "/system/media/audio/ui/VideoRecord.ogg");
    mSoundPlayer[SOUND_RECORDING_STOP] = newMediaPlayer(
        "/system/media/audio/ui/VideoStop.ogg");
}

CameraMetadata

CameraMetadataNative 和 CameraMetadata 是同一个类型，只是命名空间不一样。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

namespace android {

...
class CameraMetadata: public Parcelable {
...
}

namespace hardware {
namespace camera2 {
namespace impl {
using ::android::CameraMetadata;
typedef CameraMetadata CameraMetadataNative;
}
}
}

常见问题

API 2 + HAL 1

平台仅支持 HAL 1 时，API 2 在 openCamera 时，通过 CameraDeviceUserShim 将 API 2 转换为 API 1 ，即 HAL 1 + API 1 向下发起请求。
LegacyCameraDevice 会将 CAMERA API2 转换为 CAMERA API1 ，而 CameraDeviceUserShim 封装了 LegacyCameraDevice 。

AIDL 生成多类型文件

AIDL 可以同时生成 .java, .h, .cpp 文件，编译规则在 Android.bp 中配置。

总结

后续

  • 数据传递
    Binder 通信机制，数据传输限制在 1M ，那整个通信机制是如何传递图片的呢？以及预览的呢？传递的是什么？
    createCaptureSession 创建捕获会话时，配置输出流；当 setRepeatingRequest 发起预览请求时，回调结果为 CaptureResult ，它是如何和输出流关联的呢？
  • SurfaceFlinger 显示相关知识
  • Buffer 相关管理
    这些都是 API 1 模式下的数据流，不过有参考意义。

参考文档

峻峰飞阳
关注
  • 3
    点赞
  • 踩
  • 17
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
Camera framework/HAL（一）
weixin_42196034的博客
07-19 741
目录 一、framework 1、开启camera： 1.1、openCameraDeviceUserAsync: 1.1.1、openCameraDeviceUserAsync 一、framework base/core/java/android/hardware/camera2/CameraManager.java 调用opencamera，根据传入的参数执行不同的接口，检查是否开启camera功能。 openCamera->openCameraForUid()->ope
【Android】 Camera Framework层分析
yihanss的博客
12-14 2575
Camera应用调用Framework Camera类API 在Android Kitkat原生Camera2应用（packages/apps/Camera2/）的PhotoModule, VideoModule, WideAnglePanoramaModule类中用CameraUtil.open()方法来打开Camera。然后依次调 用：CameraHolder的open()方法，AndroidCameraManagerImpl的cameraOpen()方 法，CameraHandler的handleMe
Android Camera Framework
10-09
三星方案Android Camera Framework介绍
Android系统获取摄像头预览数据的流程
我的博客
06-04 835
Android系统获取摄像头预览数据的流程
camera应用调用之---open camera流程（1）
weixin_43503508的博客
08-01 3421
网上这个文档应该相当多，早些年其实看文章简单的过了一下，并没有真正记住，但是再自己读着走遍流程并记录下，作为完整文档一部分。 安卓系统，各种Manager提供了系统层的控制接口，CameraManager也不例外，除去apk，那camera framework应该从CameraManager入口开始。 一般:CameraManager manager =（CameraManager）Context.getSystemService(Context.CAMERA_SERVICE) 可以获取CameraMana
android framework camera分析
03-04
详细解释android平台camera 部分的框架接口。为实现camera的硬件抽象层提供参考
Android Framework 分析
03-29
下面将对 Android Framework 进行深入分析，涵盖其目录树、camera HAL 接口库、cmds 命令、runtime 环境等多方面的知识点。 一、目录树分析 Android Framework 的目录树主要包含以下几个部分： 1. /framework/...
Camera Overview
最新发布
05-29
Camera Overview 1.Camera硬件框架 2.Camera软件框架 3.mm-camera-deamon简介 4.Api1/Api2 overview 5.HAL1/HAL3 overview 6.Native FrameWork简要说明 7.Native Crash问题分析 8.Native内存泄露分析及定位
android_Camera.7z
06-22
"android_Camera.7z"这个压缩包包含的资源，主要是对Android相机框架的深入分析，特别是聚焦于framework层面的重点流程。让我们详细探讨一下Android相机系统的关键知识点。 1. **Android相机框架概述**：Android...
camera调试：RK3588 MIPI/DVP camera关键配置
qq_34341546的博客
02-07 1万+
rk3588平台，camera调试
rk3568 android11 的 frameworks/ex架构学习纪要
__Benco的博客
07-29 414
rk3568 android11 的 frameworks/ex架构学习纪要
Camera Framework Service流程总结
zhangxinjieli3的专栏
04-28 923
文章目 前言 本文主要记录些Camera Servie的一些问题的理解，对几个问题进行讨论，一些参考Camera Service 一、代码位置 向上暴露的接口 具体实现 二、关系类图 三、问题讨论 1.Buffer是在哪分配的，它是怎么轮转的 1.Buffer的数量 status_t Camera3Device::HalInterface::configureStreams(const came.........
AForge库设置摄像头亮度以及设置视频镜像
qq_38889126的博客
10-10 667
不足之处还请各位指正 private delegate void FlushClient(); /// <summary> /// 调节摄像头亮度 /// </summary> /// <param name="sender"></param> /// <param name="e"></param> private void trackB
Android 8.0系统源码分析--Camera processCaptureResult结果回传源码分析
转载和创作优秀的博客
02-01 1115
相机，从上到下概览一下，真是太大了，上面的APP->Framework->CameraServer->CameraHAL，HAL进程中Pipeline、接各种算法的Node、再往下的ISP、3A、Driver，真是太大了，想把它搞懂真不是个简单的事情。不过我们奔着要把它搞懂的目标，一点点的啃，弄懂一点少一点，我们的功力也在不断的前进中一步步的增强。 本节，我们就来看一下HAL层一帧处理完成，通过HIDL定义的接口processCaptureResult将数据回传的逻辑。我自己...
android Camera framework层解析
Android_in_China
07-23 2375
Author : chi.lei@spreadtrum.com 转载请注明出处，谢谢！ 相机服务框架类相机服务是如何运作的呢？若弄清这个问题，必须先了解组成Service Framework的各个类与Bind RPC的连接关系。图-1描述了在不同的三个部分中各个类与Binder RPC的关系。 （a）       Camera 类继承自ICameraClient类，负责在应用程序与
Camera AIDL 代码分析
峻峰飞阳的博客 blog.csdn.net/while0
06-09 940
通过阅读本文您将收获以下知识点: 一、 Camera Framework 列文件目录 二、 JNI 相关 三、 AIDL 相关 四、 IInterface 类型文件 五、Parcelable 类型文件 六、ICameraService 相关 七、 ICameraServiceProxy.aidl 文件 八、 ICamera 相关 九、 ICameraDevice 相关 十、 Services 目录下的文件介绍 十一、API1/API2 十二、QTICamera2Client 十三、Device1/Dev
REbnu5/orqHph48ucHB0eA==
热门推荐
庵中十三居士的博客
05-02 7万+
REbnu5/orqHph48ucHB0eA==
linux下camera驱动分析
06-28
### 回答1： Linux下的Camera驱动分析主要涉及以下几个方面： 1.硬件支持：Linux内核需要支持相应的摄像头硬件，这需要在内核编译时进行配置。一般来说，Linux内核已经支持大部分常见的摄像头硬件。 2.驱动程序：Linux下的摄像头驱动程序通常是V4L2（Video for Linux 2）驱动程序。V4L2是Linux内核中的一个视频设备驱动框架，它提供了一套标准的API，用于访问视频设备。 3.应用程序：在Linux下，可以使用一些应用程序来访问摄像头，如Cheese、Guvcview等。这些应用程序使用V4L2 API来访问摄像头，从而实现视频采集、录制等功能。 总之，Linux下的Camera驱动分析需要了解V4L2驱动框架以及相关的硬件支持和应用程序。 ### 回答2： Linux下的Camera驱动是指针对硬件摄像头设备的驱动程序，通过驱动设备，可以让Linux系统匹配摄像头硬件设备的固件，并且使用Linux本身的API（接口）来进行图像处理。 目前，Linux系统已经支持了多种不同的摄像头设备，支持摄像头硬件设备的厂商包括Logitech、Creative、Microsoft等。这些厂商提供的摄像头设备在Linux系统下可以通过预装的驱动程序和用户自行下载的驱动程序安装来使用。 摄像头设备对于现代计算机系统来说已经是非常重要的一种外设，尤其是在多种应用场合下开始被广泛应用，如视频会议、视频监控、虚拟现实、图像识别等应用。Linux系统下摄像头驱动的分析和开发能力，对于这些应用的实现和推广也产生了很大的影响。 在Linux系统下，摄像头驱动程序主要有以下几个部分构成： 1. V4L核心驱动 V4L（Video4Linux）核心驱动是Linux系统中用于支持摄像头设备的核心驱动程序，主要提供一些通用的API函数，如打开、关闭、读取、写入、控制摄像头设备的函数等。 2. V4L2硬件驱动 V4L2（Video4Linux2）硬件驱动是针对具体的摄像头设备所编写的驱动程序。每种摄像头硬件设备都需要编写一个与之对应的V4L2驱动程序。这些驱动程序通常被编写为内核模块的形式，在用户开机时动态加载到内核中。 3. 应用层驱动程序 应用驱动程序位于Linux系统的上层，用于针对某一具体摄像头设备的用户界面和图形操作。这些驱动程序通常是由厂商针对自己提供的摄像头设备所编写的。 总体来说，Linux下的摄像头驱动程序的分析和开发，需要对Linux系统和硬件设备的特性有深入的了解，同时还需要具备一定的编程技巧和开发能力。对于想要从事Linux下摄像头驱动领域的工程师和科研人员来说，需要扎实的理论基础和丰富的实际经验。 ### 回答3： 在Linux操作系统中，相机驱动是内核中一个非常重要的部分，它可以控制系统中所有的相机设备。相机驱动的作用是将设备和操作系统进行连接，使得用户可以使用相机功能。 首先需要了解的是Linux内核下的相机驱动类型。目前常用的相机驱动有两种，一种是V4L2驱动，另一种是USB相机驱动。V4L2是一个通用的视频输入输出设备驱动，用于媒体框架，能够支持不同类型的视频捕捉设备。而USB相机驱动则是一种特定的相机设备驱动，只支持USB接口的相机设备。 其次需要了解的是相机驱动的工作原理。相机驱动通常是由内核模块或者用户空间程序来控制的。当用户需要使用相机时，应用程序将会通过系统调用请求操作系统，操作系统再通过设备节点与相机驱动进行交互，从而控制相机的工作。 最后需要注意的是相机驱动的配置。为了使相机驱动能够正确地运行，需要对其进行配置。配置的方法有两种，一种是在内核配置中进行设置，另一种是通过命令行或配置文件进行设置。在配置过程中需要考虑到硬件平台的适配性、相机驱动的版本等因素。 总之，在Linux下相机驱动的分析包括了驱动类型、工作原理和配置等多个方面，是建立在对Linux操作系统深入了解的基础上的重要工作。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值