相机，从上到下概览一下，真是太大了，上面的APP->Framework->CameraServer->CameraHAL，HAL进程中Pipeline、接各种算法的Node、再往下的ISP、3A、Driver，真是太大了，想把它搞懂真不是个简单的事情。不过我们奔着要把它搞懂的目标，一点点的啃，弄懂一点少一点，我们的功力也在不断的前进中一步步的增强。
本节，我们就来看一下HAL层一帧处理完成，通过HIDL定义的接口processCaptureResult将数据回传的逻辑。我自己使用的Android8.0的系统源码是通过百度云盘分享的，大家可从Android 8.0系统源码分析--开篇中下载，百度云盘的下载链接和密码都有。
Camera系统也提供了非常多的dump方式，可以dump metadata、dump buffer，对我们分析问题都有非常大的帮助。
1、MTK系统中camera sensor的信息会通过kernal中imagesensor.c中的逻辑将所有camera sensor info写入到proc/driver/camera_info文件中，我们可以直接cat proc/driver/camera_info查看camera sensor的详细信息，其中记录了sensor类型，比如IMX386_mipi_raw；记录了sensor支持的分辨率大小，还有一些其他原始信息；
2、我们可以执行adb shell dumpsys media.camera > camerainfo.txt收集camera metadata信息，大家可以试一下，从这里可以轻易得到很多camera metadata信息；
3、android-8.0.0\system\media\camera\docs\docs.html文件中定义了AOSP所有的metadata，我们可以直接双击用浏览器打开它，就可以看到所有metadata，当然，每个SOC厂商肯定会自己新增一些，一般metadata的命名都是下划线拼接的，比如android.scaler.cropRegion，它的命名为ANDROID_SCALER_CROP_REGION，我们在HAL进程中就可以通过它去查询当前cropRegion的值。
4、Google提供的基于Camera API2 Demo地址：android-Camera2Basic（普通的预览拍照功能）、android-Camera2Video（录像功能）。
HIDL接口的定义有ICameraDevice.hal、ICameraDeviceSession.hal、ICameraDeviceCallback.hal，文件目录在hardware\interfaces\camera\device\xx目录下，xx为版本号，我所下载的源码有1.0和3.2两个版本，我们来看一下3.2版本下ICameraDeviceCallback.hal的定义，源码如下：
package android.hardware.camera.device@3.2;
import android.hardware.camera.common@1.0::types;
/**
*
* Callback methods for the HAL to call into the framework.
*
* These methods are used to return metadata and image buffers for a completed
* or failed captures, and to notify the framework of asynchronous events such
* as errors.
*
* The framework must not call back into the HAL from within these callbacks,
* and these calls must not block for extended periods.
*
*/
interface ICameraDeviceCallback {
/**
* processCaptureResult:
*
* Send results from one or more completed or partially completed captures
* to the framework.
* processCaptureResult() may be invoked multiple times by the HAL in
* response to a single capture request. This allows, for example, the
* metadata and low-resolution buffers to be returned in one call, and
* post-processed JPEG buffers in a later call, once it is available. Each
* call must include the frame number of the request it is returning
* metadata or buffers for. Only one call to processCaptureResult
* may be made at a time by the HAL although the calls may come from
* different threads in the HAL.
*
* A component (buffer or metadata) of the complete result may only be
* included in one process_capture_result call. A buffer for each stream,
* and the result metadata, must be returned by the HAL for each request in
* one of the processCaptureResult calls, even in case of errors producing
* some of the output. A call to processCaptureResult() with neither
* output buffers or result metadata is not allowed.
*
* The order of returning metadata and buffers for a single result does not
* matter, but buffers for a given stream must be returned in FIFO order. So
* the buffer for request 5 for stream A must always be returned before the
* buffer for request 6 for stream A. This also applies to the result
* metadata; the metadata for request 5 must be returned before the metadata
* for request 6.
*
* However, different streams are independent of each other, so it is
* acceptable and expected that the buffer for request 5 for stream A may be
* returned after the buffer for request 6 for stream B is. And it is
* acceptable that the result metadata for request 6 for stream B is
* returned before the buffer for request 5 for stream A is. If multiple
* capture results are included in a single call, camera framework must
* process results sequentially from lower index to higher index, as if
* these results were sent to camera framework one by one, from lower index
* to higher index.
*
* The HAL retains ownership of result structure, which only needs to be
* valid to access during this call.
*
* The output buffers do not need to be filled yet; the framework must wait
* on the stream buffer release sync fence before reading the buffer
* data. Therefore, this method should be called by the HAL as soon as
* possible, even if some or all of the output buffers are still in
* being filled. The HAL must include valid release sync fences into each
* output_buffers stream buffer entry, or -1 if that stream buffer is
* already filled.
*
* If the result buffer cannot be constructed for a request, the HAL must
* return an empty metadata buffer, but still provide the output buffers and
* their sync fences. In addition, notify() must be called with an
* ERROR_RESULT message.
*
* If an output buffer cannot be filled, its status field must be set to
* STATUS_ERROR. In addition, notify() must be called with a ERROR_BUFFER
* message.
*
* If the entire capture has failed, then this method still needs to be
* called to return the output buffers to the framework. All the buffer
* statuses must be STATUS_ERROR, and the result metadata must be an
* empty buffer. In addition, notify() must be called with a ERROR_REQUEST
* message. In this case, individual ERROR_RESULT/ERROR_BUFFER messages
* must not be sent.
*
* Performance requirements:
*
* This is a non-blocking call. The framework must handle each CaptureResult
* within 5ms.
*
* The pipeline latency (see S7 for definition) should be less than or equal to
* 4 frame intervals, and must be less than or equal to 8 frame intervals.
*
*/
processCaptureResult(vec<CaptureResult> results);
/**
* notify:
*
* Asynchronous notification callback from the HAL, fired for various
* reasons. Only for information independent of frame capture, or that
* require specific timing. Multiple messages may be sent in one call; a
* message with a higher index must be considered to have occurred after a
* message with a lower index.
*
* Multiple threads may call notify() simultaneously.
*
* Buffers delivered to the framework must not be dispatched to the
* application layer until a start of exposure timestamp (or input image's
* start of exposure timestamp for a reprocess request) has been received
* via a SHUTTER notify() call. It is highly recommended to dispatch this
* call as early as possible.
*
* ------------------------------------------------------------------------
* Performance requirements:
*
* This is a non-blocking call. The framework must handle each message in 5ms.
*/
notify(vec<NotifyMsg> msgs);
};
看到这些大段大段的注释，就能明白，这些接口肯定都是非常重要的，详细的注释也是很好的习惯，看Android的源码感觉确实很舒服，难怪人家的代码能成为标准，不管是命名，格式，注释，分包，分类，所有的地方都让人感觉舒服！！
好了，进入到我们本节的主题吧，ICameraDeviceCallback是HIDL定义的回调接口，processCaptureResult方法就是从HAL层回调到CameraServer的接口，CameraServer这一侧的回调类就是Camera3Device，因为在openCamera时，构造出来的Camera3Device进行初始化，Camera3Device类的initialize方法中与HAL进行连接，获取session时，将自己作为callback回调类传递到了HAL，所以后续HAL就会回调到Camera3Device类的processCaptureResult方法当中。
我们再来回顾一下Camera3Device类的initialize方法，Camera3Device文件的目录路径为frameworks\av\services\camera\libcameraservice\device3\Camera3Device.cpp，initialize方法的源码如下：
status_t Camera3Device::initialize(sp<CameraProviderManager> manager) {
ATRACE_CALL();
Mutex::Autolock il(mInterfaceLock);
Mutex::Autolock l(mLock);
ALOGV("%s: Initializing HIDL device for camera %s", __FUNCTION__, mId.string());
if (mStatus != STATUS_UNINITIALIZED) {
CLOGE("Already initialized!");
return INVALID_OPERATION;
}
if (manager == nullptr) return INVALID_OPERATION;
sp<ICameraDeviceSession> session;
ATRACE_BEGIN("CameraHal::openSession");
status_t res = manager->openSession(mId.string(), this,
/*out*/ &session);
ATRACE_END();
if (res != OK) {
SET_ERR_L("Could not open camera session: %s (%d)", strerror(-res), res);
return res;
}
res = manager->getCameraCharacteristics(mId.string(), &mDeviceInfo);
if (res != OK) {
SET_ERR_L("Could not retrive camera characteristics: %s (%d)", strerror(-res), res);
session->close();
return res;
}
std::shared_ptr<RequestMetadataQueue> queue;
auto requestQueueRet = session->getCaptureRequestMetadataQueue(
[&queue](const auto& descriptor) {
queue = std::make_shared<RequestMetadataQueue>(descriptor);
if (!queue->isValid() || queue->availableToWrite() <= 0) {
ALOGE("HAL returns empty request metadata fmq, not use it");
queue = nullptr;
// don't use the queue onwards.
}
});
if (!requestQueueRet.isOk()) {
ALOGE("Transaction error when getting request metadata fmq: %s, not use it",
requestQueueRet.description().c_str());
return DEAD_OBJECT;
}
auto resultQueueRet = session->getCaptureResultMetadataQueue(
[&queue = mResultMetadataQueue](const auto& descriptor) {
queue = std::make_unique<ResultMetadataQueue>(descriptor);
if (!queue->isValid() || queue->availableToWrite() <= 0) {
ALOGE("HAL returns empty result metadata fmq, not use it");
queue = nullptr;
// Don't use the queue onwards.
}
});
if (!resultQueueRet.isOk()) {
ALOGE("Transaction error when getting result metadata queue from camera session: %s",
resultQueueRet.description().c_str());
return DEAD_OBJECT;
}
mInterface = std::make_unique<HalInterface>(session, queue);
std::string providerType;
mVendorTagId = manager->getProviderTagIdLocked(mId.string());
return initializeCommonLocked();
}
该方法中就是调用manager->openSession(mId.string(), this, /*out*/ &session)来打开session的，manager是方法入参sp<CameraProviderManager>，调用openSession方法的第二个参数就是回调接口，传值为this，第三个是输出参数，当session在HAL进程中创建成功，则会通过回调赋值给这个输出参数。进一步来看一下CameraProviderManager类的openSession方法的实现，源码如下：
status_t CameraProviderManager::openSession(const std::string &id,
const sp<hardware::camera::device::V3_2::ICameraDeviceCallback>& callback,
/*out*/
sp<hardware::camera::device::V3_2::ICameraDeviceSession> *session) {
std::lock_guard<std::mutex> lock(mInterfaceMutex);
auto deviceInfo = findDeviceInfoLocked(id,
/*minVersion*/ {3,0}, /*maxVersion*/ {4,0});
if (deviceInfo == nullptr) return NAME_NOT_FOUND;
auto *deviceInfo3 = static_cast<ProviderInfo::DeviceInfo3*>(deviceInfo);
Status status;
hardware::Return<void> ret;
ret = deviceInfo3->mInterface->open(callback, [&status, &session]
(Status s, const sp<device::V3_2::ICameraDeviceSession>& cameraSession) {
status = s;
if (status == Status::OK) {
*session = cameraSession;
}
});
if (!ret.isOk()) {
ALOGE("%s: Transaction error opening a session for camera device %s: %s",
__FUNCTION__, id.c_str(), ret.description().c_str());
return DEAD_OBJECT;
}
return mapToStatusT(status);
}
这里的deviceInfo3->mInterface->open就会通过HIDL进入到CameraHalServer进程当中了。
好，回过头来看Camera3Device类的processCaptureResult方法，源码如下：
// Only one processCaptureResult should be called at a time, so
// the locks won't block. The locks are present here simply to enforce this.
hardware::Return<void> Camera3Device::processCaptureResult(
const hardware::hidl_vec<
hardware::camera::device::V3_2::CaptureResult>& results) {
if (mProcessCaptureResultLock.tryLock() != OK) {
// This should never happen; it indicates a wrong client implementation
// that doesn't follow the contract. But, we can be tolerant here.
ALOGE("%s: callback overlapped! waiting 1s...",
__FUNCTION__);
if (mProcessCaptureResultLock.timedLock(1000000000 /* 1s */) != OK) {
ALOGE("%s: cannot acquire lock in 1s, dropping results",
__FUNCTION__);
// really don't know what to do, so bail out.
return hardware::Void();
}
}
for (const auto& result : results) {
processOneCaptureResultLocked(result);
}
mProcessCaptureResultLock.unlock();
return hardware::Void();
}
最低0.47元/天 解锁文章
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
