本文均属自己阅读源码的点滴总结，转账请注明出处谢谢。
欢迎和大家交流。qq:1037701636 email:gzzaigcn2009@163.com
Software：系统源码Android5.1
前沿：
之前的两篇博文算是比较详细的记录了整个Camera3 HAL3架构下完全不同于HAL1的preview预览处理过程，包括主要涉及到的控制流和视频流等。比较详细的阐述了Camera2Client下streamProcessor、CallbackProcessor、CaptureSequencer等模块在Camera3架构下的功能。分析得出每个模块下均会在Camera3Device下以一个Stream的形式存在，而每个stream又是由多个buffer来构成主体的。与HAL3进行数据的交互时，以Request和result来作为数据传输的载体。在这些基础上本文将描述具体拍照Capture模式下的数据流和控制流，主要会涉及到jpegprocessor、CaptureSequencer这几个模块的工作原理。鉴于Capture模式下的数据流更复杂，在这里重点会分析数据流result回传时，每个模块的响应以及处理过程，填补前一博文的空白。
1. HAL3中Camera2Client下的take picture的入口函数
作为标准的capture picture功能的入口，主要完成了以下两件事情：
updateProcessorStream(mJpegProcessor, l.mParameters);
mCaptureSequencer->startCapture(msgType)
对于JpegProcessor模块而言，他的stream流第一次是在preview阶段进行了create与初始化，这里之所以再次调用JpegProcessor::updateStream目的是参考原先JpegProcessor stream的width与height是否变化即是否照片要求的分辨率发生了变化，如果是的话就需要delete原先的stream，重新建立一个stream。
在JpegProcessor中重点关注CpuConsumer与Surface的生产者与消费者处理模式，官方称之为Create CPU buffer queue endpoint。
2. CaptureSequencer模块
CaptureSequencer模块是take picture下操作的重点，在Camera2Client中进行了创建，首先来看CaptureSequencer线程的threadLoop函数：
bool CaptureSequencer::threadLoop() {
sp<Camera2Client> client = mClient.promote();
if (client == 0) return false;
CaptureState currentState;
{
Mutex::Autolock l(mStateMutex);
currentState = mCaptureState;
}
currentState = (this->*kStateManagers[currentState])(client);
Mutex::Autolock l(mStateMutex);
if (currentState != mCaptureState) {
if (mCaptureState != IDLE) {
ATRACE_ASYNC_END(kStateNames[mCaptureState], mStateTransitionCount);
}
mCaptureState = currentState;//保留新的状态
mStateTransitionCount++;
if (mCaptureState != IDLE) {
ATRACE_ASYNC_BEGIN(kStateNames[mCaptureState], mStateTransitionCount);
}
ALOGV("Camera %d: New capture state %s",
client->getCameraId(), kStateNames[mCaptureState]);
mStateChanged.signal();
}
if (mCaptureState == ERROR) {
ALOGE("Camera %d: Stopping capture sequencer due to error",
client->getCameraId());
return false;
}
return true;
}
const CaptureSequencer::StateManager
CaptureSequencer::kStateManagers[CaptureSequencer::NUM_CAPTURE_STATES-1] = {
&CaptureSequencer::manageIdle,
&CaptureSequencer::manageStart,
&CaptureSequencer::manageZslStart,
&CaptureSequencer::manageZslWaiting,
&CaptureSequencer::manageZslReprocessing,
&CaptureSequencer::manageStandardStart,
&CaptureSequencer::manageStandardPrecaptureWait,
&CaptureSequencer::manageStandardCapture,
&CaptureSequencer::manageStandardCaptureWait,
&CaptureSequencer::manageBurstCaptureStart,
&CaptureSequencer::manageBurstCaptureWait,
&CaptureSequencer::manageDone,
};
CaptureSequencer::CaptureState CaptureSequencer::manageIdle(
sp<Camera2Client> &/*client*/) {
status_t res;
Mutex::Autolock l(mInputMutex);
while (!mStartCapture) {
res = mStartCaptureSignal.waitRelative(mInputMutex,
kWaitDuration);
if (res == TIMED_OUT) break;
}
if (mStartCapture) {
mStartCapture = false;
mBusy = true;
return START;
}
return IDLE;
}
status_t CaptureSequencer::startCapture(int msgType) {
ALOGV("%s", __FUNCTION__);
ATRACE_CALL();
Mutex::Autolock l(mInputMutex);
if (mBusy) {
ALOGE("%s: Already busy capturing!", __FUNCTION__);
return INVALID_OPERATION;
}
if (!mStartCapture) {
mMsgType = msgType;
mStartCapture = true;
mStartCaptureSignal.signal();//启动CaptureSequencer
}
return OK;
}
2.1 START状态机
主要调用了updateCaptureRequest(l.mParameters, client)函数：
status_t CaptureSequencer::updateCaptureRequest(const Parameters ¶ms,
sp<Camera2Client> &client) {
ATRACE_CALL();
status_t res;
if (mCaptureRequest.entryCount() == 0) {
res = client->getCameraDevice()->createDefaultRequest(
CAMERA2_TEMPLATE_STILL_CAPTURE,
&mCaptureRequest);
if (res != OK) {
ALOGE("%s: Camera %d: Unable to create default still image request:"
" %s (%d)", __FUNCTION__, client->getCameraId(),
strerror(-res), res);
return res;
}
}
res = params.updateRequest(&mCaptureRequest);
if (res != OK) {
ALOGE("%s: Camera %d: Unable to update common entries of capture "
"request: %s (%d)", __FUNCTION__, client->getCameraId(),
strerror(-res), res);
return res;
}
res = params.updateRequestJpeg(&mCaptureRequest);//更新JPEG需要的参数
if (res != OK) {
ALOGE("%s: Camera %d: Unable to update JPEG entries of capture "
"request: %s (%d)", __FUNCTION__, client->getCameraId(),
strerror(-res), res);
return res;
}
return OK;
}2.2 STANDARD_START状态manageStandardCapture
该状态是启动整个take picture的重点所在：
CaptureSequencer::CaptureState CaptureSequencer::manageStandardCapture(
sp<Camera2Client> &client) {
status_t res;
ATRACE_CALL();
SharedParameters::Lock l(client->getParameters());
Vector<int32_t> outputStreams;
uint8_t captureIntent = static_cast<uint8_t>(ANDROID_CONTROL_CAPTURE_INTENT_STILL_CAPTURE);
/**
* Set up output streams in the request
* - preview
* - capture/jpeg
* - callback (if preview callbacks enabled)
* - recording (if recording enabled)
*/
outputStreams.push(client->getPreviewStreamId());//preview Stream
outputStreams.push(client->getCaptureStreamId());//capture Stream
if (l.mParameters.previewCallbackFlags &
CAMERA_FRAME_CALLBACK_FLAG_ENABLE_MASK) {
outputStreams.push(client->getCallbackStreamId());//capture callback
}
if (l.mParameters.state == Parameters::VIDEO_SNAPSHOT) {
outputStreams.push(client->getRecordingStreamId());
captureIntent = static_cast<uint8_t>(ANDROID_CONTROL_CAPTURE_INTENT_VIDEO_SNAPSHOT);
}
res = mCaptureRequest.update(ANDROID_REQUEST_OUTPUT_STREAMS,
outputStreams);
if (res == OK) {
res = mCaptureRequest.update(ANDROID_REQUEST_ID,
&mCaptureId, 1);//当前request对应的ID
}
if (res == OK) {
res = mCaptureRequest.update(ANDROID_CONTROL_CAPTURE_INTENT,
&captureIntent, 1);
}
if (res == OK) {
res = mCaptureRequest.sort();
}
if (res != OK) {
ALOGE("%s: Camera %d: Unable to set up still capture request: %s (%d)",
__FUNCTION__, client->getCameraId(), strerror(-res), res);
return DONE;
}
// Create a capture copy since CameraDeviceBase#capture takes ownership
CameraMetadata captureCopy = mCaptureRequest;
if (captureCopy.entryCount() == 0) {
ALOGE("%s: Camera %d: Unable to copy capture request for HAL device",
__FUNCTION__, client->getCameraId());
return DONE;
}
/**
* Clear the streaming request for still-capture pictures
* (as opposed to i.e. video snapshots)
*/
if (l.mParameters.state == Parameters::STILL_CAPTURE) {
// API definition of takePicture() - stop preview before taking pic
res = client->stopStream();
if (res != OK) {
ALOGE("%s: Camera %d: Unable to stop preview for still capture: "
"%s (%d)",
__FUNCTION__, client->getCameraId(), strerror(-res), res);
return DONE;
}
}
// TODO: Capture should be atomic with setStreamingRequest here
res = client->getCameraDevice()->capture(captureCopy);//启动camera3device的capture，提交capture request
if (res != OK) {
ALOGE("%s: Camera %d: Unable to submit still image capture request: "
"%s (%d)",
__FUNCTION__, client->getCameraId(), strerror(-res), res);
return DONE;
}
mTimeoutCount = kMaxTimeoutsForCaptureEnd;
return STANDARD_CAPTURE_WAIT;
}
CaptureSequencer::CaptureState CaptureSequencer::manageStandardCaptureWait(
sp<Camera2Client> &client) {
status_t res;
ATRACE_CALL();
Mutex::Autolock l(mInputMutex);
// Wait for new metadata result (mNewFrame)
while (!mNewFrameReceived) {
res = mNewFrameSignal.waitRelative(mInputMutex, kWaitDuration);//wait new 一帧metadata
if (res == TIMED_OUT) {
mTimeoutCount--;
break;
}
}
// Approximation of the shutter being closed
// - TODO: use the hal3 exposure callback in Camera3Device instead
if (mNewFrameReceived && !mShutterNotified) {
SharedParameters::Lock l(client->getParameters());
/* warning: this also locks a SharedCameraCallbacks */
shutterNotifyLocked(l.mParameters, client, mMsgType);
mShutterNotified = true;
}
// Wait until jpeg was captured by JpegProcessor
while (mNewFrameReceived && !mNewCaptureReceived) {
res = mNewCaptureSignal.waitRelative(mInputMutex, kWaitDuration);//等待JPEG数据
if (res == TIMED_OUT) {
mTimeoutCount--;
break;
}
}
if (mTimeoutCount <= 0) {
ALOGW("Timed out waiting for capture to complete");
return DONE;
}
if (mNewFrameReceived && mNewCaptureReceived) {//满足mNewFrameReceived
if (mNewFrameId != mCaptureId) {
ALOGW("Mismatched capture frame IDs: Expected %d, got %d",
mCaptureId, mNewFrameId);
}
camera_metadata_entry_t entry;
entry = mNewFrame.find(ANDROID_SENSOR_TIMESTAMP);
if (entry.count == 0) {
ALOGE("No timestamp field in capture frame!");
} else if (entry.count == 1) {
if (entry.data.i64[0] != mCaptureTimestamp) {
ALOGW("Mismatched capture timestamps: Metadata frame %" PRId64 ","
" captured buffer %" PRId64,
entry.data.i64[0],
mCaptureTimestamp);
}
} else {
ALOGE("Timestamp metadata is malformed!");
}
client->removeFrameListener(mCaptureId, mCaptureId + 1, this);
mNewFrameReceived = false;
mNewCaptureReceived = false;
return DONE;
}
return STANDARD_CAPTURE_WAIT;
}a:Vector<int32_t> outputStreams;
outputStreams.push(client->getPreviewStreamId());//preview Stream
outputStreams.push(client->getCaptureStreamId());//capture jpeg Stream
outputStreams.push(client->getCallbackStreamId());//capture callback
通过以上的操作，可以很清楚是看到，这里集合了take picture所需要使用到的stream流，对应的模块分别是：
streamProcessor、jpegProcessor、CallbackProcessor。
这个过程和Preview模式下是类似的，收集当前Camera2Client下的所有stream，并以stream的ID号作为区别。
b: 将当前操作所有的stream信息全部加入到CameraMetadata mCaptureRequest
res = mCaptureRequest.update(ANDROID_REQUEST_OUTPUT_STREAMS,
outputStreams);
if (res == OK) {
res = mCaptureRequest.update(ANDROID_REQUEST_ID,
&mCaptureId, 1);//当前request对应的ID
}
ANDROID_REQUEST_ID这项值表明，当前只存在3种Request类型：
预览Request mPreviewRequest： mPreviewRequestId(Camera2Client::kPreviewRequestIdStart),
拍照Request mCaptureRequest：mCaptureId(Camera2Client::kCaptureRequestIdStart),
录像Request mRecordingRequest： mRecordingRequestId(Camera2Client::kRecordingRequestIdStart),c. 对于STILL_CAPTURE类型的picture
client->stopStream()，实现的本质是res = device->clearStreamingRequest()，mRequestThread->clearRepeatingRequests(lastFrameNumber);
该函数是将之前Preview模式下的建立的captureRequest作delete处理，之前在预览模式下是将最终生产的capturelist加入到了一个mRepeatingRequests当中，这里通过clear使之为empty，即不会再发送Request和HAL3进行数据的交互。
d.Camera3Device capture函数
首先关注capture函数传入的参数为captureCopy，即CameraMetadata mCaptureRequest的一个copy值。
status_t Camera3Device::capture(CameraMetadata &request, int64_t* /*lastFrameNumber*/) {
ATRACE_CALL();
List<const CameraMetadata> requests;
requests.push_back(request);//对于一个CameraMetadata转为list
return captureList(requests, /*lastFrameNumber*/NULL);
}status_t Camera3Device::captureList(const List<const CameraMetadata> &requests,
int64_t *lastFrameNumber) {
ATRACE_CALL();
return submitRequestsHelper(requests, /*repeating*/false, lastFrameNumber);//非重复的，制定于拍照
}capture函数由Camera3Device来响应处理，其传入的mCaptureRequest转变为一个list，再交由submitRequestsHelper来处理，对比之前Preview下的处理方式，其startstream入口为setStreamingRequest->setStreamingRequestList->submitRequestsHelper。
这也表明了最终CameraMetadata类型的Request都是由submitRequestsHelper来完成的，所以convertMetadataListToRequestListLocked这个将CameraMetadata转换为List<sp<CaptureRequest> > RequestList的处理过程对两者来说都是一致的。但在后续处理时，对picture模式下的Request，其不再是repeating的处理，mRequestThread->queueRequestList()：
status_t Camera3Device::RequestThread::queueRequestList(
List<sp<CaptureRequest> > &requests,
/*out*/
int64_t *lastFrameNumber) {
Mutex::Autolock l(mRequestLock);
for (List<sp<CaptureRequest> >::iterator it = requests.begin(); it != requests.end();
++it) {
mRequestQueue.push_back(*it);
}......
unpauseForNewRequests();
return OK;
}最简单的理解是picture模式下是拍去几帧的数据流即可，Preview模式下是实时的获取帧，前者是几次one snop，后者是连续continuous。
到这里为止，可以说CaptureSequence已经完成了START状态机的处理。
e. 从START到STANDARD_CAPTURE_WAIT
该状态下对应的状态机处理函数为manageStandardCaptureWait：
CaptureSequencer::CaptureState CaptureSequencer::manageStandardCaptureWait(
sp<Camera2Client> &client) {
status_t res;
ATRACE_CALL();
Mutex::Autolock l(mInputMutex);
// Wait for new metadata result (mNewFrame)
while (!mNewFrameReceived) {
res = mNewFrameSignal.waitRelative(mInputMutex, kWaitDuration);//wait new 一帧metadata
if (res == TIMED_OUT) {
mTimeoutCount--;
break;
}
}
// Approximation of the shutter being closed
// - TODO: use the hal3 exposure callback in Camera3Device instead
if (mNewFrameReceived && !mShutterNotified) {
SharedParameters::Lock l(client->getParameters());
/* warning: this also locks a SharedCameraCallbacks */
shutterNotifyLocked(l.mParameters, client, mMsgType);
mShutterNotified = true;
}
// Wait until jpeg was captured by JpegProcessor
while (mNewFrameReceived && !mNewCaptureReceived) {
res = mNewCaptureSignal.waitRelative(mInputMutex, kWaitDuration);//等待JPEG数据
if (res == TIMED_OUT) {
mTimeoutCount--;
break;
}
}
if (mTimeoutCount <= 0) {
ALOGW("Timed out waiting for capture to complete");
return DONE;
}
if (mNewFrameReceived && mNewCaptureReceived) {//满足mNewFrameReceived
if (mNewFrameId != mCaptureId) {
ALOGW("Mismatched capture frame IDs: Expected %d, got %d",
mCaptureId, mNewFrameId);
}
camera_metadata_entry_t entry;
entry = mNewFrame.find(ANDROID_SENSOR_TIMESTAMP);
if (entry.count == 0) {
ALOGE("No timestamp field in capture frame!");
} else if (entry.count == 1) {
if (entry.data.i64[0] != mCaptureTimestamp) {
ALOGW("Mismatched capture timestamps: Metadata frame %" PRId64 ","
" captured buffer %" PRId64,
entry.data.i64[0],
mCaptureTimestamp);
}
} else {
ALOGE("Timestamp metadata is malformed!");
}
client->removeFrameListener(mCaptureId, mCaptureId + 1, this);
mNewFrameReceived = false;
mNewCaptureReceived = false;
return DONE;
}
return STANDARD_CAPTURE_WAIT;
}
f . Done State状态
这里先假设已经完成了wait这个状态，就会进入Done状态的执行函数manageDone()，最重要的部分如下：
if (mCaptureBuffer != 0 && res == OK) {
ATRACE_ASYNC_END(Camera2Client::kTakepictureLabel, takePictureCounter);
Camera2Client::SharedCameraCallbacks::Lock
l(client->mSharedCameraCallbacks);
ALOGV("%s: Sending still image to client", __FUNCTION__);
if (l.mRemoteCallback != 0) {
l.mRemoteCallback->dataCallback(CAMERA_MSG_COMPRESSED_IMAGE,
mCaptureBuffer, NULL);//回传压缩好的jpeg数据到上层
} else {
ALOGV("%s: No client!", __FUNCTION__);
}
}
mCaptureBuffer.clear();他将采集到的一帧jpeg压缩格式的图像，回传到APP层，便于后期写入到文件等。在以往Camera HAL1.0中这部分的数据回传玩玩都是由HAL层来完成的，这也给编码带来复杂度以及效率低下等问题。Google在Camera3.0中很好的封装了dataCallback以及notifyCallback的回调处理，将其转到Camera2Client下不同模块来做响应回调。
其中mCaptureBuffer是回传回来的真实的jpeg格式的图像数据，其本质是从stream中提取的一个buffer然后被copy到一个heap中，等待APP Callback完成后，就会释放。
完成了Done状态后，CaptureSequence又会再次进入到IDLE模式，等待下一次的take picture的处理。
3 picture模式下Camera3Device处理Request与result
对于picture模式下的Request处理，可以参考Preview模式下的RequestThread::threadLoop下的处理过程。这里主要分析result的响应过程：
在前面已经提到CaptureSequence需要wait两个signal，一般都是有其他模块来触发回调这个signal，我们先来定位这两个signal发出的位置：
void CaptureSequencer::onResultAvailable(const CaptureResult &result) {
ATRACE_CALL();
ALOGV("%s: New result available.", __FUNCTION__);
Mutex::Autolock l(mInputMutex);
mNewFrameId = result.mResultExtras.requestId;//返回帧所属的request id
mNewFrame = result.mMetadata;
if (!mNewFrameReceived) {
mNewFrameReceived = true;
mNewFrameSignal.signal();//buffer相应的result 信息,由FrameProcessor模块来触发listener
}
}
void CaptureSequencer::onCaptureAvailable(nsecs_t timestamp,
sp<MemoryBase> captureBuffer) {
ATRACE_CALL();
ALOGV("%s", __FUNCTION__);
Mutex::Autolock l(mInputMutex);
mCaptureTimestamp = timestamp;
mCaptureBuffer = captureBuffer;
if (!mNewCaptureReceived) {
mNewCaptureReceived = true;
mNewCaptureSignal.signal();//真实的一帧jpeg图像
}
}3.1.明确picture模式下，一次处理需要的stream数目
需要明确的是一次take picture需要的stream分别有JpegProcessor、CallbackProcessor、StreamingProcessor三种，第一个主要接收的是jpeg格式的帧图像，第二个主要接收的是一帧的preview模式下回调到APP的视频帧，而最后一个是直接获取一帧视频图像后直接进行显示用的视频帧。
3.2.帧数据回调响应的由来processCaptureResult函数：
无论是哪一个模块，数据回调响应最初的入口是HAL3的process_capture_result函数即processCaptureResult()函数，该函数的处理之所以复杂是因为HAL3.0中允许一次result回来的数据可以是不完整的，其中以3A相关的cameraMetadata的数据为主，这里需要说明每一帧的result回来时camera3_capture_result都是含有一个camera_metadata_t的，包含着一帧图像的各种信息tag字段，其中以3A信息为主。在processCaptureResult函数中由三个核心函数：
processPartial3AResult()：处理回传回来的部分cameraMetadata result数据；
returnOutputBuffers()：返回这次result中各个stream对应的buffer数据；
sendCaptureResult()：处理的是一次完整的cameraMetadata result数据；
3.3. FrameProcessor模块的帧Result响应，以3A回调处理为主
processPartial3AResult()函数与sendCaptureResult()函数都是将3A的result结果发送给FrameProcessor去作处理的，因为无论是Request还是result都是必然带有一个类似stream的cameraMetadata的，所以在这个模块有别于其他模块，故不需要单独的stream流来交互数据的。
if (isPartialResult) {
// Fire off a 3A-only result if possible
if (!request.partialResult.haveSent3A) {//返回的只是3A的数据
request.partialResult.haveSent3A =
processPartial3AResult(frameNumber,
request.partialResult.collectedResult,
request.resultExtras);// frame含有3A则notify 处理
}
}其内部需要确保目前收集到的result需要至少含有如下的tag的值，才算一次3A数据可True：
gotAllStates &= get3AResult(partial, ANDROID_CONTROL_AF_MODE,
&afMode, frameNumber);
gotAllStates &= get3AResult(partial, ANDROID_CONTROL_AWB_MODE,
&awbMode, frameNumber);
gotAllStates &= get3AResult(partial, ANDROID_CONTROL_AE_STATE,
&aeState, frameNumber);
gotAllStates &= get3AResult(partial, ANDROID_CONTROL_AF_STATE,
&afState, frameNumber);
gotAllStates &= get3AResult(partial, ANDROID_CONTROL_AWB_STATE,
&awbState, frameNumber); if (!gotAllStates) return false;接着对比着来看sendCaptureResult：
void Camera3Device::sendCaptureResult(CameraMetadata &pendingMetadata,
CaptureResultExtras &resultExtras,
CameraMetadata &collectedPartialResult,
uint32_t frameNumber) {
if (pendingMetadata.isEmpty())
return;
Mutex::Autolock l(mOutputLock);
// TODO: need to track errors for tighter bounds on expected frame number
if (frameNumber < mNextResultFrameNumber) {
SET_ERR("Out-of-order capture result metadata submitted! "
"(got frame number %d, expecting %d)",
frameNumber, mNextResultFrameNumber);
return;
}
mNextResultFrameNumber = frameNumber + 1;//下一帧
CaptureResult captureResult;
captureResult.mResultExtras = resultExtras;
captureResult.mMetadata = pendingMetadata;
if (captureResult.mMetadata.update(ANDROID_REQUEST_FRAME_COUNT,
(int32_t*)&frameNumber, 1) != OK) {
SET_ERR("Failed to set frame# in metadata (%d)",
frameNumber);
return;
} else {
ALOGVV("%s: Camera %d: Set frame# in metadata (%d)",
__FUNCTION__, mId, frameNumber);
}
// Append any previous partials to form a complete result
if (mUsePartialResult && !collectedPartialResult.isEmpty()) {
captureResult.mMetadata.append(collectedPartialResult);//
}
captureResult.mMetadata.sort();
// Check that there's a timestamp in the result metadata
camera_metadata_entry entry =
captureResult.mMetadata.find(ANDROID_SENSOR_TIMESTAMP);
if (entry.count == 0) {
SET_ERR("No timestamp provided by HAL for frame %d!",
frameNumber);
return;
}
// Valid result, insert into queue
List<CaptureResult>::iterator queuedResult =
mResultQueue.insert(mResultQueue.end(), CaptureResult(captureResult));
ALOGVV("%s: result requestId = %" PRId32 ", frameNumber = %" PRId64
", burstId = %" PRId32, __FUNCTION__,
queuedResult->mResultExtras.requestId,
queuedResult->mResultExtras.frameNumber,
queuedResult->mResultExtras.burstId);
mResultSignal.signal();//发送signal
}
该函数的主要工作是创建一个CaptureResult，可以看到对于之前帧回传回来的部分result，需要在这里进行组合成一帧完整的result。collectedPartialResult指的是当一次Request下发时，回传的result可能是分几次返回的，比如第一次的result只含有部分的信息，在第二次返回如果result已经被标记为完全上传回到Threadloop中，那么这里就需要对前几次的result进行组合，而前几次的result都是保存在当前帧的Request的，整个Request以唯一的一个framenumber作为索引，确保返回的result组合后是对应的同一个Request。
个人理解这个partialResult的处理机制是每次返回的Result并不一定包含了当前frameNumber帧号所需要的tag信息，而且这个每次回传的mNumPartialResults值是由HAL3.0层来决定的。在每次一的Result中，会收集
其中 isPartialResult = (result->partial_result < mNumPartialResults)决定了当前的Result是否还是一个处于partial Result的模式，是的话每次都进行collectResult，此外对于此模式下会收集3A的tag信息，调用processPartial3AResult来处理3A的值，而这个过程也是单列的处理。而一旦当前的Result返回处于非partial模式时，直接提取之前collect的Result并和当前的Result共同组成一个新的Capture Result。生成的CaptureResult会加入到mResultQueue队列。
至此分析完了HAL3返回的Captrue Result的处理过程，最终mResultSignal.signal()唤醒相应的等待线程，而这个过程就是由FrameProcessor模块来响应的。
FrameProcessorBase是一个FrameProcessor的基类，会启动一个Threadloop：
bool FrameProcessorBase::threadLoop() {
status_t res;
sp<CameraDeviceBase> device;
{
device = mDevice.promote();
if (device == 0) return false;
}
res = device->waitForNextFrame(kWaitDuration);
if (res == OK) {
processNewFrames(device);// 3A相关的处理等待
} else if (res != TIMED_OUT) {
ALOGE("FrameProcessorBase: Error waiting for new "
"frames: %s (%d)", strerror(-res), res);
}
return true;
}
status_t Camera3Device::waitForNextFrame(nsecs_t timeout) {
status_t res;
Mutex::Autolock l(mOutputLock);
while (mResultQueue.empty()) {//capture result 结果非空则继续执行
res = mResultSignal.waitRelative(mOutputLock, timeout);
if (res == TIMED_OUT) {
return res;
} else if (res != OK) {
ALOGW("%s: Camera %d: No frame in %" PRId64 " ns: %s (%d)",
__FUNCTION__, mId, timeout, strerror(-res), res);
return res;
}
}
return OK;
}线程被唤醒后调用processNewFrames来处理当前帧
void FrameProcessorBase::processNewFrames(const sp<CameraDeviceBase> &device) {
status_t res;
ATRACE_CALL();
CaptureResult result;
ALOGV("%s: Camera %d: Process new frames", __FUNCTION__, device->getId());
while ( (res = device->getNextResult(&result)) == OK) {
// TODO: instead of getting frame number from metadata, we should read
// this from result.mResultExtras when CameraDeviceBase interface is fixed.
camera_metadata_entry_t entry;
entry = result.mMetadata.find(ANDROID_REQUEST_FRAME_COUNT);
if (entry.count == 0) {
ALOGE("%s: Camera %d: Error reading frame number",
__FUNCTION__, device->getId());
break;
}
ATRACE_INT("cam2_frame", entry.data.i32[0]);
if (!processSingleFrame(result, device)) {//单独处理一帧
break;
}
if (!result.mMetadata.isEmpty()) {
Mutex::Autolock al(mLastFrameMutex);
mLastFrame.acquire(result.mMetadata);
}
}
if (res != NOT_ENOUGH_DATA) {
ALOGE("%s: Camera %d: Error getting next frame: %s (%d)",
__FUNCTION__, device->getId(), strerror(-res), res);
return;
}
return;
}
bool FrameProcessor::processSingleFrame(CaptureResult &frame,
const sp<CameraDeviceBase> &device) {//处理帧
sp<Camera2Client> client = mClient.promote();
if (!client.get()) {
return false;
}
bool isPartialResult = false;
if (mUsePartialResult) {
if (client->getCameraDeviceVersion() >= CAMERA_DEVICE_API_VERSION_3_2) {
isPartialResult = frame.mResultExtras.partialResultCount < mNumPartialResults;
} else {
camera_metadata_entry_t entry;
entry = frame.mMetadata.find(ANDROID_QUIRKS_PARTIAL_RESULT);
if (entry.count > 0 &&
entry.data.u8[0] == ANDROID_QUIRKS_PARTIAL_RESULT_PARTIAL) {
isPartialResult = true;
}
}
}
if (!isPartialResult && processFaceDetect(frame.mMetadata, client) != OK) {
return false;
}
if (mSynthesize3ANotify) {
process3aState(frame, client);
}
return FrameProcessorBase::processSingleFrame(frame, device);
}bool FrameProcessorBase::processSingleFrame(CaptureResult &result,
const sp<CameraDeviceBase> &device) {
ALOGV("%s: Camera %d: Process single frame (is empty? %d)",
__FUNCTION__, device->getId(), result.mMetadata.isEmpty());
return processListeners(result, device) == OK;//处理所有的listener
}
status_t FrameProcessorBase::processListeners(const CaptureResult &result,
const sp<CameraDeviceBase> &device) {
ATRACE_CALL();
camera_metadata_ro_entry_t entry;
// Check if this result is partial.
bool isPartialResult = false;
if (device->getDeviceVersion() >= CAMERA_DEVICE_API_VERSION_3_2) {
isPartialResult = result.mResultExtras.partialResultCount < mNumPartialResults;
} else {
entry = result.mMetadata.find(ANDROID_QUIRKS_PARTIAL_RESULT);
if (entry.count != 0 &&
entry.data.u8[0] == ANDROID_QUIRKS_PARTIAL_RESULT_PARTIAL) {
ALOGV("%s: Camera %d: This is a partial result",
__FUNCTION__, device->getId());
isPartialResult = true;
}
}
// TODO: instead of getting requestID from CameraMetadata, we should get it
// from CaptureResultExtras. This will require changing Camera2Device.
// Currently Camera2Device uses MetadataQueue to store results, which does not
// include CaptureResultExtras.
entry = result.mMetadata.find(ANDROID_REQUEST_ID);
if (entry.count == 0) {
ALOGE("%s: Camera %d: Error reading frame id", __FUNCTION__, device->getId());
return BAD_VALUE;
}
int32_t requestId = entry.data.i32[0];
List<sp<FilteredListener> > listeners;
{
Mutex::Autolock l(mInputMutex);
List<RangeListener>::iterator item = mRangeListeners.begin();
// Don't deliver partial results to listeners that don't want them
while (item != mRangeListeners.end()) {
if (requestId >= item->minId && requestId < item->maxId &&
(!isPartialResult || item->sendPartials)) {
sp<FilteredListener> listener = item->listener.promote();
if (listener == 0) {
item = mRangeListeners.erase(item);
continue;
} else {
listeners.push_back(listener);
}
}
item++;
}
}
ALOGV("%s: Camera %d: Got %zu range listeners out of %zu", __FUNCTION__,
device->getId(), listeners.size(), mRangeListeners.size());
List<sp<FilteredListener> >::iterator item = listeners.begin();
for (; item != listeners.end(); item++) {
(*item)->onResultAvailable(result);//所有注册的listener，告知有result返回
}
return OK;
}
其他模块如果想要listen FrameProcessor模块，可以调用registerListener来注册，保存在mRangeListeners之中，具体的接口如下：
status_t Camera2Client::registerFrameListener(int32_t minId, int32_t maxId,
wp<camera2::FrameProcessor::FilteredListener> listener, bool sendPartials) {
return mFrameProcessor->registerListener(minId, maxId, listener, sendPartials);
}
在这个对完整的Result的处理过程中，重点关注FrameProcessor下的3A回调与人脸检测回调，3A中的AF回回传AF的状态信息以CAMERA_MSG_FOCUS的形式通过notifyCallback. FaceDetect会以camera_frame_metadata_t的形式将人脸检测的定位的数据通过dataCallback回传，数据类型为CAMERA_MSG_PREVIEW_FRAME。
有了这些listener，在processListeners处理函数中，通过遍历mRangeListeners，来确保当前的CaptureResult 中对象的Request id和注册时的区间相匹配。在提取到适合处理当前Result的listener后，回调onResultAvailable()函数。
到这里void CaptureSequencer::onResultAvailable()就会被覆盖调用，经而我们定位到了mNewFrameReceived = Ture的回调过程。
3.4. 帧数据的回调：
上面重点是分析一个队CameraMetadata Result结果的分析，看上去还没有真正的视频帧数据的出现。对于视频流buffer的操作，上面提到了肯定是需要stream的，而不像FrameProcessor不需要建立stream来进行数据的传输。
对于数据的Callback处理，接口是returnOutputBuffers函数，该函数在preview模式下已经进行过分析，其重点就是将当前Result回来的buffer数据信息进行提取，然后分发给不同模块所维护着的Camera3Stream去作处理，本质是将当前Result返回的camera3_stream_buffer提取buffer_handle后通过queue_buffer操作后，就交由对应的Consumer去作处理。
对于直接预览的模块StreamProcessor，其Consumer是直接的SurfaceFlinger用于实时显示，CallbackProcessor是将CpuConsumer来将帧数据回传给APP使用，这些过程和Preview模式下都是类似，也是takepicture模式下同样需要处理的过程，而对于JpegProcessor而言，属于Picture模式专属，我们来看他接收到一帧HAL3 Jpeg Buffer的处理过程：
void JpegProcessor::onFrameAvailable(const BufferItem& /*item*/) {
Mutex::Autolock l(mInputMutex);
if (!mCaptureAvailable) {
mCaptureAvailable = true;
mCaptureAvailableSignal.signal();//采集到一帧jpeg图像
}
}
bool JpegProcessor::threadLoop() {
status_t res;
{
Mutex::Autolock l(mInputMutex);
while (!mCaptureAvailable) {
res = mCaptureAvailableSignal.waitRelative(mInputMutex,
kWaitDuration);
if (res == TIMED_OUT) return true;
}
mCaptureAvailable = false;
}
do {
res = processNewCapture();//处理新的jpeg采集帧
} while (res == OK);
return true;
}mCaptureConsumer->lockNextBuffer(&imgBuffer);这是从CPUConsumer中获取一个已经queuebuffer的buffer，lock过程最重要的是将这个buffer作mmap操作后映射到当前线程中。
然后通过这个虚拟地址将buffer地址copy到本进程的一个heap中，随后将这个buffer进行ummap操作。
最后是调用如下代码，去将本地的jpegbuffer传输给CaptureSequencer，所以可以说CaptureSequence虽然负责收集jpeg等数据，负责整个take picture的启动与控制，但本质上jpeg等数据的真正提取都是交由jpegprocessor、zslprocessor等模块来完成：
sp<CaptureSequencer> sequencer = mSequencer.promote();
if (sequencer != 0) {
sequencer->onCaptureAvailable(imgBuffer.timestamp, captureBuffer);//通知capturesequence有jpeg buffer到了
}到这里，就解决了CaptureSequeuer的wait状态机中的另一个wait等待的signal。
至此为止onResultAvailable()与onCaptureAvailable()均完成了回调，前者主要是由FrameProcessor来触发的，后者是有jpegProcessor来触发的，前者是回传的一帧jpeg图像的附加信息如timestamp/3A等，而后者是回传了一帧真正的jpeg图像。
下面是我小节的takepicture模式下几个模块间数据交互的过程图，本质是几个线程间Threadloop的响应与处理过程。
可以看到jpeg模式下，每次课回传给APP的数据包括原始的Callback数据流，jpegprocessor中的jpeg图像流，以及其他比如AF的状态，人脸识别后的人脸坐标原始信息camera_frame_metadata_t回传给APP。
2789
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
