Android Camera数据流分析全程记录（overlay方式）

最新推荐文章于 2024-05-04 00:30:25 发布

qianxuedegushi

最新推荐文章于 2024-05-04 00:30:25 发布

阅读量460

点赞数

分类专栏： Android camera

Android camera 专栏收录该内容

2 篇文章 0 订阅

订阅专栏

Android Camera数据流分析全程记录（overlay方式）

备注：转自 - conceptcon-ChinaUnix博客

这里为什么要研究overlay方式呢？android camera需要driver和app层需要有大量数据需要传输，如果使用非overlay方式进行数据从driver到app层的传输，使系统性能受到很到影响，使系统速度变慢，同时会影响功耗等，而在camera preview module时，通常我们是不必要将采集的数据保存下来的，而不像录像module下，需要将数据保存下来，所以overlay方式就是不经过数据回传，直接显示从driver的数据方式，采用这种方式app从无法获取到数据，所以这种方式应用在preview方式下这里我是针对android4.0版本的，相对android2.x版本的overlay已经发生了很大的变化，想要研究这方面的可以自己去了解一下，这里不再多说了开始部分我就直接在这里带过了，系统初始打开camera时，调用到app的onCreate方法，这里主要做了一下工作：
1.开始一个openCamera线程打开camera
2.实例化很多的对象，用于camera工作使用
3.实例化surfaceview和surfaceholder,并且填充了其中的surfacechanged，surfacedestoryed和surfacecreated这三个方式
4.开始一个preview线程用于preview过程
这其中3.4是我们这里要关注的重点，上面实例化了这个surfaceview将决定了我们到底是否使用overlay方式
在这里第三遍完成之后，系统会自动执行surfacechanged这个方式，每次显示区域发生改变都会自动调用这个方法，刚开始打开camera时，显示区域从无到有，因此必要这里会想调用到surfacechanged方法
我们就还是看看在这里都做了些什么事情

public void surfaceChanged(SurfaceHolder holder,int format,int w,int h){
// Make sure we have a surface in the holder before proceeding.
if(holder.getSurface()==null){
Log.d(TAG,"holder.getSurface() == null");
return;
}
Log.v(TAG,"surfaceChanged. w="+ w +". h="+ h);
// We need to save the holder for later use, even when the mCameraDevice
//isnull. This could happen if onResume()is invoked after this
//function.
mSurfaceHolder = holder;
// The mCameraDevice will be nullif it fails to connect to the camera
// hardware.In this case we will show a dialog andthen finish the
// activity, so it's OK to ignore it.
if(mCameraDevice ==null) return;
// Sometimes surfaceChanged is called after onPause or before onResume.
// Ignore it.
if(mPausing || isFinishing()) return;
setSurfaceLayout();
//Set preview display if the surface is being created. Preview was
// already started. Also restart the preview if display rotation has
// changed. Sometimes this happens when the device is held in portrait
//and camera app is opened. Rotation animation takes some timeand
// display rotation in onCreate may not be what we want.
if(mCameraState == PREVIEW_STOPPED){//这里表示第一次打开camera时，那么调用startpreview
startPreview(true);
startFaceDetection();
}else{//这里则表示camera已经打开过程中发生的显示变化，比如横屏竖频转换，所以zheli只需要重新设置previewdisplay
if(Util.getDisplayRotation(this)!= mDisplayRotation){
setDisplayOrientation();
}
if(holder.isCreating()){
//Set preview display if the surface is being created and preview
// was already started. That means preview display was settonull
//and we need toset it now.
setPreviewDisplay(holder);
}
}
//If first time initialization isnot finished, send a message todo
// it later. We want to finish surfaceChanged as soon as possible tolet
// user see preview first.
if(!mFirstTimeInitialized){
mHandler.sendEmptyMessage(FIRST_TIME_INIT);
}else{
initializeSecondTime();
}
SurfaceView preview =(SurfaceView) findViewById(R.id.camera_preview);
CameraInfo info = CameraHolder.instance().getCameraInfo()[mCameraId];
boolean mirror =(info.facing == CameraInfo.CAMERA_FACING_FRONT);
int displayRotation = Util.getDisplayRotation(this);
int displayOrientation = Util.getDisplayOrientation(displayRotation, mCameraId);
mTouchManager.initialize(preview.getHeight()/ 3, preview.getHeight()/ 3,
preview, this, mirror, displayOrientation);
}

从上面代码我们必须知道，在surface发生变化时必须调用setPreviewDisplay，根据之后的学习，在startpreview方式中真正startpreview之前同样要调用setPreviewDisplay，在setPreviewDisplay的方法中完成了很多初始化，也是在这里决定是否使用overlay方式的，我们就先看看startpreview这个方法吧

private void startPreview(boolean updateAll){
if(mPausing || isFinishing()) return;
mFocusManager.resetTouchFocus();
mCameraDevice.setErrorCallback(mErrorCallback);
//If we're previewing already, stop the preview first (this will blank
// the screen).
if(mCameraState != PREVIEW_STOPPED) stopPreview();
setPreviewDisplay(mSurfaceHolder);
setDisplayOrientation();
if(!mSnapshotOnIdle){
//If the focus mode is continuous autofocus,call cancelAutoFocus to
//resume it because it may have been paused by autoFocus call.
if(Parameters.FOCUS_MODE_CONTINUOUS_PICTURE.equals(mFocusManager.getFocusMode())){
mCameraDevice.cancelAutoFocus();
}
mFocusManager.setAeAwbLock(false);// Unlock AE and AWB.
}
if( updateAll ){
Log.v(TAG,"Updating all parameters!");
setCameraParameters(UPDATE_PARAM_INITIALIZE | UPDATE_PARAM_ZOOM | UPDATE_PARAM_PREFERENCE);
}else{
setCameraParameters(UPDATE_PARAM_MODE);
}
//setCameraParameters(UPDATE_PARAM_ALL);
// Inform the mainthread to go on the UI initialization.
if(mCameraPreviewThread !=null){
synchronized (mCameraPreviewThread){
mCameraPreviewThread.notify();
}
}
try {
Log.v(TAG,"startPreview");
mCameraDevice.startPreview();
} catch (Throwable ex){
closeCamera();
throw new RuntimeException("startPreview failed", ex);
}
mZoomState = ZOOM_STOPPED;
setCameraState(IDLE);
mFocusManager.onPreviewStarted();
if( mTempBracketingEnabled ){
mFocusManager.setTempBracketingState(FocusManager.TempBracketingStates.ACTIVE);
}
if(mSnapshotOnIdle){
mHandler.post(mDoSnapRunnable);
}
}

上面大家看到了，先调用了setPreviewDisplay，最后调用mCameraDevice.startPreview()开始preview
这里过程如下：app-->frameworks-->JNI-->camera client-->camera service-->hardware interface-->HAL
1.setPreviewDisplay方法调用时在app层最初的传入的参数是surfaceholder结构
2.到了JNI层setPreviewDisplay方法传入的参数已经是surface结构了
3.到了camera service层
spbinder(surface != 0 ? surface->asBinder() : 0);
spwindow(surface);
return setPreviewWindow(binder, window);
通过上面的转换调用同名不同参数的另外一个方法，到这里调用的参数已经转变为IBinder和ANativeWindow
4.调用hardware interface的setPreviewWindow(window)，这里只有一个ANativeWindow类型的参数
5.到了camerahal_module中转站时又发生了变化，看看下面的定义，参数变为preview_stream_ops 这个类型的结构
int camera_set_preview_window(struct camera_device * device, struct preview_stream_ops *window)
上面过程参数类型一直在变化，不过从app层一直传到这里，其实是对同一个内存地址的传输，就像张三换了身衣服，但是他还是张三一样
现在我们就直接看看HAL层的实现

/**
@brief Sets ANativeWindow object.
Preview buffers provided to CameraHal via this object. DisplayAdapter will be interfacing with it
to render buffers to display.
@param[in]window The ANativeWindow object created by Surface flinger
@return NO_ERROR If the ANativeWindow object passes validation criteria
@todo Define validation criteria for ANativeWindow object. Define error codes for scenarios
*/
status_t CameraHal::setPreviewWindow(struct preview_stream_ops *window)
{
status_t ret = NO_ERROR;
CameraAdapter::BuffersDescriptor desc;
LOG_FUNCTION_NAME;
mSetPreviewWindowCalled =true;
///If the Camera service passes a nullwindow, we destroy existing windowand free the DisplayAdapter
if(!window)//这种情况下，window是null，表示不采用overlay方式，则不需要新建displayadapter
{
if(mDisplayAdapter.get()!=NULL)
{
///NULLwindow passed, destroy the display adapter if present
CAMHAL_LOGD("NULL window passed, destroying display adapter");
mDisplayAdapter.clear();
///@remarks If there was a window previously existing, we usually expect another valid windowto be passed by the client
///@remarks so, we will wait until it passes a valid windowto begin the preview again
mSetPreviewWindowCalled =false;
}
CAMHAL_LOGD("NULL ANativeWindow passed to setPreviewWindow");
return NO_ERROR;
}elseif(mDisplayAdapter.get()==NULL)//传入的window不是null，但是还没有未使用overlay方式创建displayadapter，创建displayadapter
{
// Need to create the display adapter since it has not been created
// Create display adapter
mDisplayAdapter = new ANativeWindowDisplayAdapter();
ret = NO_ERROR;
if(!mDisplayAdapter.get()||((ret=mDisplayAdapter->initialize())!=NO_ERROR))
{
if(ret!=NO_ERROR)
{
mDisplayAdapter.clear();
CAMHAL_LOGEA("DisplayAdapter initialize failed");
LOG_FUNCTION_NAME_EXIT;
return ret;
}
else
{
CAMHAL_LOGEA("Couldn't create DisplayAdapter");
LOG_FUNCTION_NAME_EXIT;
return NO_MEMORY;
}
}
// DisplayAdapter needs to know where toget the CameraFrames from inorder to display
// Since CameraAdapter is the one that provides the frames,set it as the frame provider for DisplayAdapter
mDisplayAdapter->setFrameProvider(mCameraAdapter);
// Any dynamic errors that happen during the camera use case has to be propagated back to the application
// via CAMERA_MSG_ERROR. AppCallbackNotifier is the class that notifies such errors to the application
//Set it as the error handler for the DisplayAdapter
mDisplayAdapter->setErrorHandler(mAppCallbackNotifier.get());
// Update the display adapter with the new window that is passed from CameraService
ret = mDisplayAdapter->setPreviewWindow(window);
if(ret!=NO_ERROR)
{
CAMHAL_LOGEB("DisplayAdapter setPreviewWindow returned error %d", ret);
}
if(mPreviewStartInProgress)
{
CAMHAL_LOGDA("setPreviewWindow called when preview running");
// Start the preview since the windowisnow available
ret = startPreview();
}
}else{//传入的window不是null，并且displaadaper已经创建好，那么这里只需要将新的window与已经创建好的displayadapter关联即可
// Update the display adapter with the new window that is passed from CameraService
ret = mDisplayAdapter->setPreviewWindow(window);
if((NO_ERROR == ret)&& previewEnabled()){
restartPreview();
}elseif(ret == ALREADY_EXISTS){
// ALREADY_EXISTS should be treated as a noop in this case
ret = NO_ERROR;
}
}
LOG_FUNCTION_NAME_EXIT;
return ret;
}

这里我们重点看看新建displayadapter的过程：
1.实例化一个ANativeWindowDisplayAdapter对象
2.mDisplayAdapter->initialize()
3.mDisplayAdapter->setFrameProvider(mCameraAdapter)//这一步是关键，之后会遇到的
4.mDisplayAdapter->setErrorHandler(mAppCallbackNotifier.get())
5.mDisplayAdapter->setPreviewWindow(window);
做完了上面这些步骤之后，就是startpreview了

/**
@brief Start preview mode.
@param none
@return NO_ERROR Camera switched to VF mode
@todo Update function header with the different errors that are possible
*/
status_t CameraHal::startPreview(){
LOG_FUNCTION_NAME;
// When tunneling is enabled during VTC, startPreview happens in 2 steps:
// When the application sends the command CAMERA_CMD_PREVIEW_INITIALIZATION,
// cameraPreviewInitialization()is called, which in turn causes the CameraAdapter
//to move from loaded to idle state.And when the application calls startPreview,
// the CameraAdapter moves from idle to executing state.
//
//If the application calls startPreview() without sending the command
// CAMERA_CMD_PREVIEW_INITIALIZATION,then the function cameraPreviewInitialization()
//AND startPreview() are executed.In other words,if the application calls
// startPreview() without sending the command CAMERA_CMD_PREVIEW_INITIALIZATION,
//then the CameraAdapter moves from loaded to idle to executing state in one shot.
status_t ret = cameraPreviewInitialization();
// The flag mPreviewInitializationDone issettotrue at the end of the function
// cameraPreviewInitialization(). Therefore,if everything goes alright,then the
// flag will be set. Sometimes, the function cameraPreviewInitialization() may
// return prematurely if all the resources are not available for starting preview.
//For example,if the preview windowisnotset,then it would return NO_ERROR.
// Under such circumstances, one should return from startPreview as well and should
//not continue execution. That is why, we check the flag andnot the return value.
if(!mPreviewInitializationDone) return ret;
// Once startPreview is called, there is no need to continue to remember whether
// the function cameraPreviewInitialization() was called earlier ornot.And so
// the flag mPreviewInitializationDone is reset here. Plus, this preserves the
// current behavior of startPreview under the circumstances where the application
// calls startPreview twice or more.
mPreviewInitializationDone =false;
///Enable the display adapter if present, actual overlay enable happens when we post the buffer
if(mDisplayAdapter.get()!=NULL){
CAMHAL_LOGDA("Enabling display");
int width, height;
mParameters.getPreviewSize(&width,&height);
#if PPM_INSTRUMENTATION || PPM_INSTRUMENTATION_ABS
ret = mDisplayAdapter->enableDisplay(width, height,&mStartPreview);
#else
ret = mDisplayAdapter->enableDisplay(width, height,NULL);
#endif
if( ret != NO_ERROR ){
CAMHAL_LOGEA("Couldn't enable display");
// FIXME: At this stage mStateSwitchLock is locked and unlock is supposed to be called
// only from mCameraAdapter->sendCommand(CameraAdapter::CAMERA_START_PREVIEW)
// below. But this will never happen because of goto error. Thus at next
// startPreview()call CameraHAL will be deadlocked.
// Need to revisit mStateSwitch lock,fornow just abort the process.
CAMHAL_ASSERT_X(false,
"At this stage mCameraAdapter->mStateSwitchLock is still locked, "
"deadlock is guaranteed");
goto error;
}
}
///Send START_PREVIEW command to adapter
CAMHAL_LOGDA("Starting CameraAdapter preview mode");
ret = mCameraAdapter->sendCommand(CameraAdapter::CAMERA_START_PREVIEW);
if(ret!=NO_ERROR){
CAMHAL_LOGEA("Couldn't start preview w/ CameraAdapter");
goto error;
}
CAMHAL_LOGDA("Started preview");
mPreviewEnabled =true;
mPreviewStartInProgress =false;
return ret;
error:
CAMHAL_LOGEA("Performing cleanup after error");
//Do all the cleanup
freePreviewBufs();
mCameraAdapter->sendCommand(CameraAdapter::CAMERA_STOP_PREVIEW);
if(mDisplayAdapter.get()!=NULL){
mDisplayAdapter->disableDisplay(false);
}
mAppCallbackNotifier->stop();
mPreviewStartInProgress =false;
mPreviewEnabled =false;
LOG_FUNCTION_NAME_EXIT;
return ret;
}

上面标出的cameraPreviewInitialization()方法也十分关键，之前已经说过，之后如果需要会再做说明
Enable the display adapter if present, actual overlay enable happens when we post the buffer
说明如果display adapter不是null，这里会enable，overlay方式就启动了
我们接着往下看，看看driver获取的数据到底是怎样处理的，startpreview会通过camerahal-->cameraapapter-->V4Lcameradapter
调用到v4l2层的startpreview，下面看看他的具体是实现

status_t V4LCameraAdapter::startPreview()
{
status_t ret = NO_ERROR;
LOG_FUNCTION_NAME;
Mutex::Autolock lock(mPreviewBufsLock);
if(mPreviewing){
ret = BAD_VALUE;
goto EXIT;
}
for(int i = 0; i mPreviewBufferCountQueueable; i++){
mVideoInfo->buf.index = i;
mVideoInfo->buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
mVideoInfo->buf.memory = V4L2_MEMORY_MMAP;
ret = v4lIoctl(mCameraHandle, VIDIOC_QBUF,&mVideoInfo->buf);//请求分配内存
if(ret 0){
CAMHAL_LOGEA("VIDIOC_QBUF Failed");
goto EXIT;
}
nQueued++;
}
ret = v4lStartStreaming();
// Create and start preview thread for receiving buffers from V4L Camera
if(!mCapturing){
mPreviewThread = new PreviewThread(this);//开启PreviewThread
CAMHAL_LOGDA("Created preview thread");
}
//Update the flag to indicate we are previewing
mPreviewing =true;
mCapturing =false;
EXIT:
LOG_FUNCTION_NAME_EXIT;
return ret;
}

int V4LCameraAdapter::previewThread()
{
status_t ret = NO_ERROR;
int width, height;
CameraFrame frame;
void *y_uv[2];
int index = 0;
int stride = 4096;
char *fp =NULL;
mParams.getPreviewSize(&width,&height);
if(mPreviewing){
fp = this->GetFrame(index);
if(!fp){
ret = BAD_VALUE;
goto EXIT;
}
CameraBuffer *buffer = mPreviewBufs.keyAt(index);//获取camerabuffer
CameraFrame *lframe =(CameraFrame *)mFrameQueue.valueFor(buffer);//获取cameraframe
if(!lframe){
ret = BAD_VALUE;
goto EXIT;
}
debugShowFPS();
if( mFrameSubscribers.size()== 0 ){
ret = BAD_VALUE;
goto EXIT;
}
y_uv[0]=(void*) lframe->mYuv[0];
//y_uv[1]=(void*) lframe->mYuv[1];
//y_uv[1]=(void*)(lframe->mYuv[0]+ height*stride);
convertYUV422ToNV12Tiler ((unsigned char*)fp,(unsigned char*)y_uv[0], width, height);//convert the data
CAMHAL_LOGVB("##...index= %d.;camera buffer= 0x%x; y= 0x%x; UV= 0x%x.",index, buffer, y_uv[0], y_uv[1]);
#ifdef SAVE_RAW_FRAMES
unsigned char* nv12_buff =(unsigned char*) malloc(width*height*3/2);
//Convert yuv422i to yuv420sp(NV12)& dump the frame to a file
convertYUV422ToNV12 ((unsigned char*)fp, nv12_buff, width, height);
saveFile( nv12_buff,((width*height)*3/2));//if you want to save the data,save it
free (nv12_buff);
#endif
//填充frame结构，用于数据处理
frame.mFrameType = CameraFrame::PREVIEW_FRAME_SYNC;
frame.mBuffer = buffer;
frame.mLength = width*height*3/2;
frame.mAlignment = stride;
frame.mOffset = 0;
frame.mTimestamp = systemTime(SYSTEM_TIME_MONOTONIC);
frame.mFrameMask =(unsigned int)CameraFrame::PREVIEW_FRAME_SYNC;
if(mRecording)
{
frame.mFrameMask |=(unsigned int)CameraFrame::VIDEO_FRAME_SYNC;
mFramesWithEncoder++;
}
//这里是重点，数据回调，或者使用overlay方式显示这里是决定性调用
ret = setInitFrameRefCount(frame.mBuffer, frame.mFrameMask);
if(ret != NO_ERROR){
CAMHAL_LOGDB("Error in setInitFrameRefCount %d", ret);
}else{
ret = sendFrameToSubscribers(&frame);
}
}
EXIT:
return ret;
}

现在就开始看看setInitFrameCount方法都做了些什么

int BaseCameraAdapter::setInitFrameRefCount(CameraBuffer * buf, unsigned int mask)
{
int ret = NO_ERROR;
unsigned int lmask;
LOG_FUNCTION_NAME;
if(buf ==NULL)
{
return -EINVAL;
}
for( lmask = 1; lmask CameraFrame::ALL_FRAMES; lmask = 1){
if( lmask & mask ){
switch( lmask ){
case CameraFrame::IMAGE_FRAME:
{
setFrameRefCount(buf, CameraFrame::IMAGE_FRAME,(int) mImageSubscribers.size());
}
break;
case CameraFrame::RAW_FRAME:
{
setFrameRefCount(buf, CameraFrame::RAW_FRAME, mRawSubscribers.size());
}
break;
case CameraFrame::PREVIEW_FRAME_SYNC:
{
setFrameRefCount(buf, CameraFrame::PREVIEW_FRAME_SYNC, mFrameSubscribers.size());//这里这个mFrameSubscribers对应的key上保存着响应的callback方法
}
break;
case CameraFrame::SNAPSHOT_FRAME:
{
setFrameRefCount(buf, CameraFrame::SNAPSHOT_FRAME, mSnapshotSubscribers.size());
}
break;
case CameraFrame::VIDEO_FRAME_SYNC:
{
setFrameRefCount(buf,CameraFrame::VIDEO_FRAME_SYNC, mVideoSubscribers.size());
}
break;
case CameraFrame::FRAME_DATA_SYNC:
{
setFrameRefCount(buf, CameraFrame::FRAME_DATA_SYNC, mFrameDataSubscribers.size());
}
break;
case CameraFrame::REPROCESS_INPUT_FRAME:
{
setFrameRefCount(buf,CameraFrame::REPROCESS_INPUT_FRAME, mVideoInSubscribers.size());
}
break;
default:
CAMHAL_LOGEB("FRAMETYPE NOT SUPPORTED 0x%x", lmask);
break;
}//SWITCH
mask &=~lmask;
}//IF
}//FOR
LOG_FUNCTION_NAME_EXIT;
return ret;
}

上面我标注的部分通过enableMsgType方法实现mFrameSubscribers.add的，经callback添加到对应的key处，算是实现关联，
同样的通过disableMsgType方法实现mFrameSubscribers.removeItem的，具体在哪里调用enableMsgType和disableMsgType之后再给予说明

void BaseCameraAdapter::setFrameRefCount(CameraBuffer * frameBuf, CameraFrame::FrameType frameType,int refCount)
{
LOG_FUNCTION_NAME;
switch ( frameType )
{
case CameraFrame::IMAGE_FRAME:
case CameraFrame::RAW_FRAME:
{
Mutex::Autolock lock(mCaptureBufferLock);
mCaptureBuffersAvailable.replaceValueFor(frameBuf, refCount);
}
break;
case CameraFrame::SNAPSHOT_FRAME:
{
Mutex::Autolock lock(mSnapshotBufferLock);
mSnapshotBuffersAvailable.replaceValueFor(( unsigned int) frameBuf, refCount);
}
break;
case CameraFrame::PREVIEW_FRAME_SYNC:
{
Mutex::Autolock lock(mPreviewBufferLock)
mPreviewBuffersAvailable.replaceValueFor(frameBuf, refCount);//这里我的理解是refCount和frameBuf实现了绑定，即camerabuf保存在mPreviewBuffersAvailable对应的key处
}
break;
case CameraFrame::FRAME_DATA_SYNC:
{
Mutex::Autolock lock(mPreviewDataBufferLock);
mPreviewDataBuffersAvailable.replaceValueFor(frameBuf, refCount);
}
break;
case CameraFrame::VIDEO_FRAME_SYNC:
{
Mutex::Autolock lock(mVideoBufferLock);
mVideoBuffersAvailable.replaceValueFor(frameBuf, refCount);
}
break;
case CameraFrame::REPROCESS_INPUT_FRAME:{
Mutex::Autolock lock(mVideoInBufferLock);
mVideoInBuffersAvailable.replaceValueFor(frameBuf, refCount);
}
break;
default:
break;
};
LOG_FUNCTION_NAME_EXIT;
}

接下我们看看sendFrameToSubscribers方法的具体实现过程

status_t BaseCameraAdapter::sendFrameToSubscribers(CameraFrame *frame)
{
status_t ret = NO_ERROR;
unsigned int mask;
if(NULL== frame )
{
CAMHAL_LOGEA("Invalid CameraFrame");
return -EINVAL;
}
for( mask = 1; mask CameraFrame::ALL_FRAMES; mask = 1){
if( mask & frame->mFrameMask ){
switch( mask ){
case CameraFrame::IMAGE_FRAME:
{
#if PPM_INSTRUMENTATION || PPM_INSTRUMENTATION_ABS
CameraHal::PPM("Shot to Jpeg: ",&mStartCapture);
#endif
ret = __sendFrameToSubscribers(frame,&mImageSubscribers, CameraFrame::IMAGE_FRAME);
}
break;
case CameraFrame::RAW_FRAME:
{
ret = __sendFrameToSubscribers(frame,&mRawSubscribers, CameraFrame::RAW_FRAME);
}
break;
case CameraFrame::PREVIEW_FRAME_SYNC:
{
ret = __sendFrameToSubscribers(frame,&mFrameSubscribers, CameraFrame::PREVIEW_FRAME_SYNC);
}
break;
case CameraFrame::SNAPSHOT_FRAME:
{
ret = __sendFrameToSubscribers(frame,&mSnapshotSubscribers, CameraFrame::SNAPSHOT_FRAME);
}
break;
case CameraFrame::VIDEO_FRAME_SYNC:
{
ret = __sendFrameToSubscribers(frame,&mVideoSubscribers, CameraFrame::VIDEO_FRAME_SYNC);
}
break;
case CameraFrame::FRAME_DATA_SYNC:
{
ret = __sendFrameToSubscribers(frame,&mFrameDataSubscribers, CameraFrame::FRAME_DATA_SYNC);
}
break;
case CameraFrame::REPROCESS_INPUT_FRAME:
{
ret = __sendFrameToSubscribers(frame,&mVideoInSubscribers, CameraFrame::REPROCESS_INPUT_FRAME);
}
break;
default:
CAMHAL_LOGEB("FRAMETYPE NOT SUPPORTED 0x%x", mask);
break;
}//SWITCH
frame->mFrameMask &=~mask;
if(ret != NO_ERROR){
goto EXIT;
}
}//IF
}//FOR
EXIT:
return ret;
}

status_t BaseCameraAdapter::__sendFrameToSubscribers(CameraFrame* frame,
KeyedVectorint, frame_callback>*subscribers,
CameraFrame::FrameType frameType)
{
size_t refCount = 0;
status_t ret = NO_ERROR;
frame_callback callback =NULL;
frame->mFrameType = frameType;
if((frameType == CameraFrame::PREVIEW_FRAME_SYNC)||
(frameType == CameraFrame::VIDEO_FRAME_SYNC)||
(frameType == CameraFrame::SNAPSHOT_FRAME)){
if(mFrameQueue.size()> 0){
CameraFrame *lframe =(CameraFrame *)mFrameQueue.valueFor(frame->mBuffer);
frame->mYuv[0]= lframe->mYuv[0];
frame->mYuv[1]= frame->mYuv[0]+(frame->mLength + frame->mOffset)*2/3;
}
else{
CAMHAL_LOGDA("Empty Frame Queue");
return -EINVAL;
}
}
if(NULL!= subscribers){
refCount = getFrameRefCount(frame->mBuffer, frameType);//通过这个refCount可以找到对应的callback方法
if(refCount == 0){
CAMHAL_LOGDA("Invalid ref count of 0");
return -EINVAL;
}
if(refCount > subscribers->size()){
CAMHAL_LOGEB("Invalid ref count for frame type: 0x%x", frameType);
return -EINVAL;
}
CAMHAL_LOGVB("Type of Frame: 0x%x address: 0x%x refCount start %d",
frame->mFrameType,
( uint32_t ) frame->mBuffer,
refCount);
for( unsigned int i = 0 ; i refCount; i++){
frame->mCookie =( void *) subscribers->keyAt(i);
callback =(frame_callback) subscribers->valueAt(i);
if(!callback){
CAMHAL_LOGEB("callback not set for frame type: 0x%x", frameType);
return -EINVAL;
}
callback(frame);
}
}else{
CAMHAL_LOGEA("Subscribers is null??");
return -EINVAL;
}
return ret;
}

这里别的我们先暂且不分析，但是callback到底是从哪里来的，这个我们必须说清楚
上面在实例化displayadapter时有这样一步：3.mDisplayAdapter->setFrameProvider(mCameraAdapter)//这一步是关键，之后会遇到的
我们看看setFrameProvider这个方法的实现：

int ANativeWindowDisplayAdapter::setFrameProvider(FrameNotifier *frameProvider)
{
LOG_FUNCTION_NAME;
// Check forNULL pointer
if(!frameProvider ){
CAMHAL_LOGEA("NULL passed for frame provider");
LOG_FUNCTION_NAME_EXIT;
return BAD_VALUE;
}
//Release any previous frame providers
if(NULL!= mFrameProvider ){
delete mFrameProvider;
}
/** Dont do anything here, Just save the pointer for use when display is
actually enabled or disabled
*/
mFrameProvider = new FrameProvider(frameProvider, this, frameCallbackRelay);//实例化一个FrameProvider，这其中有一个参数非常重要：frameCallbackRelay，他的定义在下面给出
LOG_FUNCTION_NAME_EXIT;
return NO_ERROR;
}

void ANativeWindowDisplayAdapter::frameCallbackRelay(CameraFrame* caFrame)
{
if(NULL!= caFrame )
{
if(NULL!= caFrame->mCookie )
{
ANativeWindowDisplayAdapter *da =(ANativeWindowDisplayAdapter*) caFrame->mCookie;
da->frameCallback(caFrame);
}
else
{
CAMHAL_LOGEB("Invalid Cookie in Camera Frame = %p, Cookie = %p", caFrame, caFrame->mCookie);
}
}
else
{
CAMHAL_LOGEB("Invalid Camera Frame = %p", caFrame);
}
}

void ANativeWindowDisplayAdapter::frameCallback(CameraFrame* caFrame)
{
///Call queueBuffer of overlay in the context of the callback thread
DisplayFrame df;
df.mBuffer = caFrame->mBuffer;
df.mType =(CameraFrame::FrameType) caFrame->mFrameType;
df.mOffset = caFrame->mOffset;
df.mWidthStride = caFrame->mAlignment;
df.mLength = caFrame->mLength;
df.mWidth = caFrame->mWidth;
df.mHeight = caFrame->mHeight;
PostFrame(df);
}

这个回调函数在这里设置，等待数据回调，我们很有必要去看看FrameProvider这个类的构造函数，他是怎样让其他方法调用到这个回调函数的呢

FrameProvider(FrameNotifier *fn, void* cookie, frame_callback frameCallback)
:mFrameNotifier(fn), mCookie(cookie),mFrameCallback(frameCallback){}

这个构造函数还是很有意思，没有任何实现，只是通过传入的三个参数实例化了三个对象而已
1.mFrameNotifier(fn), //这里mFrameNotifier就是camerasdapter
2.mCookie(cookie),
3.mFrameCallback(frameCallback)//mFrameCallback指向我们定义好的callback方法
我们接着就需要到之前已经提到过的startPreview方法中cameraPreviewInitialization的方法中去看了

/**
@brief Set preview mode related initialization
-> Camera Adapter set params
-> Allocate buffers
->Set use buffers for preview
@param none
@return NO_ERROR
@todo Update function header with the different errors that are possible
*/
status_t CameraHal::cameraPreviewInitialization()
{
status_t ret = NO_ERROR;
CameraAdapter::BuffersDescriptor desc;
CameraFrame frame;
unsigned int required_buffer_count;
unsigned int max_queueble_buffers;
#if PPM_INSTRUMENTATION || PPM_INSTRUMENTATION_ABS
gettimeofday(&mStartPreview,NULL);
#endif
LOG_FUNCTION_NAME;
if(mPreviewInitializationDone){
return NO_ERROR;
}
if( mPreviewEnabled ){
CAMHAL_LOGDA("Preview already running");
LOG_FUNCTION_NAME_EXIT;
return ALREADY_EXISTS;
}
if(NULL!= mCameraAdapter ){
ret = mCameraAdapter->setParameters(mParameters);
}
if((mPreviewStartInProgress ==false)&&(mDisplayPaused ==false)){
ret = mCameraAdapter->sendCommand(CameraAdapter::CAMERA_QUERY_RESOLUTION_PREVIEW,(int)&frame);
if( NO_ERROR != ret ){
CAMHAL_LOGEB("Error: CAMERA_QUERY_RESOLUTION_PREVIEW %d", ret);
return ret;
}
///Update the current preview width and height
mPreviewWidth = frame.mWidth;
mPreviewHeight = frame.mHeight;
}
///If we don't have the preview callback enabled and display adapter,
if(!mSetPreviewWindowCalled ||(mDisplayAdapter.get()==NULL)){
CAMHAL_LOGD("Preview not started. Preview in progress flag set");
mPreviewStartInProgress =true;
ret = mCameraAdapter->sendCommand(CameraAdapter::CAMERA_SWITCH_TO_EXECUTING);
if( NO_ERROR != ret ){
CAMHAL_LOGEB("Error: CAMERA_SWITCH_TO_EXECUTING %d", ret);
return ret;
}
return NO_ERROR;
}
if((mDisplayAdapter.get()!=NULL)&&(!mPreviewEnabled )&&( mDisplayPaused ))
{
CAMHAL_LOGDA("Preview is in paused state");
mDisplayPaused =false;
mPreviewEnabled =true;
if( NO_ERROR == ret )
{
ret = mDisplayAdapter->pauseDisplay(mDisplayPaused);
if( NO_ERROR != ret )
{
CAMHAL_LOGEB("Display adapter resume failed %x", ret);
}
}
//restart preview callbacks
if(mMsgEnabled & CAMERA_MSG_PREVIEW_FRAME)
{
mAppCallbackNotifier->enableMsgType (CAMERA_MSG_PREVIEW_FRAME);
}
signalEndImageCapture();
return ret;
}
required_buffer_count = atoi(mCameraProperties->get(CameraProperties::REQUIRED_PREVIEW_BUFS));
///Allocate the preview buffers
ret = allocPreviewBufs(mPreviewWidth, mPreviewHeight, mParameters.getPreviewFormat(), required_buffer_count, max_queueble_buffers);
if( NO_ERROR != ret )
{
CAMHAL_LOGEA("Couldn't allocate buffers for Preview");
goto error;
}
if( mMeasurementEnabled )
{
ret = mCameraAdapter->sendCommand(CameraAdapter::CAMERA_QUERY_BUFFER_SIZE_PREVIEW_DATA,
(int)&frame,
required_buffer_count);
if( NO_ERROR != ret )
{
return ret;
}
///Allocate the preview data buffers
ret = allocPreviewDataBufs(frame.mLength, required_buffer_count);
if( NO_ERROR != ret ){
CAMHAL_LOGEA("Couldn't allocate preview data buffers");
goto error;
}
if( NO_ERROR == ret )
{
desc.mBuffers = mPreviewDataBuffers;
desc.mOffsets = mPreviewDataOffsets;
desc.mFd = mPreviewDataFd;
desc.mLength = mPreviewDataLength;
desc.mCount =( size_t ) required_buffer_count;
desc.mMaxQueueable =(size_t) required_buffer_count;
mCameraAdapter->sendCommand(CameraAdapter::CAMERA_USE_BUFFERS_PREVIEW_DATA,
(int)&desc);
}
}
///Pass the buffers to Camera Adapter
desc.mBuffers = mPreviewBuffers;
desc.mOffsets = mPreviewOffsets;
desc.mFd = mPreviewFd;
desc.mLength = mPreviewLength;
desc.mCount =( size_t ) required_buffer_count;
desc.mMaxQueueable =(size_t) max_queueble_buffers;
ret = mCameraAdapter->sendCommand(CameraAdapter::CAMERA_USE_BUFFERS_PREVIEW,
(int)&desc);
if( NO_ERROR != ret )
{
CAMHAL_LOGEB("Failed to register preview buffers: 0x%x", ret);
freePreviewBufs();
return ret;
}
mAppCallbackNotifier->startPreviewCallbacks(mParameters, mPreviewBuffers, mPreviewOffsets, mPreviewFd, mPreviewLength, required_buffer_count);
///Start the callback notifier
ret = mAppCallbackNotifier->start();
if( ALREADY_EXISTS == ret )
{
//Already running,donothing
CAMHAL_LOGDA("AppCallbackNotifier already running");
ret = NO_ERROR;
}
elseif( NO_ERROR == ret ){
CAMHAL_LOGDA("Started AppCallbackNotifier..");
mAppCallbackNotifier->setMeasurements(mMeasurementEnabled);
}
else
{
CAMHAL_LOGDA("Couldn't start AppCallbackNotifier");
goto error;
}
if(ret == NO_ERROR) mPreviewInitializationDone =true;
return ret;
error:
CAMHAL_LOGEA("Performing cleanup after error");
//Do all the cleanup
freePreviewBufs();
mCameraAdapter->sendCommand(CameraAdapter::CAMERA_STOP_PREVIEW);
if(mDisplayAdapter.get()!=NULL)
{
mDisplayAdapter->disableDisplay(false);
}
mAppCallbackNotifier->stop();
mPreviewStartInProgress =false;
mPreviewEnabled =false;
LOG_FUNCTION_NAME_EXIT;
return ret;
}

我们就看看这个方法的是实现吧：mAppCallbackNotifier->enableMsgType (CAMERA_MSG_PREVIEW_FRAME);

status_t AppCallbackNotifier::enableMsgType(int32_t msgType)
{
if( msgType & CAMERA_MSG_PREVIEW_FRAME ){
mFrameProvider->enableFrameNotification(CameraFrame::PREVIEW_FRAME_SYNC);
}
if( msgType & CAMERA_MSG_POSTVIEW_FRAME ){
mFrameProvider->enableFrameNotification(CameraFrame::SNAPSHOT_FRAME);
}
if(msgType & CAMERA_MSG_RAW_IMAGE){
mFrameProvider->enableFrameNotification(CameraFrame::RAW_FRAME);
}
return NO_ERROR;
}

int FrameProvider::enableFrameNotification(int32_t frameTypes)
{
LOG_FUNCTION_NAME;
status_t ret = NO_ERROR;
///Enable the frame notification to CameraAdapter (which implements FrameNotifier interface)
mFrameNotifier->enableMsgType(frameTypesMessageNotifier::FRAME_BIT_FIELD_POSITION, mFrameCallback,NULL, mCookie);
LOG_FUNCTION_NAME_EXIT;
return ret;
}

这里这个enableMsgType其实就是前面已经提到过的那个enableMsgType方法，实现callback方法add到响应的key上
这里这个mFrameNotifier是FrameNotifier的对象，FrameNotifier这个类继承于MessageNotifier
而BaseCameraAdapter继承于CameraAdapter，CameraAdapter又继承于FrameNotifier，所以mFrameNotifier对象调用的enableMsgType方法其实是一个虚函数，
最终调用的是BaseCameraAdapter这个类中定义的enableMsgType方法，我们来看一看他的实现：

void BaseCameraAdapter::enableMsgType(int32_t msgs, frame_callback callback, event_callback eventCb, void* cookie)
{
Mutex::Autolock lock(mSubscriberLock);
LOG_FUNCTION_NAME;
int32_t frameMsg =((msgs >> MessageNotifier::FRAME_BIT_FIELD_POSITION)& EVENT_MASK);
int32_t eventMsg =((msgs >> MessageNotifier::EVENT_BIT_FIELD_POSITION)& EVENT_MASK);
if( frameMsg != 0 )
{
CAMHAL_LOGVB("Frame message type id=0x%x subscription request", frameMsg);
switch ( frameMsg )
{
case CameraFrame::PREVIEW_FRAME_SYNC:
mFrameSubscribers.add((int) cookie, callback);
break;
case CameraFrame::FRAME_DATA_SYNC:
mFrameDataSubscribers.add((int) cookie, callback);
break;
case CameraFrame::SNAPSHOT_FRAME:
mSnapshotSubscribers.add((int) cookie, callback);
break;
case CameraFrame::IMAGE_FRAME:
mImageSubscribers.add((int) cookie, callback);
break;
case CameraFrame::RAW_FRAME:
mRawSubscribers.add((int) cookie, callback);
break;
case CameraFrame::VIDEO_FRAME_SYNC:
mVideoSubscribers.add((int) cookie, callback);
break;
case CameraFrame::REPROCESS_INPUT_FRAME:
mVideoInSubscribers.add((int) cookie, callback);
break;
default:
CAMHAL_LOGEA("Frame message type id=0x%x subscription no supported yet!", frameMsg);
break;
}
}
if( eventMsg != 0)
{
CAMHAL_LOGVB("Event message type id=0x%x subscription request", eventMsg);
if( CameraHalEvent::ALL_EVENTS == eventMsg )
{
mFocusSubscribers.add((int) cookie, eventCb);
mShutterSubscribers.add((int) cookie, eventCb);
mZoomSubscribers.add((int) cookie, eventCb);
mMetadataSubscribers.add((int) cookie, eventCb);
}
else
{
CAMHAL_LOGEA("Event message type id=0x%x subscription no supported yet!", eventMsg);
}
}
LOG_FUNCTION_NAME_EXIT;
}

这里通过mFrameSubscribers.add((int) cookie, callback)这个方法将mFrameCallback回调函数与key相关联
所以上面可以通过callback = (frame_callback) subscribers->valueAt(i);
这个方法获取callback的实现，因为上面已经实现了关联，所以数据最终是通过上面分析道的方法继续进行数据流显示

void ANativeWindowDisplayAdapter::frameCallback(CameraFrame* caFrame)
{
///Call queueBuffer of overlay in the context of the callback thread
DisplayFrame df;
df.mBuffer = caFrame->mBuffer;
df.mType = (CameraFrame::FrameType) caFrame->mFrameType;
df.mOffset = caFrame->mOffset;
df.mWidthStride = caFrame->mAlignment;
df.mLength = caFrame->mLength;
df.mWidth = caFrame->mWidth;
df.mHeight = caFrame->mHeight;
PostFrame(df);//这里填充了DisplayFrame这个结构，并调用PostFrome实现显示
}

这里PostFrame成了我要研究的主要内容，将数据以DisplayFrame结构的方式打包之后到底是怎么实现显示的呢？？

status_t ANativeWindowDisplayAdapter::PostFrame(ANativeWindowDisplayAdapter::DisplayFrame &dispFrame)
{
status_t ret = NO_ERROR;
uint32_t actualFramesWithDisplay = 0;
android_native_buffer_t *buffer =NULL;
GraphicBufferMapper &mapper = GraphicBufferMapper::get();
int i;
///@todo Do cropping based on the stabilized frame coordinates
///@todo Insert logic to drop frames here based on refresh rate of
///display or rendering rate whichever is lower
///Queue the buffer to overlay
if(NULL== mANativeWindow ){
return NO_INIT;
}
if(!mBuffers ||!dispFrame.mBuffer){
CAMHAL_LOGEA("NULL sent to PostFrame");
return BAD_VALUE;
}
for( i = 0; i mBufferCount; i++)
{
if( dispFrame.mBuffer ==&mBuffers[i])
{
break;
}
}
mFramesType.add((int)mBuffers[i].opaque ,dispFrame.mType );
if( mDisplayState == ANativeWindowDisplayAdapter::DISPLAY_STARTED &&
(!mPaused || CameraFrame::CameraFrame::SNAPSHOT_FRAME == dispFrame.mType)&&
!mSuspend)
{
Mutex::Autolock lock(mLock);
uint32_t xOff =(dispFrame.mOffset% PAGE_SIZE);
uint32_t yOff =(dispFrame.mOffset / PAGE_SIZE);
//Set crop only if current x and y offsets donot match with frame offsets
if((mXOff!=xOff)||(mYOff!=yOff))
{
CAMHAL_LOGDB("Offset %d xOff = %d, yOff = %d", dispFrame.mOffset, xOff, yOff);
uint8_t bytesPerPixel;
///Calculate bytes per pixel based on the pixel format
if(strcmp(mPixelFormat,(const char *) CameraParameters::PIXEL_FORMAT_YUV422I)== 0)
{
bytesPerPixel = 2;
}
elseif(strcmp(mPixelFormat,(const char *) CameraParameters::PIXEL_FORMAT_RGB565)== 0)
{
bytesPerPixel = 2;
}
elseif(strcmp(mPixelFormat,(const char *) CameraParameters::PIXEL_FORMAT_YUV420SP)== 0)
{
bytesPerPixel = 1;
}
else
{
bytesPerPixel = 1;
}
CAMHAL_LOGVB(" crop.left = %d crop.top = %d crop.right = %d crop.bottom = %d",
xOff/bytesPerPixel, yOff ,(xOff/bytesPerPixel)+mPreviewWidth, yOff+mPreviewHeight);
// We'll ignore any errors here,if the surface is
// already invalid, we'll know soon enough.
mANativeWindow->set_crop(mANativeWindow, xOff/bytesPerPixel, yOff,
(xOff/bytesPerPixel)+mPreviewWidth, yOff+mPreviewHeight);
///Update the current x and y offsets
mXOff = xOff;
mYOff = yOff;
}
{
buffer_handle_t *handle =(buffer_handle_t *) mBuffers[i].opaque;
// unlock buffer before sending to display
mapper.unlock(*handle);
ret = mANativeWindow->enqueue_buffer(mANativeWindow, handle);
}
if( NO_ERROR != ret ){
CAMHAL_LOGE("Surface::queueBuffer returned error %d", ret);
}
mFramesWithCameraAdapterMap.removeItem((buffer_handle_t *) dispFrame.mBuffer->opaque);
// HWComposer has not minimum buffer requirement. We should be able to dequeue
// the buffer immediately
TIUTILS::Message msg;
mDisplayQ.put(&msg);
#if PPM_INSTRUMENTATION || PPM_INSTRUMENTATION_ABS
if( mMeasureStandby )
{
CameraHal::PPM("Standby to first shot: Sensor Change completed - ",&mStandbyToShot);
mMeasureStandby =false;
}
elseif(CameraFrame::CameraFrame::SNAPSHOT_FRAME == dispFrame.mType)
{
CameraHal::PPM("Shot to snapshot: ",&mStartCapture);
mShotToShot =true;
}
elseif( mShotToShot )
{
CameraHal::PPM("Shot to shot: ",&mStartCapture);
mShotToShot =false;
}
#endif
}
else
{
Mutex::Autolock lock(mLock);
buffer_handle_t *handle =(buffer_handle_t *) mBuffers[i].opaque;
// unlock buffer before giving it up
mapper.unlock(*handle);
// cancel buffer and dequeue another one
ret = mANativeWindow->cancel_buffer(mANativeWindow, handle);
if( NO_ERROR != ret ){
CAMHAL_LOGE("Surface::cancelBuffer returned error %d", ret);
}
mFramesWithCameraAdapterMap.removeItem((buffer_handle_t *) dispFrame.mBuffer->opaque);
TIUTILS::Message msg;
mDisplayQ.put(&msg);
ret = NO_ERROR;
}
return ret;
}

qianxuedegushi

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
Android Camera数据流分析全程记录（overlay方式）

Android Camera数据流分析全程记录（overlay方式）备注：转自 - conceptcon-ChinaUnix博客这里为什么要研究overlay方式呢？android camera需要driver和app层需要有大量数据需要传输，如果使用非overlay方式进行数据从driver到app层的传输，使系统性能受到很到影响，使系统速度变慢，同时会影响功耗等，而在camera ...
复制链接

扫一扫

专栏目录