Camera: Focus 影响因素与状态管理及下发（一）

        手机Camera的Auto-Focue(AF)功能是由俩部分构成的，一种是ACTIVE的状态改变，也就是人为的控制UI，触发聚焦功能，点击改变相对应区域的聚焦状态，下发处理，即上文所做UI操作；还有一种是PASSIVE的状态改变，这一部分是底层自己触发的Auto-Focus聚焦功能，也是本文研究的状态变化流程。

1.Auto-Focus(AF)模式需要处理的状态类型：

        AF status由8种状态的变化来反应聚焦功能所包括的所有变化场景，下面会就我自己的理解将8种状态代表的意义进行一下说明，在底层Auto-Focus算法处理阶段有对应不同情况更为详细的处理说明。

PASSIVE_SCAN(1): 初始一个新的scan view；开始AF scan; Lens将会移动

PASSIVE_FOCUSED(2): AF scan结束；Lens将会锁定

PASSIVE_UNFOCUSED(6): AF scan结束; Lens依然会锁定; 但是scan fail

FOCUSED_LOCKED(4): 将会立即改变相关状态;  Lens目前是锁定状态

NOT_FOCUSED_LOCKED(5): 将会立即改变相关状态；Lens目前是锁定状态

还有三种由UI触发的ACTIVE_SCAN(3)、ACTIVE_FOCUSED(0)、ACTIVE_UNFOCUSED(0)主动聚焦的调整状态

详细的算法对各个状态变化的说明可参考NdkCameraMetadataTags.h文件，一些说明如我一下图片所示：

 

 

2.Auto-Focus(AF)状态的变化流程

        1.在预览数据流没有拿到相应的buffer之前，AF status为PASSIVE_UNFOCUSED，此时获取到的AF Mode为CONTROL_AF_MODE_CONTINUOUS_PICTURE，此时会下发CONTROL_AF_REGIONS、CONTROL_AF_MODE、CONTROL_AF_TRIGGER。

        需要注意的是AF的状态不是应用层去维护的，改变机制是HAL层进行控制的，而应用层只能获取，但是应用层有可以通过其他的一些Tag的下发去驱使HAL层将AF Status改为相对应的状态。这一部分会在后面的下发管理中进行进一步的研究。

 下图为没有拿到预览帧时CaptureRequest阶段相关的Tag下发。即使拿到了预览帧数据，若没有UI触发Focus相关状态改变的情况发生时，AF Status、AF Mode、AF Regions也是不会发生改变的。

private void addBaselineCaptureKeysToRequest(CaptureRequest.Builder builder) {
        builder.set(CaptureRequest.CONTROL_AF_REGIONS, mAFRegions);
        builder.set(CaptureRequest.CONTROL_AE_REGIONS, mAERegions);
        builder.set(CaptureRequest.CONTROL_AF_MODE, mCurrentFocusMode);
        builder.set(CaptureRequest.CONTROL_AF_TRIGGER, CameraMetadata.CONTROL_AF_TRIGGER_IDLE);
        if (mKeyFlashCalibrationRequest != null) {
            builder.set(mKeyFlashCalibrationRequest, FLASH_CALIBRATION_ON);
        }
        LogHelper.d(TAG, "[configCaptureRequest] mAERegions[0]:" + mAERegions[0] + ", mAFRegions[0]:" + mAFRegions[0] + ", mCurrentFocusMode:" + mCurrentFocusMode);
    }

        2.拿到预览帧数据之后，应用层先更新FouceView相关的状态为可重绘的状态，然后根据预览Session回调上来的onCaptureCompleted方法获取此时Auto-Focus(AF)的状态并更新应用层用于接收的AF status。

case ICameraMode.MODE_DEVICE_STATE_PREVIEWING:
                mNeedPlayFocusSound = true;
                mNeedShowFocusUi = true;
                //将Focus设置为可重绘的状态
                mFocusListener.disableUpdateFocusState(false);
                mFocusStateUpdateDisabled = false;
                
                if (!isLockActive() && !isContinuousFocusMode()) {
                    mFocusListener.restoreContinue();
                    if (mSettingChangeRequester != null) {
                        //此时下发的AF Mode、AF Regions状态和预览帧没上来时候的下发是一样的
                        mSettingChangeRequester.sendSettingChangeRequest();
                    }
                }
                break;

@Override
        public void onCaptureCompleted(CameraCaptureSession session, CaptureRequest request,
                                       TotalCaptureResult result) {
            super.onCaptureCompleted(session, request, result);
            updateAeState(request, result);

            autofocusStateChangeDispatcher(result,false);

            synchronized (mFocusQueue) {
                    mFocusQueue.clear();
                    cancelAutoFocus();
                    mFocus.resetTouchFocusWhenCaptureDone();
                    if (mKeyFlashCalibrationResult != null) {
                        showFlashCalibrationResult(result);
                    }
                }

            }
        }

    private void autofocusStateChangeDispatcher(CaptureResult result,boolean isTimeout) {
        long currentFrameNumber = result.getFrameNumber();

        mLastControlAfStateFrameNumber = result.getFrameNumber();
        //得到了此刻预览session拿到的AF_STATE的状态
        int resultAFState = result.get(CaptureResult.CONTROL_AF_STATE);
        LogHelper.w(TAG, "onCaptureCompleted [autofocusStateChangeDispatcher] resultAFState:" + resultAFState);
        notifyFocusStateChanged(resultAFState, mLastControlAfStateFrameNumber);

    }


    private void notifyFocusStateChanged(int afState, long frameNumber) {

        if (mNeedWaitActiveScanDone) {
            String flashValue = mFocus.getCurrentFlashValue();
            if ((!FLASH_OFF_VALUE.equals(flashValue)) &&
                    (afState == CaptureResult.CONTROL_AF_STATE_NOT_FOCUSED_LOCKED)) {
                LogHelper.d(TAG, "[notifyFocusStateChanged]  need trigger AF again");
                if (!mFocusQueue.isEmpty()&&AUTOFOCUS.equals(mFocusQueue.peek())){
                    mFocusQueue.clear();
                }
                //AF state并不为CONTROL_AF_STATE_NOT_FOCUSED_LOCKED
                doAfTriggerBeforeCapture();
                mNeedWaitActiveScanDone = false;
                return;
            } else if (afState == CaptureResult.CONTROL_AF_STATE_NOT_FOCUSED_LOCKED ||
                    afState == CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED) {
                mNeedWaitActiveScanDone = false;
            }
        }

        if (isActiveFocusDone(afState)) {
            if(mNeedWaitTafDone){
                mNeedWaitTafDone = false;
            }
            synchronized (mFocusQueue) {
                if (!mFocusQueue.isEmpty() && AUTOFOCUS.equals(mFocusQueue.peek())) {
                    LogHelper.d(TAG, "[notifyFocusStateChanged] mFocusQueue " + mFocusQueue.size() +
                            " before add cancelAutoFocus");
                    mFocusQueue.clear();
                    mFocusQueue.add(CANCEL_AUTOFOCUS);
                }
            }
            synchronized (mLock) {
                if (mFocusStateListener != null && frameNumber >= mLastAfTriggerFrameNumber) {
                    if (mAeTriggerStarted){
                        if (mAeState == CaptureResult.CONTROL_AE_STATE_CONVERGED
                                ||mAeState == CaptureResult.CONTROL_AE_STATE_FLASH_REQUIRED){
                            if (mLastResultAFState != afState){
                                //通过应用层的FocueState监听事件更新相关状态
                                mFocusStateListener.onFocusStatusUpdate(state, frameNumber);
                                mAeTriggerStarted = false;
                    }
                }
            }
        }

        //Focus为PASSIVE的活动状态时通过这边更新相关的应用层state
        synchronized (mLock) {
            if (mFocusStateListener != null && mLastResultAFState != afState) {

                //通过应用层的FocueState监听事件更新相关状态
                mFocusStateListener.onFocusStatusUpdate(state, frameNumber);
                LogHelper.d(TAG, "[notifyFocusStateChanged] mLastResultAFState " +
                        mLastResultAFState + ",afState " + afState);
                mLastResultAFState = afState;
            }
        }
    }

        3.应用层通过onFocusStateUpdate的函数去接受底层拿到的Focus State，并且进行相关的一系列的处理。

    private void onFocusStateUpdate(AutoFocusState state) {
        mFocusStateStatusResponder.statusChanged(FOCUS_STATE_KEY, state.toString());

        if (mFocusViewController == null) {
            LogHelper.w(mTag, "[onFocusStateUpdate] mFocusViewController is null");
            return;
        }
        if (!mIsAutoFocusTriggered && mIsEvChanging) {
            LogHelper.w(mTag, "[onFocusStateUpdate] mIsEvChanging when not touch");
            return;
        }
        LogHelper.d(mTag, "[onFocusStateUpdate] AutoFocusState state:" + state.toString());
        switch (state) {
            case PASSIVE_SCAN:
                //Focus Active的活动状态具有最高优先级
                if (isLockActive() || mFocusViewController.isActiveFocusRunning()) {
                    return;
                }

                //这边是通过是否拿到预览帧数据来控制
                if (mFocusStateUpdateDisabled) {
                    return;
                }

                mIsEvChanging = false;
                mModeHandler.post(new Runnable() {
                    @Override
                    public void run() {
                        if (mFocusArea != null || mMeteringArea != null) {
                            resetFocusArea();
                            if (mSettingChangeRequester != null) {
                                //此时因为Focus是PASSIVE的活动状态，所以下发的Tag应该没有差别
                                mSettingChangeRequester.sendSettingChangeRequest();
                            }
                        }
                    }
                });
                if (mNeedShowFocusUi) {
                    mFocusViewController.clearFocusUi();
                    //这边涉及到一些其他的feature对于Focus的影响，比如相机检测到人脸时，Focus会停止工作
                    if (!mIsFaceExist && !isRestrictedToAutoOnly()) {
                        mFocusViewController.showPassiveFocusAtCenter();
                    }
                }
                break;
            case ACTIVE_SCAN:
                // Unused, manual scans are triggered via the UI
                break;
            case PASSIVE_FOCUSED:
            case PASSIVE_UNFOCUSED:
                if (isLockActive() || mFocusViewController.isActiveFocusRunning()) {
                    LogHelper.w(mTag, "[onFocusStateUpdate] ignore the state " + state);
                    return;
                }
                mFocusViewController.stopFocusAnimations();
                break;
            case ACTIVE_FOCUSED:
            case ACTIVE_UNFOCUSED:
                mIsAutoFocusTriggered = false;
                mModeHandler.post(new Runnable() {
                    @Override
                    public void run() {
                        if (mNeedDoAfLock) {
                            mNeedDoAfLock = false;
                            //这边下发的Tag以及相对应的values是不同的，会有一些改变
                            mSettingChangeRequester.sendSettingChangeRequest();
                        }

                    }
                });

                if (mNeedTriggerShutterButton) {
                    if (!isLockActive()) {
                        mFocusViewController.clearFocusUi();
                    } else {
                        mFocusViewController.stopFocusAnimations();
                    }
                    mAppUi.triggerShutterButtonClick(FOCUS_SHUTTER_PRIORITY);
                    mAppUi.setCameraFocusing(false);
                } else {
                    if (mNeedPlayFocusSound) {
                        mCameraContext.getSoundPlayback().play(ISoundPlayback
                                .FOCUS_COMPLETE);
                    }
                    //the focus indicator should be dark 2s later
                    mFocusViewController.stopFocusAnimations();
                    //change to continue focus 3s later ,note that focus indicator
                    // position will not change when continue focus move callback has
                    // been received.
                    //no need to clear focus UI when AE/AF lock done
                    if (LockState.STATE_LOCKING.equals(mLockState)) {
                        mLockState = LockState.STATE_LOCKED;
                    } else {
                        resetTouchFocus();
                    }
                }
                break;
            default:
                break;
        }
        LogHelper.d(mTag, "[onFocusStateUpdate]-");
    }

3.其他模块对Auto-Focus(AF) feature的影响

        对于应用层来说，可以通过添加一系列的转态监听函数去check对Focus有影响的Feature的改变，进而做出一系列的改变。下图share了5种Feature对Focus的影响处理：

mFocusStateStatusResponder = mStatusMonitor.getStatusResponder(FOCUS_STATE_KEY);
        mStatusMonitor.registerValueChangedListener(FOCUS_MODE_KEY, mFocusStatusChangeListener);
        mStatusMonitor.registerValueChangedListener(FACE_EXIST_KEY, mFocusStatusChangeListener);
        mStatusMonitor.registerValueChangedListener(KEY_CSHOT, mFocusStatusChangeListener);
        mStatusMonitor.registerValueChangedListener(HIGH_SPEED_KEY, mFocusStatusChangeListener);
        mStatusMonitor.registerValueChangedListener(EXPOSURE_VIEW_KEY, mFocusStatusChangeListener);

private StatusMonitor.StatusChangeListener mFocusStatusChangeListener = new StatusMonitor
            .StatusChangeListener() {

        @Override
        public void onStatusChanged(String key, String value) {
            LogHelper.d(mTag, "[onStatusChanged]+ key: " + key + "," +
                    "value: " + value + ",mLockState = " + mLockState);
            switch (key) {
                case EXPOSURE_VIEW_KEY:
                    boolean isEvChanging = Boolean.parseBoolean(value);
                    if (!LockState.STATE_LOCKING.equals(mLockState)) {
                        //这边会根据UI选择的曝光值来调整Focus的亮暗程度
                        onExposureViewStatusChanged(isEvChanging);
                    }
                    break;
                
                //FocusMode:OFF/AUTO/MACRO/CONTINOUS_VIDEO/CONTINOUS_PICTURE 4种，不同的mode会影响到focus state的状态
                case FOCUS_MODE_KEY:
                    mFocusViewController.clearFocusUi();
                    updateAfModeState();
                    mFocusListener.updateFocusCallback();
                    break;

                //检测到人脸后，会走人脸检测的算法流程，这个和聚焦的算法是有冲突的
                case FACE_EXIST_KEY:
                    mIsFaceExist = Boolean.parseBoolean(value);
                    break;

                //ContinousShot下的Focus是需要有特殊的处理
                case KEY_CSHOT:
                    if (VALUE_CSHOT_START.equals(value)) {
                        boolean isNeedAfTriggerDone = mFocusListener.needWaitAfTriggerDone();
                        if (mNeedResetTouchFocus) {
                            mModeHandler.removeMessages(RESET_TOUCH_FOCUS);
                        }
                        if (isNeedAfTriggerDone) {
                            mNeedDoCancelAutoFocus = true;
                            mNeedPlayFocusSound = false;
                            mModeHandler.post(new Runnable() {
                                @Override
                                public void run() {
                                    mFocusListener.setWaitCancelAutoFocus(true);
                                }
                            });
                        } else {
                            mFocusStateStatusResponder
                                    .statusChanged(FOCUS_STATE_KEY, "ACTIVE_FOCUSED");
                        }
                    } else if (VALUE_CSHOT_STOP.equals(value)) {
                        if (mNeedDoCancelAutoFocus) {
                            mModeHandler.post(new Runnable() {
                                @Override
                                public void run() {
                                    mFocusListener.setWaitCancelAutoFocus(false);
                                }
                            });
                            mNeedDoCancelAutoFocus = false;
                            mNeedPlayFocusSound = true;
                        }
                    }
                    break;

                //慢录模式下session 拍照capture下发的流程和普通模式是不一样的
                case HIGH_SPEED_KEY:
                    mIsHighSpeedRequest = Boolean.parseBoolean(value);
                    break;
                default:
                    break;
            }
        }
    };

//曝光状态下对聚焦的处理
private void onExposureViewStatusChanged(boolean isEvChanging) {
        if (!mPreviewStarted) {
            return;
        }
        if (mIsEvChanging != isEvChanging) {
            mIsEvChanging = isEvChanging;
            if (mIsEvChanging) {
                mFocusViewController.highlightFocusView();
            } else {
                mFocusViewController.lowlightFocusView();
            }
            if (isContinuousFocusMode()) {
                mModeHandler.post(new Runnable() {
                    @Override
                    public void run() {
                        if (mIsEvChanging) {
                            mFocusListener.autoFocus();
                        } else {
                            mFocusListener.cancelAutoFocus();
                        }
                    }
                });
            }

            if (mNeedResetTouchFocus) {
                if (mIsEvChanging) {
                    mModeHandler.removeMessages(RESET_TOUCH_FOCUS);
                } else {
                    mModeHandler.sendEmptyMessageDelayed(RESET_TOUCH_FOCUS,
                            sFocusHoldMills);
                }
            }
        }
    }

小结：

        用一篇去梳理了AF State的获取以及从应用层角度去研究了Focus与各个模块之间的互斥、融合的处理方式；接下来会继续梳理应用层是如何通过Tag下发与改变机制驱使HAL层完成相对应的AF State的转变以及相关算法的处理。