ndk实例总结：基于libuvc的双usbCamera处理

ndk实例总结

ndk实例总结：jni实例
ndk实例总结：opencv图像处理
ndk实例总结：安卓Camera与usbCamera原始图像处理
ndk实例总结补充：使用V4L2采集usb图像分析
ndk实例总结：使用fmpeg播放rtsp流
ndk实例总结：基于libuvc的双usbCamera处理
ndk实例总结补充：使用libuvc采集usb图像分析
ndk实例总结：jni日志存储

前言

在Android上进行usb camera相关的开发，离不开两种方式，一种基于v4l2接口，我在ndk实例总结补充：使用V4L2采集usb图像分析
中有过分析，另一种基于libuvc，github上有一个开源项目UVCCamera，但是封装的非常复杂，很难在其基础上进行定制开发，而且不适合用来学习

所以这个项目基于libuvc原本的api进行定制开发，实现了双摄像头的预览与拍照，以及更改摄像头参数等相关功能

项目当中用到了从UVCCamera项目里拆出来的一些包装类，比如USBMonitor、USBVendorId、DeviceFilter等，so库使用了UVCCamera已经编译好的so库，相比libuvc源码，它在Android上做了很多优化

项目构架

CMakeLists相关配置与ndk实例总结系列的其他项目类似，这里不再介绍了

流程分析

libuvc与v4l2最大的不同就是libuvc是基于usb设备的vid、pid、fd、busNum、devNum之类的信息来操控摄像头，而v4l2是通过/dev/video*的设备号来操控

因此大致流程就是：获取usbDevice相关信息，将usb信息传递到jni中使用libuvc api打开摄像头，以及在jni中获取摄像头原始图像并处理，最后绘制到SurfaceView上

初始化

@Override
public void onCreate(Bundle savedInstanceState) {
    mManager = UsbCameraManager.getInstance();
    mManager.init(this.getApplicationContext(), this);
    // mManager.setStreamMode(UsbCameraManager.ASYNC_CALLBACK);
    mManager.setStreamMode(UsbCameraManager.SYNC_POLLING);
}

@Override
protected void onStart() {
    super.onStart();
    // 注册BroadcastReceiver监视USB事件
    mManager.register();
}

public void init(Context context, final OnUsbConnectListener listener) {
    this.mContext = context;

    Arrays.fill(mDeviceArr, null);
    Arrays.fill(mCtrlBlockArr, null);
    Arrays.fill(mCamViewArr, null);
    Arrays.fill(mIsOpenArr, null);

    mUsbMonitor = new USBMonitor(context, new USBMonitor.OnDeviceConnectListener() {
        @Override
        public void onAttach(UsbDevice device) {
            Log.i(TAG, "onAttach: ");
            if (listener != null) {
                listener.onAttachDev(device);
            }
        }

        @Override
        public void onDetach(UsbDevice device) {
            Log.i(TAG, "onDetach: ");
            if (listener != null) {
                listener.onDetachDev(device);
            }
        }

        @Override
        public void onConnect(UsbDevice device, USBMonitor.UsbControlBlock ctrlBlock, boolean createNew) {
            Log.i(TAG, "onConnect: ");
            if (listener != null) {
                listener.onConnectDev(device, ctrlBlock, createNew);
            }
        }

        @Override
        public void onDisconnect(UsbDevice device, USBMonitor.UsbControlBlock ctrlBlock) {
            Log.i(TAG, "onDisconnect: ");
            if (listener != null) {
                listener.onDisconnectDev(device, ctrlBlock);
            }
        }

        @Override
        public void onCancel(UsbDevice device) {
            Log.i(TAG, "onCancel: ");
        }
    });
}

init方法中初始化USBMonitor类，并且实例化了几个回调状态，分别对应摄像头的插拔状态，在调用了mManager.register方法后系统广播就会回调相关的USB状态

@IntDef({ASYNC_CALLBACK, SYNC_POLLING})
@interface StreamMode {
}

public void setStreamMode(@StreamMode int streamMode) {
    this.streamMode = streamMode;
}

streamMode代表获取摄像头原始帧的两种方式，一种是异步回调方式（传递callback函数到libuvc开启流的函数中，每帧图像就会通过callback函数回调），另一种是主动轮寻方式（主动调用libuvc的获取帧函数，调用一次获取一帧）

根据项目自己选择使用哪种方式，比如需要预览就使用异步回调方式，只拍照不预览就可以选择主动轮寻方式

摄像头打开与关闭

摄像头打开与关闭的逻辑在USB回调事件中，在onAttach中获取权限，onConnect中打开摄像头，onDetach或onDisconnect中关闭摄像头

@Override
public void onAttachDev(UsbDevice usbDevice) {
    Log.i(TAG, "onAttachDev: USB_DEVICE_ATTACHED " + usbDevice.toString());
    runOnUiThread(() -> Toast.makeText(this, "USB_DEVICE_ATTACHED", Toast.LENGTH_SHORT).show());
    if (!mManager.isRequestPermission(usbDevice)) {
        mManager.requestPermission(usbDevice);
    }
}

@Override
public void onDetachDev(UsbDevice usbDevice) {
    Log.i(TAG, "onDetachDev: USB_DEVICE_DETACHED " + usbDevice.toString());
    runOnUiThread(() -> Toast.makeText(this, "USB_DEVICE_DETACHED", Toast.LENGTH_SHORT).show());
    mManager.closeCamera(usbDevice);
}

@Override
public void onConnectDev(UsbDevice usbDevice, USBMonitor.UsbControlBlock usbControlBlock, boolean createNew) {
    Log.i(TAG, "onConnectDev: USB_DEVICE_CONNECT " + usbDevice.toString());
    if (!mManager.isCameraOpened(usbControlBlock)) {
        boolean ok = false;
        if (mManager.findCameraViewIndex(cameraPreviewOne) < 0) {
            ok = mManager.openCamera(usbDevice, usbControlBlock, cameraPreviewOne);
        } else if (mManager.findCameraViewIndex(cameraPreviewTwo) < 0) {
            ok = mManager.openCamera(usbDevice, usbControlBlock, cameraPreviewTwo);
        }
        if (ok) setParam(usbDevice);
        // 如果有摄像头打开失败的话可以调用摄像头复位接口，比如MTK平台的切换OTG接口
        if (mManager.isCameraOpenFail()) resetUsb();
    }
}

@Override
public void onDisconnectDev(UsbDevice usbDevice, USBMonitor.UsbControlBlock usbControlBlock) {
    Log.i(TAG, "onDisconnectDev: USB_DEVICE_DISCONNECT " + usbDevice.toString());
    mManager.closeCamera(usbDevice);
}

获取权限与关闭摄像头逻辑都比较简单，这里主要介绍下打开摄像头，首先需要判断摄像头是否已打开，isCameraOpened会判断UsbControlBlock对象是否存在，不存在则表示未打开

public boolean isCameraOpened(USBMonitor.UsbControlBlock ctrlBlock) {
    return findUsbControlBlockIndex(ctrlBlock) >= 0;
}

public int findUsbControlBlockIndex(USBMonitor.UsbControlBlock ctrlBlock) {
    for (int i = 0; i < mCtrlBlockArr.length; ++i) {
        if (ctrlBlock.equals(mCtrlBlockArr[i])) {
            return i;
        }
    }
    return -1;
}

然后看一下打开摄像头相关代码，首先判断UsbDevice对象是否存在（该对象在获取到权限时会保存），然后打开摄像头并保存UsbControlBlock对象，最后通过jni调用libuvc的打开摄像头相关代码

public boolean openCamera(UsbDevice dev, USBMonitor.UsbControlBlock ctrlBlock, CameraViewInterface cameraView) {
    int index = findUsbDeviceIndex(dev);
    if (index >= 0) {
        Log.i(TAG, "openCamera: index " + index);
        Log.i(TAG, "openCamera: cameraView " + cameraView);
        int result = openCamera(index, ctrlBlock);
        if (result == 0) {
            mCtrlBlockArr[index] = ctrlBlock;
            mIsOpenArr[index] = true;
            if (cameraView != null) {
                cameraView.openCamera(index);
                mCamViewArr[index] = cameraView;
            }
            return true;
        } else {
            mIsOpenArr[index] = false;
            Log.e(TAG, "openCamera: error " + result);
        }
    } else {
        Log.e(TAG, "openCamera: usbDevice not found");
    }
    return false;
}

public int openCamera(int index, USBMonitor.UsbControlBlock ctrlBlock) {
    Log.i(TAG, "openCamera: ");
    Log.i(TAG, "getVendorId: " + ctrlBlock.getVendorId());
    Log.i(TAG, "getProductId: " + ctrlBlock.getProductId());
    Log.i(TAG, "getFileDescriptor: " + ctrlBlock.getFileDescriptor());
    Log.i(TAG, "getBusNum: " + ctrlBlock.getBusNum());
    Log.i(TAG, "getDevNum: " + ctrlBlock.getDevNum());
    Log.i(TAG, "getUSBFSName: " + ctrlBlock.getUSBFSName());
    UsbCameraLib.setStreamMode(streamMode);
    return UsbCameraLib.connect(index, ctrlBlock.getVendorId(), ctrlBlock.getProductId(),
            ctrlBlock.getFileDescriptor(), ctrlBlock.getBusNum(), ctrlBlock.getDevNum(),
            ctrlBlock.getUSBFSName());
}

public int findUsbDeviceIndex(UsbDevice dev) {
    for (int i = 0; i < mDeviceArr.length; ++i) {
        UsbDevice usbDevice = mDeviceArr[i];
        if (dev.equals(usbDevice)) {
            return i;
        }
    }
    return -1;
}

public int findCameraViewIndex(CameraViewInterface cameraView) {
    for (int i = 0; i < mCamViewArr.length; ++i) {
        if (cameraView.equals(mCamViewArr[i])) {
            return i;
        }
    }
    return -1;
}

摄像头打开成功后可以设置摄像头参数参数，可以设置曝光方式（自动或手动），曝光时长（手动曝光可用），增益（gain）和亮度

private void setParam(UsbDevice usbDevice) {
    mManager.setParam(usbDevice, 0, 0.0157f, 0, 50);
    // mManager.setParam(usbDevice, 1, 0.015f, 15, 50);
}

如果在openCamera中传入CameraViewInterface的话则表示需要预览图像，CameraPreview这个SurfaceView实现了这个接口，会保存摄像头的index值用来具体对应不同的摄像头，以及控制打开摄像头的标志位

public interface CameraViewInterface {
    void openCamera(int index);
    void closeCamera();
}

public class CameraPreview extends SurfaceView implements SurfaceHolder.Callback, Runnable, CameraViewInterface {
    ...

    @Override
    public void openCamera(int index) {
        Log.i(TAG, "openCamera: " + index);
        isOpen = true;
        this.index = index;
    }

    @Override
    public void closeCamera() {
        Log.i(TAG, "closeCamera: " + index);
        isOpen = false;
        this.index = -1;
    }
}

预览图像

使用SurfaceView进行绘制，在while循环中通过jni将原始帧数据放入bitmap中

@Override
public void run() {
    while (!Thread.interrupted()) {

        if (isOpen && index >= 0) {
            UsbCameraLib.pixeltobmp(index, bmp);
        }

        Canvas canvas = getHolder().lockCanvas();
        if (canvas != null) {
            canvas.drawBitmap(bmp, mSrcRect, mDstRect, null);
            getHolder().unlockCanvasAndPost(canvas);
        }

        frame++;
    }
}

@Override
public void surfaceCreated(SurfaceHolder holder) {
    Log.i(TAG, "surfaceCreated");
    if (bmp == null) {
        bmp = Bitmap.createBitmap(IMG_WIDTH, IMG_HEIGHT, Bitmap.Config.RGB_565);
    }

    mSrcRect = new Rect(0, 0, bmp.getWidth(), bmp.getHeight());
    mDstRect = new Rect(0, 0, getWidth(), getHeight());

    mainLoop = new Thread(this);
    mainLoop.start();
}

@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
    Log.i(TAG, "surfaceChanged");
}

@Override
public void surfaceDestroyed(SurfaceHolder holder) {
    Log.i(TAG, "surfaceDestroyed");
    closeCamera();
    mainLoop.interrupt();
    if (bmp != null) {
        bmp.recycle();
    }
}

看一下jni中的pixel_to_bmp函数，会将rgb565的图片像素数据直接memcpy到传递下来的bitmap对象中

void pixel_to_bmp(JNIEnv *env, jclass thiz, int index, jobject bitmap) {
    int ret;
    uvc_frame_t *rgb565 = uvc_allocate_frame(IMG_WIDTH * IMG_HEIGHT * 2);
    ret = get_frame(rgb565, index);
    if (ret) {
        LOGE("pixel_to_bmp get_frame error %d", ret);
        uvc_free_frame(rgb565);
        return;
    }

    AndroidBitmapInfo info;
    void *pixels;
    int i;
    int *colors;

    if ((ret = AndroidBitmap_getInfo(env, bitmap, &info)) < 0) {
        LOGE("pixel_to_bmp AndroidBitmap_getInfo() failed ! error=%d", ret);
        uvc_free_frame(rgb565);
        return;
    }

    int width = info.width;
    int height = info.height;

    if (info.format != ANDROID_BITMAP_FORMAT_RGB_565) {
        LOGE("pixel_to_bmp Bitmap format is not RGB_565!");
        return;
    }

    if ((ret = AndroidBitmap_lockPixels(env, bitmap, &pixels)) < 0) {
        LOGE("pixel_to_bmp AndroidBitmap_lockPixels() failed ! error=%d", ret);
        uvc_free_frame(rgb565);
        return;
    }

    colors = (int *) pixels;

    memcpy(colors, rgb565->data, rgb565->data_bytes);
    uvc_free_frame(rgb565);

    AndroidBitmap_unlockPixels(env, bitmap);
}

最后完整代码可以参考demo

ndk开发基础学习系列

JNI和NDK编程（一）JNI的开发流程
JNI和NDK编程（二）NDK的开发流程
JNI和NDK编程（三）JNI的数据类型和类型签名
JNI和NDK编程（四）JNI调用Java方法的流程

完整demo

https://github.com/GavinAndre/AndroidJNIDemo