Android在后台读取UVC摄像头的帧数据流并推送

Android在后台读取UVC摄像头的帧数据流并推送

  1. 添加UvcCamera依赖库
    使用原版的 saki4510t/UVCCamera 在预览过程中断开可能会闪退,这里使用的是
    jiangdongguo/AndroidUSBCamera 中修改的版本,下载到本地即可。
    https://github.com/jiangdongguo/AndroidUSBCamera

  2. 监听UVC连接回调

    mUSBMonitor = USBMonitor(context, mOnDeviceConnectListener)
    mUSBMonitor.register()

    public interface OnDeviceConnectListener {
        void onAttach(UsbDevice device);

        void onDetach(UsbDevice device);

        void onConnect(UsbDevice device, UsbControlBlock ctrlBlock, boolean createNew);

        void onDisconnect(UsbDevice device, UsbControlBlock ctrlBlock);

        void onCancel(UsbDevice device);
    }
  1. 检测到UVC后连接该设备

USB连接上会回调,onAttach, 本地判断连接上的USB设备是否UVC,如果是的话可以尝试调用连接该对象。调用mUSBMonitor.requestPermission(cam)就会请求权限并且连接该对象。连接成功后会回调 onConnect。

    var connectJob: Disposable? = null
    override fun onAttach(device: UsbDevice?) {
        BLLog.i(TAG, "onAttach")
        BLLog.toast("USB_DEVICE_ATTACHED")
        connectJob?.dispose()
        connectJob = CommonUtils.runDelayed(1000) {
            autoConnectUvcDevice()
        }
    }
    private fun autoConnectUvcDevice() {
        val context = BLSession.getApplicationContext()
        val filter: List<DeviceFilter> =
            DeviceFilter.getDeviceFilters(context, R.xml.device_filter_uvc)
        val devs: List<UsbDevice> = mUSBMonitor?.getDeviceList(filter[0]) ?: listOf()

        val cam = findUsbCam(devs)
        BLLog.log2File(
            TAG, "autoConnect 共有USB数量:${devs.size}, Uvc: ${cam?.productName}"
        )
        if (cam == null) {
            BLLog.i(TAG, "未连接USB摄像头")
        } else {
            mUSBMonitor.requestPermission(cam)
        }
    }

device_filter_uvc.xml

<usb>
	<usb-device class="239" subclass="2" />	<!-- all device of UVC -->
</usb>

如何判断该连接对象是UVC对象,如果名字中包含USBCam或 interfaceClass = USB_CLASS_VIDEO

    private fun findUsbCam(devs: List<UsbDevice>): UsbDevice? {
        for (dev in devs) {
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
                val name = "" + dev.productName + dev.manufacturerName
                BLLog.i(TAG, "findUsbCam name:$name")
                if (name.contains("USBCam")) {
                    return dev
                }
                for (i in 0 until dev.interfaceCount) {
                    val inter = dev.getInterface(i)
                    BLLog.i(TAG, "getInterface($i):$inter")
                    if (inter.interfaceClass == UsbConstants.USB_CLASS_VIDEO) {
                        return dev
                    }
                }
            }
        }
        return null
    }
  1. 在onConnect中保存UvcCamera对象
            override fun onConnect(
                device: UsbDevice?,
                ctrlBlock: USBMonitor.UsbControlBlock?,
                createNew: Boolean
            ) {
                BLLog.i(TAG, "onConnect  ${device?.productName}")
                synchronized(mSync) {
                    try {
                        // 保存最新的Uvc对象
                        val camera = UVCCamera();
                        camera.open(ctrlBlock)
                        BLLog.log2File(
                            TAG,
                            "supportedSize:" + camera.supportedSize + "thread:" + Thread.currentThread().name
                        )
                        if (applyPreviewSize(camera)) {
                            mUVCCamera?.destroy()
                            mUVCCamera = camera
                            previewStatus = PreViewStatus.None
                        } else {
                            BLLog.console(TAG, "UVC不支持此分辨率:$defPreviewSize")
                            if (mUVCCamera == null) {
                                mUVCCamera = camera
                            }
                        }
                        uvcChangedSub.onNext(true)
                    } catch (e:Exception){
                        BLLog.log2File(TAG, "onConnect Recv exception: $e")
                    }
                }
            }
  1. 预览并获取YUC视频帧
    如果需要预览到UI中显示,需要创建SurfaceView或者TextureView.
        mUVCCamera?.setPreviewDisplay(previewSurface)
        mUVCCamera?.startPreview()

如果不需要预览到UI中显示,可以new一个SurfaceTexture对象传进去即可;必须要调用预览才能获取到YUV数据。

        val surfaceTexture = SurfaceTexture(0)
        mUVCCamera?.setPreviewTexture(surfaceTexture)
        BLLog.i(TAG, "startPreviewWithAir")
        mUVCCamera?.startPreview()

预览后获取YUV帧流:

        val width = mUVCCamera?.previewSize?.width ?: defPreviewSize.width
        val height = mUVCCamera?.previewSize?.height ?: defPreviewSize.height

        yuvCallback = callback
        mUVCCamera?.setFrameCallback({ buffer: ByteBuffer ->
//            BLLog.i(TAG, "onFrame ${width}*${height}")
            // yuv格式
            if (acceptFrame()) {
                val format = MediaFormat.createVideoFormat("", width, height)
                val data = ByteArray(buffer.remaining())
                buffer.get(data)
                val frame = YuvFrameData(format, data, width, height)
                yuvCallback?.onGetYuvData(frame)
            }
        }, UVCCamera.PIXEL_FORMAT_NV21)

获取到的YUV帧可以使用其他推流SDK进行推流即可,比如使用阿里云推流SDK推流。
这完成可以在后台进行推流,不需要UI上展示,节省设备的性能。

  1. 连接类参考:
// Uvc设备连接器
object UvcConnector : BaseBussModel(ModelType.Shared) {
    private val TAG = UvcConnector::class.java.simpleName
    private val KEY_UVC_PREVIEW_SIZE = "KEY_UVC_PREVIEW_SIZE"

    // 默认支持:640*480, 1920*1080
    private var defPreviewSize = MySize.parseSize("1920*1080")!!

    enum class PreViewStatus {
        None,
        Visible,
        Air,
    }

    private lateinit var mUSBMonitor: USBMonitor

    @Volatile
    private var mUVCCamera: UVCCamera? = null
    private val mSync = Object()
    private var previewStatus = PreViewStatus.None

    @Volatile
    private var yuvCallback: IMediaKit.OnYuvListener? = null

    // 状态变更消息
    private var uvcChangedSub = PublishSubject.create<Boolean>()

    override fun onStartUp() {
        super.onStartUp()
        BLLog.i(TAG, "onStartUp")

        val context = BLSession.getApplicationContext()
        mUSBMonitor = USBMonitor(context, mOnDeviceConnectListener)
        mUSBMonitor.register()

        CommonUtils.runAsync(::loadPreviewSize)
    }

    override fun onShutdown() {
        BLLog.i(TAG, "onShutdown")
        mUSBMonitor.unregister()
        mUSBMonitor.destroy()
        super.onShutdown()
    }

    fun hasUvcDevice(): Boolean {
        return mUVCCamera != null
    }

    fun previewStatus(): PreViewStatus {
        return previewStatus
    }

    fun getSubject() = uvcChangedSub

    fun getNowSize(): MySize? {
        return mUVCCamera?.previewSize?.let {
            MySize(it.width, it.height)
        }
    }

    fun getExpSize(): MySize {
        return defPreviewSize
    }

    fun startPreview(previewSurface: Surface): CallResult {
        BLLog.i(TAG, "startPreview")
        if (!hasUvcDevice()) {
            return CallResult(false, "未连接设备")
        }

        if (previewStatus == PreViewStatus.Air) {
            mUVCCamera?.stopPreview()
        }

        if (!applyPreviewSize(mUVCCamera)) {
            BLLog.console(TAG, "UVC不支持此分辨率:$defPreviewSize")
            return CallResult(false, "UVC不支持此分辨率:$defPreviewSize")
        }
        mUVCCamera?.setPreviewDisplay(previewSurface)
        mUVCCamera?.startPreview()
        previewStatus = PreViewStatus.Visible

        if (yuvCallback != null) {
            setYuvCallback(yuvCallback!!)
        }
        uvcChangedSub.onNext(true)
        return CallResult(true, "成功")
    }

    fun stopPreview() {
        BLLog.i(TAG, "stopPreview")
        if (previewStatus != PreViewStatus.Visible) {
            return
        }

        mUVCCamera?.stopPreview()
        previewStatus = PreViewStatus.None

        // 需要接收数据
        if (yuvCallback != null) {
            startPreviewWithAir()
            setYuvCallback(yuvCallback!!)
        }
        uvcChangedSub.onNext(true)
    }

    fun clearYuvCallback() {
        yuvCallback = null

        if (previewStatus == PreViewStatus.Air) {
            mUVCCamera?.stopPreview()
            previewStatus = PreViewStatus.None

            uvcChangedSub.onNext(true)
        }
    }

    fun setYuvCallback(callback: IMediaKit.OnYuvListener): Boolean {
        if (mUVCCamera == null) {
            return false
        }

        if (previewStatus == PreViewStatus.None) {
            startPreviewWithAir()
        }

        val width = mUVCCamera?.previewSize?.width ?: defPreviewSize.width
        val height = mUVCCamera?.previewSize?.height ?: defPreviewSize.height

        yuvCallback = callback
        mUVCCamera?.setFrameCallback({ buffer: ByteBuffer ->
//            BLLog.i(TAG, "onFrame ${width}*${height}")
            // yuv格式
            if (acceptFrame()) {
                val format = MediaFormat.createVideoFormat("", width, height)
                val data = ByteArray(buffer.remaining())
                buffer.get(data)
                val frame = YuvFrameData(format, data, width, height)
                yuvCallback?.onGetYuvData(frame)
            }
        }, UVCCamera.PIXEL_FORMAT_NV21)
        return true
    }

    private fun acceptFrame(): Boolean {
        return Random.nextInt(30) <= 25
    }

    private fun startPreviewWithAir() {
        if (!applyPreviewSize(mUVCCamera)) {
            BLLog.console(TAG, "UVC不支持此分辨率:$defPreviewSize")
            return
        }

        val surfaceTexture = SurfaceTexture(0)
        mUVCCamera?.setPreviewTexture(surfaceTexture)
        BLLog.i(TAG, "startPreviewWithAir")
        mUVCCamera?.startPreview()
        previewStatus = PreViewStatus.Air
        uvcChangedSub.onNext(true)
    }

    private fun findUsbCam(devs: List<UsbDevice>): UsbDevice? {
        for (dev in devs) {
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
                val name = "" + dev.productName + dev.manufacturerName
                BLLog.i(TAG, "findUsbCam name:$name")
                if (name.contains("USBCam")) {
                    return dev
                }
                for (i in 0 until dev.interfaceCount) {
                    val inter = dev.getInterface(i)
                    BLLog.i(TAG, "getInterface($i):$inter")
                    if (inter.interfaceClass == UsbConstants.USB_CLASS_VIDEO) {
                        return dev
                    }
                }
            }
        }
        return null
    }

    private fun autoConnectUvcDevice() {
        val context = BLSession.getApplicationContext()
        val filter: List<DeviceFilter> =
            DeviceFilter.getDeviceFilters(context, R.xml.device_filter_uvc)
        val devs: List<UsbDevice> = mUSBMonitor?.getDeviceList(filter[0]) ?: listOf()

        val cam = findUsbCam(devs)
        BLLog.log2File(
            TAG, "autoConnect 共有USB数量:${devs.size}, Uvc: ${cam?.productName}"
        )
        if (cam == null) {
            BLLog.i(TAG, "未连接USB摄像头")
        } else {
            mUSBMonitor.requestPermission(cam)
        }
    }

    private fun applyPreviewSize(camera: UVCCamera?):Boolean {
        if (camera == null){
            return false
        }

        try {
            camera.setPreviewSize(
                defPreviewSize.width,
                defPreviewSize.height,
                UVCCamera.FRAME_FORMAT_MJPEG
            )
        } catch (e: IllegalArgumentException) {
            BLLog.log2File(TAG, "setPreviewSize1 $defPreviewSize: $e")
            try {
                // fallback to YUV mode
                camera.setPreviewSize(
                    defPreviewSize.width,
                    defPreviewSize.height,
                    UVCCamera.DEFAULT_PREVIEW_MODE
                )
            } catch (e1: IllegalArgumentException) {
                BLLog.log2File(TAG, "setPreviewSize2 $defPreviewSize: $e1")
                return false
            }
        }
        return true
    }

    private val mOnDeviceConnectListener: USBMonitor.OnDeviceConnectListener =
        object : USBMonitor.OnDeviceConnectListener {
            var connectJob: Disposable? = null
            override fun onAttach(device: UsbDevice?) {
                BLLog.i(TAG, "onAttach")
                BLLog.toast("USB_DEVICE_ATTACHED")
                connectJob?.dispose()
                connectJob = CommonUtils.runDelayed(1000) {
                    autoConnectUvcDevice()
                }
            }

            override fun onDetach(device: UsbDevice?) {
                BLLog.i(TAG, "onDetach")
                BLLog.toast("USB_DEVICE_DETACHED")
                synchronized(mSync) {
                    if (mUVCCamera != null) {
                        mUVCCamera?.destroy()
                        mUVCCamera = null
                        previewStatus = PreViewStatus.None
                        uvcChangedSub.onNext(true)
                    }
                }
            }

            override fun onConnect(
                device: UsbDevice?,
                ctrlBlock: USBMonitor.UsbControlBlock?,
                createNew: Boolean
            ) {
                BLLog.i(TAG, "onConnect  ${device?.productName}")
                synchronized(mSync) {
                    try {
                        // 保存最新的Uvc对象
                        val camera = UVCCamera();
                        camera.open(ctrlBlock)
                        BLLog.log2File(
                            TAG,
                            "supportedSize:" + camera.supportedSize + "thread:" + Thread.currentThread().name
                        )
                        if (applyPreviewSize(camera)) {
                            mUVCCamera?.destroy()
                            mUVCCamera = camera
                            previewStatus = PreViewStatus.None
                        } else {
                            BLLog.console(TAG, "UVC不支持此分辨率:$defPreviewSize")
                            if (mUVCCamera == null) {
                                mUVCCamera = camera
                            }
                        }
                        uvcChangedSub.onNext(true)
                    } catch (e:Exception){
                        BLLog.log2File(TAG, "onConnect Recv exception: $e")
                    }
                }
            }

            override fun onDisconnect(device: UsbDevice?, ctrlBlock: USBMonitor.UsbControlBlock?) {
                BLLog.i(TAG, "onDisconnect ${device?.productName}")
                synchronized(mSync) {
                    mUVCCamera?.destroy()
                    mUVCCamera = null
                    previewStatus = PreViewStatus.None
                    uvcChangedSub.onNext(true)
                }
            }

            override fun onCancel(device: UsbDevice?) {
                BLLog.i(TAG, "onCancel")
            }
        }

    fun setPreviewSize(size: MySize) {
        defPreviewSize = size
        SharedPreferenceHelper.saveCustom(KEY_UVC_PREVIEW_SIZE, size.toString())
    }

    private fun loadPreviewSize() {
        val str = SharedPreferenceHelper.loadCustom(KEY_UVC_PREVIEW_SIZE, "")
        defPreviewSize = MySize.parseSize(str) ?: defPreviewSize
    }
}
  • 9
    点赞
  • 9
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
STM32F407是一款强大的微控制器,可以用来读取UVC摄像头的视频数据。首先,我们需要将UVC摄像头连接到STM32F407的USB接口。接着,通过配置USB的相关寄存器,使其工作在UVC模式下。 在程序中,我们需要使用STM32 HAL库提供的相关函数来初始化USB接口和UVC功能。首先,我们需要初始化USB设备,设置相关参数,如设备描述符、配置描述符等。然后,我们需要设置UVC功能,包括视频流描述符、控制描述符等。为了读取视频数据,我们需要配置一个端点来接收视频数据。 一旦USB和UVC功能都初始化完成,我们可以通过中断或轮询的方式来读取视频数据。当有视频数据到达时,我们可以通过USB中断或者查询USB状态寄存器来检测。一旦检测到有数据到达,我们可以通过读取USB的FIFO寄存器来获取数据。 获取到的视频数据可能是压缩格式,我们需要根据摄像头的规格来解析数据。例如,若使用MJPEG格式,我们可以根据摄像头规格解析JPEG数据。如果是其他格式,我们需要使用相应的算法来解析。 最后,我们可以将解析后的视频数据用于后续的处理或显示出来。我们可以将视频数据传输到其他设备,如LCD显示屏或计算机上进行处理和展示。 需要注意的是,由于STM32F407的存储空间有限,对于较大的视频数据,可能需要使用一些缓冲技术或者剪裁数据来保存和处理。此外,对于高分辨率和高率的视频,STM32F407的处理能力可能会有限,因此需要根据实际需求进行优化和选择适当的编码方式。
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值