Jetson Xavier安装使用多个摄像头

效果展示：

单目全向视觉感知：

通过无人机前后左右四个摄像头，获取周围环境信息

---

四摄读取问题

Jetson Xavier NX虽然带有四个USB接口，但是一般情况下无法同时读取四个USB摄像头数据，直接使用 VideoCapture 函数创建实例虽然能够使用 isOpened 函数确定四个摄像头都已打开，但是使用 read 函数时却发现只能读取两个摄像头数据（在读取第二个以上的usb摄像头图像帧时会报错），即使降低分辨率也无法同时读取四个

# front back left right
assume_cameras = [0, 1, 2, 3]

camera_client_dict = {
    "front_camera": cv2.VideoCapture(assume_cameras[0]),
    "back_camera":  cv2.VideoCapture(assume_cameras[1]),
    "left_camera":  cv2.VideoCapture(assume_cameras[2]),
    "right_camera": cv2.VideoCapture(assume_cameras[3])
}

for capture in camera_client_dict:
    if camera_client_dict[capture].isOpened():
        ret, frame = camera_client_dict[capture].read()
    else:
        frame = empty_image.copy()

---

原因分析：

使用OpenCV库读取摄像头帧时存在USB带宽分配机制问题

与一般的Ubuntu系统一样，jetson xavier NX/AGX 在使用OpenCV库处理USB摄像头时对USB带宽分配的策略都是先到先得的机制，并且貌似是直接赋予USB单个端口的全部带宽，导致开始读取前两个摄像头数据推流之后，USB总线带宽几乎所剩无几，无法读取下一个摄像头数据。

---

解决方案：

限制每个USB端口的带宽

在NX上博主并未成功实现，但是该方案是可行的。

使用两个CSI摄像头和两个USB摄像头

# USB(front back) && CSI(left right)
assume_cameras = [0, 1, 2, 3]

camera_client_dict = {
    "front_camera": cv2.VideoCapture(assume_cameras[2]),
    "back_camera":  cv2.VideoCapture(assume_cameras[3]),
    "left_camera":  cv2.VideoCapture("nvarguscamerasrc sensor-id=0 ! "
                        "video/x-raw(memory:NVMM), "
                        "width=1280, height=720, format=NV12, framerate=30/1 ! "
                        "nvvidconv flip-method=0 ! "
                        "video/x-raw, width=640, height=360, format=BGRx ! "
                        "videoconvert ! video/x-raw, format=BGR ! appsink "),
    "right_camera": cv2.VideoCapture("nvarguscamerasrc sensor-id=1 ! "
                        "video/x-raw(memory:NVMM), "
                        "width=1280, height=720, format=NV12, framerate=30/1 ! "
                        "nvvidconv flip-method=0 ! "
                        "video/x-raw, width=640, height=360, format=BGRx ! "
                        "videoconvert ! video/x-raw, format=BGR ! appsink "))
}

for capture in camera_client_dict:
    if camera_client_dict[capture].isOpened():
        ret, frame = camera_client_dict[capture].read()
    else:
        frame = empty_image.copy()

使用jetson-inference库

按照 jetson-inference 链接中的 Building the Project from Source 步骤编译，可以取消勾选所有模型下载，编译安装后在python文件中导入 jetson.utils 库

import cv2
import jetson.utils
import numpy as np

# N为不同的摄像头编号
source = jetson.utils.videoSource('/dev/videoN')

while True:
	_frame = source.Capture()
	frame = np.array(_frame)
	frame = np.asarray(frame)
	
	cv2.imshow("result", frame)
	key = cv2.waitKey(1)
	
	if key == 27 or key == ord('q):
		cv2.destroyAllWindows()
		break