因为直接用cv2.VideoCapture拉rtsp有较大的延迟,所以采用gstreamer来读。
参考rk3588使用gstreamer推流_rk3588 gstreamer-CSDN博客
安装gstreamer及编码器
搜索软件包,在https://launchpad.net/上搜索软件名+ppa
找到需要的包后下面命令安装
sudo add-apt-repository ppa:george-coolpi/multimedia
sudo apt update
如果不需要了下面命令删除
sudo add-apt-repository -r ppa:george-coolpi/multimedia
sudo apt-get update
安装gstreamer
apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio
下面安装插件gstreamer1.0-rockchip
sudo apt-get install gstreamer1.0-rockchip
查看支持的编码器
gst-inspect-1.0 |grep mpp
这时已经支持硬件编码了
安装支持gstreamer的cv2
添加ppa仓库
sudo add-apt-repository ppa:trinitronx/focal-backport-buildeps
sudo apt update
查看包名
apt list | grep opencv
安装python3-opencv (不用管上面的包名,直接按python3-opencv就行)
sudo apt-get install python3-opencv
查看是否支持gstreamer
import cv2
print(cv2.getBuildInformation())
GStreamer是yes就代表支持GStreamer的opencv安装成功了
拉流测试
import cv2
in_gst= (
"rtspsrc location=rtsp://192.168.144.25:8554/main.264 latency=0 "
"! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink"
)
# 创建VideoCapture对象
cap = cv2.VideoCapture(in_gst, cv2.CAP_GSTREAMER)
# 检查视频是否成功打开
if not cap.isOpened():
print("Error: Cannot open video stream or file")
exit()
# 读取视频帧
while True:
ret, frame = cap.read()
if not ret:
print("Failed to grab frame")
break
# 显示结果帧
cv2.imshow('Frame', frame)
# 按'q'键退出
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# 释放VideoCapture对象
cap.release()
cv2.destroyAllWindows()
有几个点需要注意一下
解码方面,我使用mppvideodec替换avdec_h264 后拉不到流,视频是全黑的
使用上面的代码会有严重的卡顿,可以添加queue缓解。除此之外添加drop=1好像也有效果
in_gst = "rtspsrc location=rtsp://192.168.144.25:8554/main.264 latency=0 ! rtph264depay ! queue ! h264parse ! queue ! avdec_h264 ! queue ! videoconvert ! queue ! appsink drop=1"
直接在终端拉流:
gst-launch-1.0 rtspsrc location=rtsp://192.168.144.25:8554/main.264 latency=0 ! rtph264depay ! queue ! h264parse ! queue ! mppvideodec ! queue ! autovideosink
使用这个版本的opencv和pip直接安装的opencv有区别,这个版本的opencv含有pip安装的opencv-contrib-python的功能,读取usb相机的时候无法直接使用cv2.VideoCapture(0)读取,需要变成
cap = cv2.VideoCapture(0,cv2.CAP_V4L2)
此外也可以使用GStreamer读取usb相机
in_gst = 'v4l2src device=/dev/video0 ! video/x-raw, width=640, height=480, framerate=30/1 ! videoconvert ! appsink'
cap = cv2.VideoCapture(in_gst,cv2.CAP_V4L2)
暂时先记录这么多