第一步:选定算法
商汤MMACTION2部署
注:因为要部署到ARM平台,所以本文大部分代码的使用全都是基于编译,而不是直接pip安装。
Step1:TX2安装pytorch
https://forums.developer.nvidia.com/t/pytorch-for-jetson-version-1-8-0-now-available/72048
源码编译
先把cython装好,不然装numpy的时候报错,注意顺序
源码编译python setup.py bdist之后,到dist文件夹下找whl安装,不要从官网直接找whl安装,因为会报与cudnn版本不匹配的问题!!
Step2:TX2安装mmcv(高于1.2.8版本,readme里面说高于1.1.1就行,其实不行)
git clone 下来之后,git checkout v1.1.2即可切换版本,然后修改setup.py部分内容
Step3:TX2源码编译decord
注意,3rdpart里面有一些代码很重要,如果git clone --recursive不成功,那么就只能一个个下载下来之后编译,注意,cmake编译的参数CUDA不能用,只能用CPU剪视频。
https://github.com/dmlc/decord.git
Step3:TX2安装mmaction2
先安装torchvision 0.8.1
pip install -v -e . # or “python setup.py develop”
百度PaddlePaddle部署
第二步:rtsp推流收流框架搭建
服务器测试
一个线程获取rtsp,并将视频循环截取保存。视频分辨率320x240
# coding: utf-8
import datetime
import cv2
import os
rtsp = 'rtsp://admin:vsislab123@180.201.0.199:554//Streaming/Channels/1'
# 初始化摄像头
cap = cv2.VideoCapture(rtsp)
fourcc = cv2.VideoWriter_fourcc('M','J','P','G')
size = (320, 240)
i = 0
frame_count = 0
while cap.isOpened():
isSuccess, frame = cap.read()
if isSuccess:
if frame_count % 300 == 0 or frame_count == 0:
frame_count = 0
# i = datetime.datetime.now().strftime("%Y%m%d%H%M%S")
if i == 4:
i = 0
i = i + 1
filename = "./realtime_video/" + str(i) + '.avi'
#filename = 'demo.avi'
print(filename)
video_writer = cv2.VideoWriter(filename, fourcc, 24, size)
frame_resized = cv2.resize(frame, (320, 240), interpolation = cv2.INTER_AREA)
video_writer.write(frame_resized)
frame_count = frame_count + 1
print(frame_count)
if frame_count % 300 == 0:
video_writer.release()
# cv2.waitKey(0)
portion = os.path.splitext(filename)
newname = portion[0] + '.mp4'
os.rename(filename, newname)
cv2.namedWindow('show',cv2.WINDOW_NORMAL)
cv2.imshow('show', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
一个线程送入TSN分析
#!bin/bash
# judge whether there exists a video or not
while :
do
for video in /home/ljh/Documents/action_detect/mmaction2/realtime_video/*.mp4
do
if test -f $video
then
echo $video is video
# step3 run demo
python3 demo/demo.py configs/recognition/mytest/tsn_r50_1x1x3_100e_kinetics400_rgb.py checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth $video ./demo/label_map_k400.txt
fi
done
done
可以本地直接查看组合成的视频:rtsp://localhost:8554/vlc
看完之后,将组合成的视频推流
TX2端
同上,就是延迟过高
参考:
https://blog.csdn.net/qq_24282081/article/details/107285072
https://blog.csdn.net/qq_43229471/article/details/105973982
http://element-ui.cn/article/show-79718.aspx