1.环境安装
没啥好讲的
2.运行推理代码
2.1测试视频
这里可以使用官方提供的api(输入数据为视频)
import torch
from mmaction.apis import init_recognizer, inference_recognizer
config_file = 'configs/recognition/tsn/tsn_r50_video_inference_1x1x3_100e_kinetics400_rgb.py'
# download the checkpoint from model zoo and put it in `checkpoints/`
checkpoint_file = 'checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth'
# assign the desired device.
device = 'cpu' # or 'cpu'
device = torch.device(device)
# build the model from a config file and a checkpoint file
model = init_recognizer(config_file, checkpoint_file, device=device)
# test a single video and show the result:
video = 'demo/demo.mp4'
labels = 'tools/data/kinetics/label_map_k400.txt'
results = inference_recognizer(model, video)
#[(6, 29.616436), (279, 10.754838), (288, 9.908399), (57, 9.189911), (194, 8.305308)]
#这里输出标签的类别以及得分值
# show the results
labels = open('tools/data/kinetics/label_map_k400.txt').readlines()
labels = [x.strip() for x in labels]
results = [(labels[k[0]], k[1]) for k in results]
print(f'The top-5 labels with corresponding scores are:')
for result in results:
print(f'{result[0]}: ', result[1])
最后输出
2.1测试图片帧
这里用到ffmpeg视频转图片帧
如果没有安装
conda install -c conda-forge ffmpeg-python
或者
sudo apt install ffmpeg
ffmpeg -i C:\mmaction2\demo\1.MP4 -vf fps=4 C:\mmaction2\demo\output\%3d.jpg
ffmpeg -i /home/tao/Videos/1.MP4 -vf fps=4 /home/tao/mmaction2/video/%3d.jpg
将v文件下的C:\mmaction2\demo\1.MP4视频,按照每秒4帧的方式裁剪,得到的图片保留在:C:\mmaction2\demo\output\当中
运行的时候报错
FileNotFoundError: [Errno 2] No such file or directory: '/home/tao/mmaction2/video/img_00001.jpg'
此时将mmaction2/mmaction/apis/inference.py 中的136行
# filename_tmpl = cfg.data.test.get('filename_tmpl', 'img_{:05}.jpg')
filename_tmpl = cfg.data.test.get('filename_tmpl', '{:03}.jpg')
mmaction2中将模型转onnx的脚本
mmaction运行demo
https://github.com/open-mmlab/mmaction2/blob/master/demo/README.md
https://github.com/open-mmlab/mmaction2/blob/master/demo/README.md#skeleton-based-action-recognition-demo
运行基于骨架点的行为识别demo
使用 Faster RCNN 作为人体检测器,HRNetw32 作为姿势估计器,PoseC3D 作为基于骨架的动作识别器和基于骨架的时空动作检测器。每8帧做动作检测预测,每1帧输出1帧到输出视频。输出视频的 FPS 为 24。
python demo/demo_skeleton.py demo/ntu_sample.avi demo/skeleton_demo.mp4 --config configs/skeleton/posec3d/slowonly_r50_u48_240e_ntu120_xsub_keypoint.py --checkpoint checkpoints/slowonly_r50_u48_240e_ntu120_xsub_keypoint-6736b03f.pth --det-config demo/faster_rcnn_r50_fpn_2x_coco.py --det-checkpoint checkpoints/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth --det-score-thr 0.9 --pose-config demo/hrnet_w32_coco_256x192.py --pose-checkpoint checkpoints/hrnet_w32_coco_256x192-c78dce93_20200708.pth --label-map tools/data/skeleton/label_map_ntu120.txt
环境报错
ModuleNotFoundError: No module named 'mmaction'
可以尝试
pip install mmaction
vscode配置运行参数
```bash
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python: Current File",
"type": "python",
"request": "launch",
"program": "${file}",
"console": "integratedTerminal",
"justMyCode": true,
// "args": [
// "configs/recognition/tsn/tsn_r50_video_inference_1x1x3_100e_kinetics400_rgb.py",
// "checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth",
// "demo/demo.mp4",
// "tools/data/kinetics/label_map_k400.txt",
// ]
"args": [
"demo/1.MP4","demo/skeleton_demo.mp4",
"--config","configs/skeleton/posec3d/slowonly_r50_u48_240e_ntu120_xsub_keypoint.py",
"--checkpoint","checkpoints/slowonly_r50_u48_240e_ntu120_xsub_keypoint-6736b03f.pth",
"--det-config","demo/faster_rcnn_r50_fpn_2x_coco.py",
"--det-checkpoint","checkpoints/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth",
// "--det-score-thr","0,9",
"--pose-config","demo/hrnet_w32_coco_256x192.py",
"--pose-checkpoint","checkpoints/hrnet_w32_coco_256x192-c78dce93_20200708.pth",
"--label-map","tools/data/skeleton/label_map_ntu120.txt"
]
}
]
}
推理完以后转onnx
```bash
python tools/deployment/pytorch2onnx.py configs/skeleton/posec3d/slowonly_r50_u48_240e_ntu120_xsub_keypoint.py checkpoints/slowonly_r50_u48_240e_ntu120_xsub_keypoint-6736b03f.pth --is-localizer --shape 1 1 3 32 224 224 --verify
python tools/deployment/pytorch2onnx.py
configs/yolo/yolov3_d53_mstrain-608_273e_coco.py
checkpoints/yolo/yolov3_d53_mstrain-608_273e_coco.pth
–output-file checkpoints/yolo/yolov3_d53_mstrain-608_273e_coco.onnx
–input-img demo/demo.jpg
–test-img tests/data/color.jpg
–shape 608 608
–show
–verify
–dynamic-export
–cfg-options
model.test_cfg.deploy_nms_pre=-1 \