mmaction2基于骨架点的行为识别demo复现

深度学习扛把子

已于 2023-01-13 15:29:14 修改

阅读量1.4k

点赞数

文章标签： pytorch

于 2023-01-11 15:29:35 首次发布

本文链接：https://blog.csdn.net/qq_16792139/article/details/128645620

版权

该文详细介绍了如何利用mmaction2库进行视频识别，包括环境配置、模型加载、推理代码的执行。此外，还提到了将模型转换为ONNX格式的步骤，以及在VSCode中配置运行参数的过程。文章中涉及的技术包括FasterR-CNN人体检测器、HRNet姿势估计器和PoseC3D骨架动作识别器。

摘要由CSDN通过智能技术生成

1.环境安装

没啥好讲的

2.运行推理代码

2.1测试视频

这里可以使用官方提供的api（输入数据为视频）

import torch

from mmaction.apis import init_recognizer, inference_recognizer

config_file = 'configs/recognition/tsn/tsn_r50_video_inference_1x1x3_100e_kinetics400_rgb.py'
# download the checkpoint from model zoo and put it in `checkpoints/`
checkpoint_file = 'checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth'

# assign the desired device.
device = 'cpu' # or 'cpu'
device = torch.device(device)

 # build the model from a config file and a checkpoint file
model = init_recognizer(config_file, checkpoint_file, device=device)

# test a single video and show the result:
video = 'demo/demo.mp4'
labels = 'tools/data/kinetics/label_map_k400.txt'
results = inference_recognizer(model, video)
#[(6, 29.616436), (279, 10.754838), (288, 9.908399), (57, 9.189911), (194, 8.305308)]
#这里输出标签的类别以及得分值
# show the results
labels = open('tools/data/kinetics/label_map_k400.txt').readlines()
labels = [x.strip() for x in labels]
results = [(labels[k[0]], k[1]) for k in results]

print(f'The top-5 labels with corresponding scores are:')
for result in results:
    print(f'{result[0]}: ', result[1])

最后输出
在这里插入图片描述
2.1测试图片帧
这里用到ffmpeg视频转图片帧
如果没有安装

conda install -c conda-forge ffmpeg-python

或者

sudo apt install ffmpeg

ffmpeg -i C:\mmaction2\demo\1.MP4 -vf fps=4 C:\mmaction2\demo\output\%3d.jpg

ffmpeg -i /home/tao/Videos/1.MP4 -vf fps=4 /home/tao/mmaction2/video/%3d.jpg

将v文件下的C:\mmaction2\demo\1.MP4视频，按照每秒4帧的方式裁剪，得到的图片保留在：C:\mmaction2\demo\output\当中
运行的时候报错

FileNotFoundError: [Errno 2] No such file or directory: '/home/tao/mmaction2/video/img_00001.jpg'

此时将mmaction2/mmaction/apis/inference.py 中的136行

# filename_tmpl = cfg.data.test.get('filename_tmpl', 'img_{:05}.jpg')
filename_tmpl = cfg.data.test.get('filename_tmpl', '{:03}.jpg')

mmaction2中将模型转onnx的脚本

mmaction运行demo

https://github.com/open-mmlab/mmaction2/blob/master/demo/README.md

https://github.com/open-mmlab/mmaction2/blob/master/demo/README.md#skeleton-based-action-recognition-demo

运行基于骨架点的行为识别demo
使用 Faster RCNN 作为人体检测器，HRNetw32 作为姿势估计器，PoseC3D 作为基于骨架的动作识别器和基于骨架的时空动作检测器。每8帧做动作检测预测，每1帧输出1帧到输出视频。输出视频的 FPS 为 24。

python demo/demo_skeleton.py demo/ntu_sample.avi demo/skeleton_demo.mp4     --config configs/skeleton/posec3d/slowonly_r50_u48_240e_ntu120_xsub_keypoint.py     --checkpoint checkpoints/slowonly_r50_u48_240e_ntu120_xsub_keypoint-6736b03f.pth     --det-config demo/faster_rcnn_r50_fpn_2x_coco.py     --det-checkpoint checkpoints/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth     --det-score-thr 0.9     --pose-config demo/hrnet_w32_coco_256x192.py     --pose-checkpoint checkpoints/hrnet_w32_coco_256x192-c78dce93_20200708.pth     --label-map tools/data/skeleton/label_map_ntu120.txt

环境报错

ModuleNotFoundError: No module named 'mmaction'

可以尝试

pip install mmaction

vscode配置运行参数

```bash
{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "justMyCode": true,
            // "args": [
            //     "configs/recognition/tsn/tsn_r50_video_inference_1x1x3_100e_kinetics400_rgb.py",
            //     "checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth",
            //     "demo/demo.mp4",
            //     "tools/data/kinetics/label_map_k400.txt",
            // ]
            "args": [
                "demo/1.MP4","demo/skeleton_demo.mp4",
                "--config","configs/skeleton/posec3d/slowonly_r50_u48_240e_ntu120_xsub_keypoint.py",
                "--checkpoint","checkpoints/slowonly_r50_u48_240e_ntu120_xsub_keypoint-6736b03f.pth",
                "--det-config","demo/faster_rcnn_r50_fpn_2x_coco.py",
                "--det-checkpoint","checkpoints/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth",
                // "--det-score-thr","0,9",
                "--pose-config","demo/hrnet_w32_coco_256x192.py",
                "--pose-checkpoint","checkpoints/hrnet_w32_coco_256x192-c78dce93_20200708.pth",
                "--label-map","tools/data/skeleton/label_map_ntu120.txt"
            ]
        }
    ]
}

推理完以后转onnx

```bash
python tools/deployment/pytorch2onnx.py configs/skeleton/posec3d/slowonly_r50_u48_240e_ntu120_xsub_keypoint.py checkpoints/slowonly_r50_u48_240e_ntu120_xsub_keypoint-6736b03f.pth --is-localizer --shape 1 1 3 32 224 224  --verify

python tools/deployment/pytorch2onnx.py
configs/yolo/yolov3_d53_mstrain-608_273e_coco.py
checkpoints/yolo/yolov3_d53_mstrain-608_273e_coco.pth
–output-file checkpoints/yolo/yolov3_d53_mstrain-608_273e_coco.onnx
–input-img demo/demo.jpg
–test-img tests/data/color.jpg
–shape 608 608
–show
–verify
–dynamic-export
–cfg-options
model.test_cfg.deploy_nms_pre=-1 \