MMPose学习笔记
学习链接:mmpose教程-CSDN博客
视频学习:同济子豪兄的个人空间-同济子豪兄个人主页-哔哩哔哩视频 (bilibili.com)
官方教程:概述 — MMPose 1.2.0 文档
实时查看gpu使用率
pip install gpustat
watch --color -n1 gpustat -cpu
文章目录
安装教程
-
创建环境
conda create --n yxy python=3.8 conda activate yxy
-
安装必要依赖
conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3 -c pytorch pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.11.0/index.html pip install -U openmim -i https://pypi.douban.com/simple mim install mmengine
-
安装mmpose
git clone https://github.com/open-mmlab/mmpose.git cd mmpose pip install -r requirements.txt pip install -v -e . # "-v" 表示输出更多安装相关的信息 # "-e" 表示以可编辑形式安装,这样可以在不重新安装的情况下,让本地修改直接生效
-
验证安装
mim download mmpose --config td-hm_hrnet-w48_8xb32-210e_coco-256x192 --dest .
python mmpose/demo/image_demo.py \ mmpose/tests/data/coco/000000000785.jpg \ td-hm_hrnet-w48_8xb32-210e_coco-256x192.py \ td-hm_hrnet-w48_8xb32-210e_coco-256x192-0e67c616_20220913.pth \ --out-file vis_results.jpg \ --draw-heatmap
-
安装mmdet
git clone https://github.com/open-mmlab/mmdetection.git -b 3.x cd mmdetection pip install -v -e .
快速开始
1.使用命令测试数据
# 单 GPU 测试
python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [--out ${RESULT_FILE}] [--fuse-conv-bn] \
[--eval ${EVAL_METRICS}] [--gpu_collect] [--tmpdir ${TMPDIR}] [--cfg-options ${CFG_OPTIONS}] \
[--launcher ${JOB_LAUNCHER}] [--local_rank ${LOCAL_RANK}]
# 多 GPU 测试
./tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${GPU_NUM} [--out ${RESULT_FILE}] [--eval ${EVAL_METRICS}] \
[--gpu-collect] [--tmpdir ${TMPDIR}] [--options ${OPTIONS}] [--average-clips ${AVG_TYPE}] \
[--launcher ${JOB_LAUNCHER}] [--local_rank ${LOCAL_RANK}]
例如:
-
在coco测试resnet50,验证mAP
tools/dist_test.sh configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res50_coco_256x192.py checkpoints/SOME_CHECKPOINT.pth 1 --eval mAP
-
4 块 GPU 在 COCO 数据集下测试 ResNet。在线下载模型权重,并验证
mAP
指标tools/dist_test.sh configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/res50_coco_256x192.py https://download.openmmlab.com/mmpose/top_down/resnet/res50_coco_256x192-ec54d7f3_20200709.pth 4 --eval mAP
-
提供了丰富的脚本,方便大家快速运行演示。 下面是 多人人体姿态估计 的演示示例,此处我们使用了人工标注的人体框作为输入
python demo/top_down_img_demo.py \ configs/body/2D_Kpt_SV_RGB_Img/topdown_hm/coco/hrnet_w48_coco_256x192.py \ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \ --img-root tests/data/coco/ \ --json-file tests/data/coco/test_coco.json\ --out-img-root vis_results
-
若使用mmdet检测结果作为输入,我们提供了mmdet检测人的代码
python demo/top_down_img_demo_with_mmdet.py \ ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \ --img-root ${IMG_ROOT} --img ${IMG_FILE} \ --out-img-root ${OUTPUT_DIR} \ [--show --device ${GPU_ID}] \ [--bbox-thr ${BBOX_SCORE_THR} --kpt-thr ${KPT_SCORE_THR}]
如
python demo/topdown_demo_with_mmdet.py mmdetection/configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py \ mmdetection/checkpoints/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \ configs/top_down/hrnet/coco/hrnet_w48_coco_256x192.py \ hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \ --img-root tests/data/coco/ \ --img 000000196141.jpg \ --out-img-root vis_results\ --show
-
若没有安装mmdet话,只能测试他们提供的图片
下面我们以coco数据集为例,测试玩一下这个库,他们提供了已经标注好的jsonpython demo/top_down_img_demo.py \ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py\ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth\ --img-root tests/data/coco/\ --json-file tests/data/coco/test_coco.json\ --out-img-root vis_results
-
如果你安装了mmdet话,任意一张图片都可以预测
直接使用以下代码,mmdet可以自己检测到人,不需要在手动标注了-
预测图片
python demo/topdown_demo_with_mmdet.py \ demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \ https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py \ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth\ --input tests/data/111/000000000785.jpg \ --output-root vis_results \ --draw-heatmap
-
预测视频
python demo/top_down_video_demo_with_mmdet.py \ demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py\ https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \ configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py \ https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth\ --video-path demo/resources/1.mp4 \ --out-video-root vis_results
-
2.Project
2D 动物图片姿态识别推理
python demo/topdown_demo_with_mmdet.py \
${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
--input ${INPUT_PATH} --det-cat-id ${DET_CAT_ID} \
[--show] [--output-root ${OUTPUT_DIR}] [--save-predictions] \
[--draw-heatmap ${DRAW_HEATMAP}] [--radius ${KPT_RADIUS}] \
[--kpt-thr ${KPT_SCORE_THR}] [--bbox-thr ${BBOX_SCORE_THR}] \
[--device ${GPU_ID or CPU}]
python demo/topdown_demo_with_mmdet.py \
demo/mmdetection_cfg/rtmdet_m_8xb32-300e_coco.py \
https://download.openmmlab.com/mmdetection/v3.0/rtmdet/rtmdet_m_8xb32-300e_coco/rtmdet_m_8xb32-300e_coco_20220719_112220-229f527c.pth \
configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
--input tests/data/animalpose/ca110.jpeg \
--show --draw-heatmap --det-cat-id=15
如果使用了 heatmap-based 模型同时设置了 --draw-heatmap
,预测的热图也会跟随关键点一同可视化出来。
--det-cat-id=15
参数用来指定模型只检测 cat
类型,这是基于 COCO 数据集的数据。
2D 动物视频姿态识别推理
python demo/topdown_demo_with_mmdet.py \
demo/mmdetection_cfg/rtmdet_m_8xb32-300e_coco.py \
https://download.openmmlab.com/mmdetection/v3.0/rtmdet/rtmdet_m_8xb32-300e_coco/rtmdet_m_8xb32-300e_coco_20220719_112220-229f527c.pth \
configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
--input demo/resources/demo_dog.mp4 \
--output-root vis_results \
--draw-heatmap \
--det-cat-id=16
使用 Inferencer 进行 2D 动物姿态识别推理
Inferencer 提供一个更便捷的推理接口,使得用户可以绕过模型的配置文件和 checkpoint 路径直接使用 model aliases ,支持包括图片路径、视频路径、图片文件夹路径和 webcams 在内的多种输入方式,例如可以这样使用:
python demo/inferencer_demo.py \
tests/data/ap10k \
--pose2d animal\
--vis-out-dir vis_results/ap10k
2D 人体姿态 Top-Down 图片检测
使用整张图片作为输入进行检测
(此时输入的整张图片会被当作 bounding box 使用)
python demo/image_demo.py \
${IMG_FILE} ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
--out-file ${OUTPUT_FILE} \
[--device ${GPU_ID or CPU}] \
[--draw_heatmap]
如:
python demo/image_demo.py \
tests/data/coco/000000000785.jpg \
configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w48_8xb32-210e_coco-256x192.py \
https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \
--out-file vis_results.jpg \
--draw-heatmap
使用 CPU 推理:
python demo/image_demo.py \
tests/data/coco/000000000785.jpg \
configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w48_8xb32-210e_coco-256x192.py \
https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \
--out-file vis_results.jpg \
--draw-heatmap \
--device=cpu
使用 MMDet 做人体 bounding box 检测
python demo/topdown_demo_with_mmdet.py \
${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
--input ${INPUT_PATH} \
[--output-root ${OUTPUT_DIR}] [--save-predictions] \
[--show] [--draw-heatmap] [--device ${GPU_ID or CPU}] \
[--bbox-thr ${BBOX_SCORE_THR}] [--kpt-thr ${KPT_SCORE_THR}]
python demo/topdown_demo_with_mmdet.py \
demo/mmdetection_cfg/rtmdet_m_640-8xb32_coco-person.py \
https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth \
configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-420e_body8-256x192.py \
https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth \
--input tests/data/111/000000197388.jpg \
--draw-heatmap \
--output-root vis_results/1.jpg
python demo/inferencer_demo.py \
tests/data/111 \
--pose2d human\
--vis-out-dir vis_results/111
2D 人体全身姿态 Top-Down 图片识别
使用整张图片作为输入进行检测
此时输入的整张图片会被当作 bounding box 使用。
python demo/image_demo.py \
${IMG_FILE} ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
--out-file ${OUTPUT_FILE} \
[--device ${GPU_ID or CPU}] \
[--draw_heatmap]
python demo/image_demo.py \
tests/data/coco/000000000785.jpg \
configs/wholebody_2d_keypoint/topdown_heatmap/coco-wholebody/td-hm_vipnas-res50_dark-8xb64-210e_coco-wholebody-256x192.py \
https://download.openmmlab.com/mmpose/top_down/vipnas/vipnas_res50_wholebody_256x192_dark-67c0ce35_20211112.pth \
--out-file vis_results_1.jpg\
--draw-heatmap
使用 MMDet 进行人体 bounding box 检测
python demo/topdown_demo_with_mmdet.py \
${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
--input ${INPUT_PATH} \
[--output-root ${OUTPUT_DIR}] [--save-predictions] \
[--show] [--draw-heatmap] [--device ${GPU_ID or CPU}] \
[--bbox-thr ${BBOX_SCORE_THR}] [--kpt-thr ${KPT_SCORE_THR}]
python demo/topdown_demo_with_mmdet.py \
demo/mmdetection_cfg/rtmdet_m_640-8xb32_coco-person.py \
https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth \
configs/wholebody_2d_keypoint/topdown_heatmap/coco-wholebody/td-hm_hrnet-w48_dark-8xb32-210e_coco-wholebody-384x288.py \
https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_wholebody_384x288_dark-f5726563_20200918.pth \
--input tests/data/coco/000000196141.jpg \
--output-root vis_results/
--show
训练自己的模型
步骤:
-
文件夹规整 data/annotations data/images
-
配置文件 configs//body_2d_keypoint/rtmpose/mpii configs/base/datasets/mpii
-
模型训练
先使用mmdet进行检测训练,再使用mmpose进行姿态估计训练
# 1.训练mmdet CUDA_VISIBLE_DEVICE=0,1 PORT=29501 mmdetection/tools/dist_train.sh {configs} 2 # 2.训练mmpose CUDA_VISIBLE_DEVICE=0,1 PORT=29501 mmpose/tools/dist_train.sh {configs} 2
不使用mmdet,推理时直接使用预训练模型
CUDA_VISIBLE_DEVICE=0,1 PORT=29501 mmpose/tools/dist_train.sh {configs} 2
-
模型精简 (模型本身不变,只是删除不必要的数据,节省空间)
python tools/misc/publish_model.py \ work_dirs/rtmpose-m_8xb64-210e_mpii-256x256/best_PCK_epoch_210.pth \ checkpoint/rtmpose-m_mpii_epoch_210.pth
-
模型推理 (可以用自己训练的det模型,也可用pose自带的det模型)
python demo/topdown_demo_with_mmdet.py \ demo/mmdetection_cfg/rtmdet_m_640-8xb32_coco-person.py \ https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth \ configs/body_2d_keypoint/rtmpose/mpii/rtmpose-m_8xb64-210e_mpii-256x256.py \ work_dirs/rtmpose-m_8xb64-210e_mpii-256x256/best_PCK_epoch_210.pth \ --input tests/data/mpii/004645041.jpg \ --draw-heatmap \ --output-root vis_results/