论文链接:3D human pose estimation in video with temporal convolutions and semi-supervised training
代码链接:https://github.com/facebookresearch/VideoPose3D
环境搭建:ubuntu20.04+pytorch1.8+cuda10.2+python3.6+anaconda
1.启动anaconda环境
新建环境命令:
conda create -n pytorch18cu102py36 python=3.6
安装pytorch:
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=10.2 -c pytorch
2.设置pycharm环境
3.下载先验模型
mkdir checkpoint
cd checkpoint
wget https://dl.fbaipublicfiles.com/video-pose-3d/pretrained_h36m_cpn.bin
wget https://dl.fbaipublicfiles.com/video-pose-3d/pretrained_humaneva15_detectron.bin
cd ..
4.数据集下载和测试:请参考https://blog.csdn.net/weixin_42001132/article/details/103437104
5.in the wild环境搭建
5.1 下载先验模型
cd checkpoint
wget https://dl.fbaipublicfiles.com/video-pose-3d/pretrained_h36m_detectron_coco.bin
5.2 视频预处理(这部分我没有做,我选择视频本来就比较短)
ffmpeg -i input.mp4 -ss 1:00 -to 1:30 -c copy output.mp4
5.3 在detectron2推断2d关键点
配置detectron2 根据环境配置https://detectron2.readthedocs.io/en/latest/tutorials/install.html
python -m pip install detectron2 -f \
https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.8/index.html
cd inference
python infer_video_d2.py \
--cfg COCO-Keypoints/keypoint_rcnn_R_101_FPN_3x.yaml \
--output-dir output_directory \
--image-ext mp4 \
input_directory
5.3 创建 a custom dataset
替换自己输出路径即可
python prepare_data_2d_custom.py -i /home/fll/文档/VideoPose3D-main/output_directory -o myvideos
5.4 生成视频
替换1.mp4和视频路径即可
python run.py -d custom -k myvideos -arc 3,3,3,3,3 -c checkpoint --evaluate pretrained_h36m_detectron_coco.bin --render --viz-subject 1.mp4 --viz-action custom --viz-camera 0 --viz-video dataset/1.mp4 --viz-output output.mp4 --viz-size 6