一、前言
由于本业余人士水平有限,在复现过程中经历了种种报错,最终皇天不负有心人,我成功把demo跑出来了。源码地址:https://github.com/facebookresearch/SlowFast,但是由于官网一直有人维护,git下来的版本是最新代码,复现会出现各种问题。所以选择一个较老版本:参考https://blog.csdn.net/weixin_48841074/article/details/116807805
git clone https://gitee.com/qiang_sun/SlowFast.git
二、复现步骤
-
在AutoDL平台租用GPU(RTX3090*1,显存24GB,CPU内存80GB,系统盘25GB)。框架(Pytorch 1.8.1,Python 3.8(Ubuntu18.04),Cuda 11.1)。(RTX3090算力为8.6,cuda版本至少为11.x)**注意:**JupyterLab的工作目录为
/root
目录,而非/
系统根目录。
-
打开终端,安装依赖:
-
更新pip
pip install --upgrade pip
-
安装fvcore
# 在AutoDL平台上可以参考帮助文档使用学术资源加速
pip install 'git+https://github.com/facebookresearch/fvcore'
-
关闭加速
unset http_proxy && unset https_proxy
-
# 在实例AutoPanel手动切换pip软件源(首选阿里源,若慢则重选清华源)
-
安装simplejson
pip install simplejson
-
安装PyAv
pip install av==8.0.3
-
安装iopath
pip install -U iopath
-
安装psutil
pip install psutil
-
安装opencv-python
pip install opencv-python
-
安装tensorboard
pip install tensorboard
-
安装cython
pip install cython
-
下载detectron2,这里与官网方式不同。(要求pytorch>=1.8)参考 https://github.com/facebookresearch/detectron2/releases
# 在AutoDL平台上可以参考帮助文档使用学术资源加速
git clone https://github.com/facebookresearch/detectron2
-
关闭加速
unset http_proxy && unset https_proxy
-
安装detectron2
pip install -e detectron2
-
下载SlowFsast
git clone https://gitee.com/qiang_sun/SlowFast.git
-
导入路径,路径如下图所示,修改成你自己的路径即可
export PYTHONPATH=/root/SlowFast/slowfast:$PYTHONPATH
-
安装pandas和scikit-learn,有可能上面的过程引起了删除,我没去找具体原因
pip install pandas pip install scikit-learn
-
进入SlowFast目录
cd SlowFast
-
编译,可以干点别的事等着,非常慢
-
python setup.py build develop
-
4. 修改部分代码
-
在/SlowFast/demo/AVA目录下新建ava.json,文件内容如下:
{"bend/bow (at the waist)": 0, "crawl": 1, "crouch/kneel": 2, "dance": 3, "fall down": 4, "get up": 5, "jump/leap": 6, "lie/sleep": 7, "martial art": 8, "run/jog": 9, "sit": 10, "stand": 11, "swim": 12, "walk": 13, "answer phone": 14, "brush teeth": 15, "carry/hold (an object)": 16, "catch (an object)": 17, "chop": 18, "climb (e.g., a mountain)": 19, "clink glass": 20, "close (e.g., a door, a box)": 21, "cook": 22, "cut": 23, "dig": 24, "dress/put on clothing": 25, "drink": 26, "drive (e.g., a car, a truck)": 27, "eat": 28, "enter": 29, "exit": 30, "extract": 31, "fishing": 32, "hit (an object)": 33, "kick (an object)": 34, "lift/pick up": 35, "listen (e.g., to music)": 36, "open (e.g., a window, a car door)": 37, "paint": 38, "play board game": 39, "play musical instrument": 40, "play with pets": 41, "point to (an object)": 42, "press": 43, "pull (an object)": 44, "push (an object)": 45, "put down": 46, "read": 47, "ride (e.g., a bike, a car, a horse)": 48, "row boat": 49, "sail boat": 50, "shoot": 51, "shovel": 52, "smoke": 53, "stir": 54, "take a photo": 55, "text on/look at a cellphone": 56, "throw": 57, "touch (an object)": 58, "turn (e.g., a screwdriver)": 59, "watch (e.g., TV)": 60, "work on a computer": 61, "write": 62, "fight/hit (a person)": 63, "give/serve (an object) to (a person)": 64, "grab (a person)": 65, "hand clap": 66, "hand shake": 67, "hand wave": 68, "hug (a person)": 69, "kick (a person)": 70, "kiss (a person)": 71, "lift (a person)": 72, "listen to (a person)": 73, "play with kids": 74, "push (another person)": 75, "sing to (e.g., self, a person, a group)": 76, "take (an object) from (a person)": 77, "talk to (e.g., self, a person, a group)": 78, "watch (a person)": 79}
-
修改/SlowFast/demo/AVA/SLOWFAST_32x2_R101_50_50.yaml,内容改为如下:
TRAIN: ENABLE: False DATASET: ava BATCH_SIZE: 16 EVAL_PERIOD: 1 CHECKPOINT_PERIOD: 1 AUTO_RESUME: True CHECKPOINT_FILE_PATH: "/root/SlowFast/configs/AVA/c2/SLOWFAST_32x2_R101_50_50.pkl" #path to pretrain model CHECKPOINT_TYPE: pytorch DATA: NUM_FRAMES: 32 SAMPLING_RATE: 2 TRAIN_JITTER_SCALES: [256, 320] TRAIN_CROP_SIZE: 224 TEST_CROP_SIZE: 256 INPUT_CHANNEL_NUM: [3, 3] DETECTION: ENABLE: True ALIGNED: False AVA: BGR: False DETECTION_SCORE_THRESH: 0.8 TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"] SLOWFAST: ALPHA: 4 BETA_INV: 8 FUSION_CONV_CHANNEL_RATIO: 2 FUSION_KERNEL_SZ: 5 RESNET: ZERO_INIT_FINAL_BN: True WIDTH_PER_GROUP: 64 NUM_GROUPS: 1 DEPTH: 101 TRANS_FUNC: bottleneck_transform STRIDE_1X1: False NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]] SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]] SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]] NONLOCAL: LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]] GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]] INSTANTIATION: dot_product POOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]] BN: USE_PRECISE_STATS: False NUM_BATCHES_PRECISE: 200 SOLVER: MOMENTUM: 0.9 WEIGHT_DECAY: 1e-7 OPTIMIZING_METHOD: sgd MODEL: NUM_CLASSES: 80 ARCH: slowfast MODEL_NAME: SlowFast LOSS_FUNC: bce DROPOUT_RATE: 0.5 HEAD_ACT: sigmoid TEST: ENABLE: False DATASET: ava BATCH_SIZE: 8 DATA_LOADER: NUM_WORKERS: 2 PIN_MEMORY: True NUM_GPUS: 1 NUM_SHARDS: 1 RNG_SEED: 0 OUTPUT_DIR: . #TENSORBOARD: # MODEL_VIS: # TOPK: 2 DEMO: ENABLE: True LABEL_FILE_PATH: "/root/SlowFast/demo/AVA/ava.json" #WEBCAM: 0 #下面两个文件夹是自己建立的(VINPUT和VOUTPUT) INPUT_VIDEO: "/root/SlowFast/VINPUT/1.mp4" #自己上传一段测试视频 OUTPUT_FILE: "/root/SlowFast/VOUTPUT/2.mp4" DETECTRON2_CFG: "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml" DETECTRON2_WEIGHTS: https://dl.fbaipublicfiles.com/detectron2/COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl
-
运行代码
# 先进入到SlowFast文件夹
python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50.yaml
运行效果:
三、遇到的问题
1.安装最新detectron2(V0.6)需要保证pytorch>=1.8
2.安装PyAv要指定版本av==8.0.3