slowfast训练

数据集介绍
https://github.com/facebookresearch/SlowFast/blob/main/projects/pytorchvideo/README.md
PyTorchVideo数据集包括
Kinetics
Kinetics数据集下载 https://www.deepmind.com/open-source/kinetics
Charades
Something-something v2
三个数据集

1.首先安装完环境

2.然后准备数据集

https://github.com/facebookresearch/SlowFast/blob/main/slowfast/datasets/DATASET.md

然后运行

python tools/run_net.py   --cfg configs/Kinetics/C2D_8x8_R50.yaml   DATA.PATH_TO_DATA_DIR /ActivityNet/Crawler/Kinetics/data/kinetics-400_train.csv   NUM_GPUS 1   TRAIN.BATCH_SIZE 16

这里我们尝试AVA数据集进行训练
首先要准备数据集

https://github.com/facebookresearch/SlowFast/blob/main/slowfast/datasets/DATASET.md

2.1然后运行脚本下载

（数据在./data/ava/videos下）

DATA_DIR="./data/ava/videos"

if [[ ! -d "${DATA_DIR}" ]]; then
  echo "${DATA_DIR} doesn't exist. Creating it.";
  mkdir -p ${DATA_DIR}
fi

wget https://s3.amazonaws.com/ava-dataset/annotations/ava_file_names_trainval_v2.1.txt

for line in $(cat ava_file_names_trainval_v2.1.txt)
do
  wget https://s3.amazonaws.com/ava-dataset/trainval/$line -P ${DATA_DIR}
done

如果运行过程中报错

[[: not found

是因为sh命令无法识别"[[]]"表达式。改为用./ 来替代sh执行
2.2裁剪每个视频的第15分钟到30分钟

IN_DATA_DIR="./data/ava/videos"
OUT_DATA_DIR="./data/ava/videos_15min"

if [[ ! -d "${OUT_DATA_DIR}" ]]; then
  echo "${OUT_DATA_DIR} doesn't exist. Creating it.";
  mkdir -p ${OUT_DATA_DIR}
fi

for video in $(ls -A1 -U ${IN_DATA_DIR}/*)
do
  out_name="${OUT_DATA_DIR}/${video##*/}"
  if [ ! -f "${out_name}" ]; then
    ffmpeg -ss 900 -t 901 -i "${video}" "${out_name}"
  fi
done

2.3将视频提取为图片帧

IN_DATA_DIR="./data/ava/videos_15min"
OUT_DATA_DIR="./data/ava/frames"

if [[ ! -d "${OUT_DATA_DIR}" ]]; then
  echo "${OUT_DATA_DIR} doesn't exist. Creating it.";
  mkdir -p ${OUT_DATA_DIR}
fi

for video in $(ls -A1 -U ${IN_DATA_DIR}/*)
do
  video_name=${video##*/}

  if [[ $video_name = *".webm" ]]; then
    video_name=${video_name::-5}
  else
    video_name=${video_name::-4}
  fi

  out_video_dir=${OUT_DATA_DIR}/${video_name}/
  mkdir -p "${out_video_dir}"

  out_name="${out_video_dir}/${video_name}_%06d.jpg"

  ffmpeg -i "${video}" -r 30 -q:v 1 "${out_name}"
done

2.4下载annotations

DATA_DIR="./data/ava/annotations"

if [[ ! -d "${DATA_DIR}" ]]; then
  echo "${DATA_DIR} doesn't exist. Creating it.";
  mkdir -p ${DATA_DIR}
fi

wget https://research.google.com/ava/download/ava_train_v2.1.csv -P ${DATA_DIR}
wget https://research.google.com/ava/download/ava_val_v2.1.csv -P ${DATA_DIR}
wget https://research.google.com/ava/download/ava_action_list_v2.1_for_activitynet_2018.pbtxt -P ${DATA_DIR}
wget https://research.google.com/ava/download/ava_train_excluded_timestamps_v2.1.csv -P ${DATA_DIR}
wget https://research.google.com/ava/download/ava_val_excluded_timestamps_v2.1.csv -P ${DATA_DIR}

2.5新建frame_lits目录下载train.csv和val.csv

wget https://dl.fbaipublicfiles.com/video-long-term-feature-banks/data/ava/frame_lists/train.csv
wget https://dl.fbaipublicfiles.com/video-long-term-feature-banks/data/ava/frame_lists/val.csv

准备完数据之后运行
预训练模型记得下载

python tool/run_net.py --cfg configs/AVA/SLOWFAST_32x2_R50_SHORT5.yaml

如果报错

RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

将yaml修改为

NUM_GPUS: 1 #8

参考文献
https://mp.weixin.qq.com/s?__biz=MzI4MDYzNzg4Mw==&mid=2247553678&idx=5&sn=d556f7c9be6b48c3f3ab199b0c855c5b&chksm=ebb72a5adcc0a34c0ed14e9366055ce455ef9560f4b4ec9f1460d414be1c44ab1231534199bb&scene=27