2023 安装 facebookresearch slowfast 自定义数据集训练 yolo数据集转ava数据集

已于 2023-11-09 16:14:18 修改
阅读量4k 收藏 49
点赞数 12
文章标签： slowfast 1024程序员节
于 2023-10-13 01:26:43 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/WhiffeYF/article/details/133801160
版权

目录

0 前言

facebookresearch SlowFast ：https://github.com/facebookresearch/SlowFast

b站视频：https://www.bilibili.com/video/BV1Uh4y1q7R2/

1 准备

平台：Autodl：https://www.autodl.com/
环境：
PyTorch 1.8.1
Python 3.8(ubuntu18.04)
Cuda 11.1

在这里插入图片描述
需要先将这两个权重下载到AI平台的：/root/slowfastFile

model_final_280758.pkl：https://dl.fbaipublicfiles.com/detectron2/COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl

SLOWFAST_32x2_R101_50_50.pkl：https://dl.fbaipublicfiles.com/pyslowfast/model_zoo/ava/SLOWFAST_32x2_R101_50_50.pkl

2 安装与运行

2.1 基础环境

pip install 'git+https://gitee.com/YFwinston/fvcore'

pip install simplejson

conda install av -c conda-forge -y

conda install x264 ffmpeg -c conda-forge -y

pip install -U iopath

pip install psutil

pip install opencv-python

// pip install torchvision

pip install tensorboard

pip install moviepy

pip install pytorchvideo

pip install 'git+https://gitee.com/YFwinston/fairscale'

2.2 detectron2_repo 安装

// pip install -U torch torchvision cython
pip install -U  cython

pip install -U 'git+https://gitee.com/YFwinston/fvcore.git' 'git+https://gitee.com/YFwinston/cocoapi.git#subdirectory=PythonAPI'

git clone https://gitee.com/YFwinston/detectron2 detectron2_repo

pip install typing-extensions==4.3.0

pip install -e detectron2_repo

pip install pillow
pip install pyyaml

pip install scipy
pip install pandas
pip install scikit-learn

2.3 slowfast 安装

git clone https://gitee.com/YFwinston/slowfast

export PYTHONPATH=/root/slowfast:$PYTHONPATH

cd slowfast
python setup.py build develop
cd /root


cd /root/slowfast/demo/AVA
cp /root/autodl-nas/slowfast/ava.json ./
cp /root/autodl-nas/slowfast/SLOWFAST_32x2_R101_50_50s.yaml ./
cd /root


cd /root

git clone https://gitee.com/YFwinston/pytorchvideo.git
cd pytorchvideo
pip install -e .
cd /root

2.4 相关文件

2.4.1 ava.json

然后在/root/slowfast/demo/AVA下面的ava.json写入：

{"bend/bow (at the waist)": 0, "crawl": 1, "crouch/kneel": 2, "dance": 3, "fall down": 4, "get up": 5, "jump/leap": 6, "lie/sleep": 7, "martial art": 8, "run/jog": 9, "sit": 10, "stand": 11, "swim": 12, "walk": 13, "answer phone": 14, "brush teeth": 15, "carry/hold (an object)": 16, "catch (an object)": 17, "chop": 18, "climb (e.g., a mountain)": 19, "clink glass": 20, "close (e.g., a door, a box)": 21, "cook": 22, "cut": 23, "dig": 24, "dress/put on clothing": 25, "drink": 26, "drive (e.g., a car, a truck)": 27, "eat": 28, "enter": 29, "exit": 30, "extract": 31, "fishing": 32, "hit (an object)": 33, "kick (an object)": 34, "lift/pick up": 35, "listen (e.g., to music)": 36, "open (e.g., a window, a car door)": 37, "paint": 38, "play board game": 39, "play musical instrument": 40, "play with pets": 41, "point to (an object)": 42, "press": 43, "pull (an object)": 44, "push (an object)": 45, "put down": 46, "read": 47, "ride (e.g., a bike, a car, a horse)": 48, "row boat": 49, "sail boat": 50, "shoot": 51, "shovel": 52, "smoke": 53, "stir": 54, "take a photo": 55, "text on/look at a cellphone": 56, "throw": 57, "touch (an object)": 58, "turn (e.g., a screwdriver)": 59, "watch (e.g., TV)": 60, "work on a computer": 61, "write": 62, "fight/hit (a person)": 63, "give/serve (an object) to (a person)": 64, "grab (a person)": 65, "hand clap": 66, "hand shake": 67, "hand wave": 68, "hug (a person)": 69, "kick (a person)": 70, "kiss (a person)": 71, "lift (a person)": 72, "listen to (a person)": 73, "play with kids": 74, "push (another person)": 75, "sing to (e.g., self, a person, a group)": 76, "take (an object) from (a person)": 77, "talk to (e.g., self, a person, a group)": 78, "watch (a person)": 79}

2.4.2 SLOWFAST_32x2_R101_50_50s.yaml

然后在/root/slowfast/demo/AVA下面的SLOWFAST_32x2_R101_50_50s.yaml 写入：

TRAIN:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 1
  EVAL_PERIOD: 1
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  CHECKPOINT_FILE_PATH: /root/autodl-nas/slowfast/SLOWFAST_32x2_R101_50_50.pkl  #path to pretrain model
  CHECKPOINT_TYPE: pytorch
DATA:
  NUM_FRAMES: 16
  SAMPLING_RATE: 2
  TRAIN_JITTER_SCALES: [256, 320]
  TRAIN_CROP_SIZE: 224
  TEST_CROP_SIZE: 256
  INPUT_CHANNEL_NUM: [3, 3]
DETECTION:
  ENABLE: True
  ALIGNED: False
AVA:
  BGR: False
  DETECTION_SCORE_THRESH: 0.8
  TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
SLOWFAST:
  ALPHA: 4
  BETA_INV: 8
  FUSION_CONV_CHANNEL_RATIO: 2
  FUSION_KERNEL_SZ: 5
RESNET:
  ZERO_INIT_FINAL_BN: True
  WIDTH_PER_GROUP: 64
  NUM_GROUPS: 1
  DEPTH: 101
  TRANS_FUNC: bottleneck_transform
  STRIDE_1X1: False
  NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
  SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]
  SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:
  LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]
  GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
  INSTANTIATION: dot_product
  POOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:
  USE_PRECISE_STATS: False
  NUM_BATCHES_PRECISE: 200
SOLVER:
  MOMENTUM: 0.9
  WEIGHT_DECAY: 1e-7
  OPTIMIZING_METHOD: sgd
MODEL:
  NUM_CLASSES: 80
  ARCH: slowfast
  MODEL_NAME: SlowFast
  LOSS_FUNC: bce
  DROPOUT_RATE: 0.5
  HEAD_ACT: sigmoid
TEST:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 1
DATA_LOADER:
  NUM_WORKERS: 1
  PIN_MEMORY: True
 
NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: .
#TENSORBOARD:
#  MODEL_VIS:
#    TOPK: 2
DEMO:
  ENABLE: True
  LABEL_FILE_PATH: "demo/AVA/ava.json" # Add local label file path here.
  INPUT_VIDEO: "/root/autodl-nas/slowfast/1.mp4"
  OUTPUT_FILE: "/root/autodl-nas/slowfast/1_1.mp4"
  DETECTRON2_CFG: "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"
  DETECTRON2_WEIGHTS: "/root/autodl-nas/slowfast/model_final_280758.pkl"

2.5 demo测试

在/home/slowfast/demo/中传入一个视频：1.mp4
执行：

cd /root/slowfast/
python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50s.yaml

在这里插入图片描述

2.6 相关报错

2.6.1 报错1

This maybe due to another process holding this lock file. If you are sure no other Matplotlib process is running, remove this file and try again.

2.6.1 解决1

pip install --upgrade matplotlib

2.6.2 报错2

ModuleNotFoundError: No module named ‘torch._six’

2.6.2 解决2

 vim /root/slowfast/slowfast/datasets/multigrid_helper.py

在这里插入图片描述

if TORCH_MAJOR >= 1 and TORCH_MINOR >= 8:
    _int_classes = int
else:
    int_classes = int
    _int_classes = int
    #from torch._six import int_classes as _int_classes

2.6.3 报错3

报错：

    ERROR: Command errored out with exit status 1:
     command: /root/miniconda3/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/root/detectron2_repo/setup.py'"'"'; __file__='"'"'/root/detectron2_repo/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-yoefyzdp
         cwd: /root/detectron2_repo/
    Complete output (13 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/root/detectron2_repo/setup.py", line 10, in <module>
        import torch
      File "/root/miniconda3/lib/python3.8/site-packages/torch/__init__.py", line 1429, in <module>
        from torch import optim as optim
      File "/root/miniconda3/lib/python3.8/site-packages/torch/optim/__init__.py", line 8, in <module>
        from .adadelta import Adadelta
      File "/root/miniconda3/lib/python3.8/site-packages/torch/optim/adadelta.py", line 4, in <module>
        from .optimizer import (Optimizer, _use_grad_for_differentiable, _default_to_fused_or_foreach,
      File "/root/miniconda3/lib/python3.8/site-packages/torch/optim/optimizer.py", line 23, in <module>
        from typing_extensions import ParamSpec, Self, TypeAlias
    ImportError: cannot import name 'Self' from 'typing_extensions' (/root/miniconda3/lib/python3.8/site-packages/typing_extensions.py)
    ----------------------------------------
WARNING: Discarding file:///root/detectron2_repo. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

在这里插入图片描述

2.6.3 解决3

pip install typing-extensions==4.3.0

2.6.4 报错4

Traceback (most recent call last):
  File "tools/run_net.py", line 6, in <module>
    from slowfast.utils.misc import launch_job
  File "/root/slowfast/slowfast/utils/misc.py", line 19, in <module>
    import slowfast.utils.logging as logging
  File "/root/slowfast/slowfast/utils/logging.py", line 15, in <module>
    import slowfast.utils.distributed as du
  File "/root/slowfast/slowfast/utils/distributed.py", line 12, in <module>
    from pytorchvideo.layers.distributed import (  # noqa
ImportError: cannot import name 'cat_all_gather' from 'pytorchvideo.layers.distributed' (/root/miniconda3/lib/python3.8/site-packages/pytorchvideo/layers/distributed.py)

在这里插入图片描述

2.6.4 解决4

cd /root

git clone https://gitee.com/YFwinston/pytorchvideo.git
cd pytorchvideo
pip install -e .
cd /root

3 yolo数据集转ava数据集

https://github.com/Whiffe/SCB-dataset/tree/main/yolo2ava

python yolo2ava.py --yolo_path /root/5k_HRW_yolo_Dataset --ava_path /root/autodl-tmp/SCB-ava-Dataset4

执行后的目录结构
SCB-ava-Dataset4
├── annotations
│ ├── ava_action_list_v2.2_for_activitynet_2019.pbtxt
│ ├── ava_included_timestamps_v2.2.txt
│ ├── ava_test_excluded_timestamps_v2.2.csv
│ ├── ava_train_excluded_timestamps_v2.2.csv
│ ├── ava_train_v2.2.csv
│ ├── ava_val_excluded_timestamps_v2.2.csv
│ ├── ava_val_v2.2.csv
│ └── person_box_67091280_iou90
│ ├── ava_detection_train_boxes_and_labels_include_negative_v2.2.csv
│ └── ava_detection_val_boxes_and_labels.csv
├── frame_lists
│ ├── train.csv
│ └── val.csv
└── frames
├── 0001001
│ ├── 0001001_000001.png
│ ├── 0001001_000002.png
│ ├── …
│
├── 0001002
│ ├── 0001002_000001.png
│ ├── 0001002_000002.png
│ ├── …

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述
在这里插入图片描述

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

在这里插入图片描述

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

可以用下面的命令查看当前路径的文件占了多少空间

du -sh

4 训练

3.1 训练的相关配置文件

3.1.1 SLOWFAST_32x2_R101_50_50s.yaml

configs/AVA/c2/SLOWFAST_32x2_R101_50_50s.yaml

TRAIN:
  ENABLE: True
  DATASET: ava
  BATCH_SIZE: 4
  EVAL_PERIOD: 1
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  CHECKPOINT_FILE_PATH: /root/autodl-nas/slowfast/SLOWFAST_32x2_R101_50_50.pkl 
  CHECKPOINT_TYPE: pytorch
DATA:
  NUM_FRAMES: 32
  SAMPLING_RATE: 2
  TRAIN_JITTER_SCALES: [256, 320]
  TRAIN_CROP_SIZE: 224
  TEST_CROP_SIZE: 256
  INPUT_CHANNEL_NUM: [3, 3]
  PATH_TO_DATA_DIR: '/root/autodl-tmp/SCB-ava-Dataset4'
DETECTION:
  ENABLE: True
  ALIGNED: False

AVA:
  FRAME_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/frames'
  FRAME_LIST_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/frame_lists'
  ANNOTATION_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/annotations'
  DETECTION_SCORE_THRESH: 0.8
  TRAIN_PREDICT_BOX_LISTS: [
    "ava_train_v2.2.csv",
    "person_box_67091280_iou90/ava_detection_train_boxes_and_labels_include_negative_v2.2.csv",
  ]
  TEST_PREDICT_BOX_LISTS: [
    "person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
  FULL_TEST_ON_VAL: True

SLOWFAST:
  ALPHA: 4
  BETA_INV: 8
  FUSION_CONV_CHANNEL_RATIO: 2
  FUSION_KERNEL_SZ: 5
RESNET:
  ZERO_INIT_FINAL_BN: True
  WIDTH_PER_GROUP: 64
  NUM_GROUPS: 1
  DEPTH: 101
  TRANS_FUNC: bottleneck_transform
  STRIDE_1X1: False
  NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
  SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]
  SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:
  LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]
  GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
  INSTANTIATION: dot_product
  POOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:
  USE_PRECISE_STATS: False
  NUM_BATCHES_PRECISE: 200
SOLVER:
  MOMENTUM: 0.9
  WEIGHT_DECAY: 1e-7
  OPTIMIZING_METHOD: sgd
MODEL:
  NUM_CLASSES: 80
  ARCH: slowfast
  MODEL_NAME: SlowFast
  LOSS_FUNC: bce
  DROPOUT_RATE: 0.5
  HEAD_ACT: sigmoid
TEST:
  ENABLE: True
  DATASET: ava
  BATCH_SIZE: 8
DATA_LOADER:
  NUM_WORKERS: 2
  PIN_MEMORY: True
NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: '/root/autodl-tmp/'

3.1.2 ava_action_list_v2.2_for_activitynet_2019.pbtxt

SCB-ava-Dataset4/annotations/ava_action_list_v2.2_for_activitynet_2019.pbtxt

item {
  name: "hand-raising"
  id: 1
}
item {
  name: "reading"
  id: 2
}
item {
  name: "writing"
  id: 3
}

3.1.3 ava_included_timestamps_v2.2.txt

SCB-ava-Dataset4/annotations/ava_included_timestamps_v2.2.txt

0002

3.1.4 ava_train/test_excluded_timestamps_v2.2.csv

空
SCB-ava-Dataset4/annotations/ava_test_excluded_timestamps_v2.2.csv
SCB-ava-Dataset4/annotations/ava_train_excluded_timestamps_v2.2.csv

ava_helper.py

ava默认是从902帧开始，我们可以从第1帧开始，不过需要修改代码，位置在slowfast/slowfast/datasets/ava_helper.py，
AVA_VALID_FRAMES = range(2, 3)
在这里插入图片描述

3.2 训练

python tools/run_net.py \
  --cfg configs/AVA/c2/SLOWFAST_32x2_R101_50_50s.yaml

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述
在这里插入图片描述

3.3 相关报错

3.3.1 报错1

报错：
Traceback (most recent call last):
File “tools/run_net.py”, line 52, in
main()
File “tools/run_net.py”, line 27, in main
launch_job(cfg=cfg, init_method=args.init_method, func=train)
File “/root/slowfast/slowfast/utils/misc.py”, line 430, in launch_job
func(cfg=cfg)
File “/root/slowfast/tools/train_net.py”, line 681, in train
train_epoch(
File “/root/slowfast/tools/train_net.py”, line 276, in train_epoch
del inputs
UnboundLocalError: local variable ‘inputs’ referenced before assignment
在这里插入图片描述

3.3.1 解决1

解决：
https://github.com/facebookresearch/SlowFast/issues/547
在这里插入图片描述
更改batch_size的大小，报错是因为batch_size大于视频数量

3.3.2 报错2

报错：
Traceback (most recent call last):
File “tools/run_net.py”, line 52, in
main()
File “tools/run_net.py”, line 27, in main
launch_job(cfg=cfg, init_method=args.init_method, func=train)
File “/root/slowfast/slowfast/utils/misc.py”, line 430, in launch_job
func(cfg=cfg)
File “/root/slowfast/tools/train_net.py”, line 681, in train
train_epoch(
File “/root/slowfast/tools/train_net.py”, line 110, in train_epoch
optim.set_lr(optimizer, lr)
File “/root/slowfast/slowfast/models/optimizer.py”, line 267, in set_lr
param_group[“lr”] = new_lr * param_group[“layer_decay”]
KeyError: ‘layer_decay’

在这里插入图片描述

3.3.2 解决2

解决：
https://github.com/facebookresearch/SlowFast/issues/654

slowfast/slowfast/models/optimizer.py
在这里插入图片描述


def set_lr(optimizer, new_lr):
    """
    Sets the optimizer lr to the specified value.
    Args:
        optimizer (optim): the optimizer using to optimize the current network.
        new_lr (float): the new learning rate to set.
    """
    for param_group in optimizer.param_groups:
        try:
            param_group["lr"] = new_lr * param_group["layer_decay"]
        except:
            param_group["lr"] = new_lr * 1.0

3.3.3 报错3

错误：

/root/slowfast/ava_evaluation/metrics.py:41: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.
  or labels.dtype != np.bool
Traceback (most recent call last):
  File "tools/run_net.py", line 52, in <module>
    main()
  File "tools/run_net.py", line 27, in main
    launch_job(cfg=cfg, init_method=args.init_method, func=train)
  File "/root/slowfast/slowfast/utils/misc.py", line 430, in launch_job
    func(cfg=cfg)
  File "/root/slowfast/tools/train_net.py", line 748, in train
    eval_epoch(
  File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/root/slowfast/tools/train_net.py", line 420, in eval_epoch
    val_meter.log_epoch_stats(cur_epoch)
  File "/root/slowfast/slowfast/utils/meters.py", line 235, in log_epoch_stats
    self.finalize_metrics(log=False)
  File "/root/slowfast/slowfast/utils/meters.py", line 208, in finalize_metrics
    self.full_map = evaluate_ava(
  File "/root/slowfast/slowfast/utils/ava_eval_helper.py", line 162, in evaluate_ava
    results = run_evaluation(categories, groundtruth, detections, excluded_keys)
  File "/root/slowfast/slowfast/utils/ava_eval_helper.py", line 238, in run_evaluation
    metrics = pascal_evaluator.evaluate()
  File "/root/slowfast/ava_evaluation/object_detection_evaluation.py", line 307, in evaluate
    ) = self._evaluation.evaluate()
  File "/root/slowfast/ava_evaluation/object_detection_evaluation.py", line 799, in evaluate
    precision, recall = metrics.compute_precision_recall(
  File "/root/slowfast/ava_evaluation/metrics.py", line 41, in compute_precision_recall
    or labels.dtype != np.bool
  File "/root/miniconda3/lib/python3.8/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'bool'.
`np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

在这里插入图片描述

3.3.3 解决3

成功解决AttributeError: module ‘numpy‘ has no attribute ‘bool‘.
pip uninstall numpy

pip install numpy==1.23.2

报错：

Traceback (most recent call last):
  File "tools/run_net.py", line 52, in <module>
    main()
  File "tools/run_net.py", line 27, in main
    launch_job(cfg=cfg, init_method=args.init_method, func=train)
  File "/root/slowfast/slowfast/utils/misc.py", line 430, in launch_job
    func(cfg=cfg)
  File "/root/slowfast/tools/train_net.py", line 748, in train
    eval_epoch(
  File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/root/slowfast/tools/train_net.py", line 420, in eval_epoch
    val_meter.log_epoch_stats(cur_epoch)
  File "/root/slowfast/slowfast/utils/meters.py", line 235, in log_epoch_stats
    self.finalize_metrics(log=False)
  File "/root/slowfast/slowfast/utils/meters.py", line 208, in finalize_metrics
    self.full_map = evaluate_ava(
  File "/root/slowfast/slowfast/utils/ava_eval_helper.py", line 162, in evaluate_ava
    results = run_evaluation(categories, groundtruth, detections, excluded_keys)
  File "/root/slowfast/slowfast/utils/ava_eval_helper.py", line 221, in run_evaluation
    pascal_evaluator.add_single_detected_image_info(
  File "/root/slowfast/ava_evaluation/object_detection_evaluation.py", line 275, in add_single_detected_image_info
    self._evaluation.add_single_detected_image_info(
  File "/root/slowfast/ava_evaluation/object_detection_evaluation.py", line 710, in add_single_detected_image_info
    ) = self.per_image_eval.compute_object_detection_metrics(
  File "/root/slowfast/ava_evaluation/per_image_evaluation.py", line 114, in compute_object_detection_metrics
    scores, tp_fp_labels = self._compute_tp_fp(
  File "/root/slowfast/ava_evaluation/per_image_evaluation.py", line 210, in _compute_tp_fp
    scores, tp_fp_labels = self._compute_tp_fp_for_single_class(
  File "/root/slowfast/ava_evaluation/per_image_evaluation.py", line 310, in _compute_tp_fp_for_single_class
    ) = self._get_overlaps_and_scores_box_mode(
  File "/root/slowfast/ava_evaluation/per_image_evaluation.py", line 253, in _get_overlaps_and_scores_box_mode
    gt_non_group_of_boxlist = np_box_list.BoxList(
  File "/root/slowfast/ava_evaluation/np_box_list.py", line 58, in __init__
    raise ValueError(
ValueError: Invalid box data. data must be a numpy array of N*[y_min, x_min, y_max, x_max]

在这里插入图片描述

3.3.4 问题4

Traceback (most recent call last):
  File "tools/run_net.py", line 52, in <module>
    main()
  File "tools/run_net.py", line 27, in main
    launch_job(cfg=cfg, init_method=args.init_method, func=train)
  File "/data/SCB/slowfast/slowfast/utils/misc.py", line 430, in launch_job
    func(cfg=cfg)
  File "/data/SCB/slowfast/tools/train_net.py", line 681, in train
    train_epoch(
  File "/data/SCB/slowfast/tools/train_net.py", line 176, in train_epoch
    scaler.step(optimizer)
  File "/root/miniconda3/envs/slowfast/lib/python3.8/site-packages/torch/cuda/amp/grad_scaler.py", line 315, in step
    return optimizer.step(*args, **kwargs)
  File "/root/miniconda3/envs/slowfast/lib/python3.8/site-packages/torch/optim/optimizer.py", line 280, in wrapper
    out = func(*args, **kwargs)
  File "/root/miniconda3/envs/slowfast/lib/python3.8/site-packages/torch/optim/optimizer.py", line 33, in _use_grad
    ret = func(self, *args, **kwargs)
  File "/root/miniconda3/envs/slowfast/lib/python3.8/site-packages/torch/optim/sgd.py", line 76, in step
    sgd(params_with_grad,
  File "/root/miniconda3/envs/slowfast/lib/python3.8/site-packages/torch/optim/sgd.py", line 222, in sgd
    func(params,
  File "/root/miniconda3/envs/slowfast/lib/python3.8/site-packages/torch/optim/sgd.py", line 306, in _multi_tensor_sgd
    torch._foreach_add_(bufs, device_grads, alpha=1 - dampening)
RuntimeError: The size of tensor a (80) must match the size of tensor b (3) at non-singleton dimension 0

在这里插入图片描述

解决：

这里报错的原因是预训练模型的原因，不应该加载预训练模型，注释TRAIN.CHECKPOINT_FILE_PATH即可
在这里插入图片描述

4 测试

4.1 配置文件

slowfast/configs/AVA/c2/SLOWFAST_32x2_R101_50_50s2.yaml

TRAIN:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 4
  EVAL_PERIOD: 1
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  CHECKPOINT_FILE_PATH: /root/autodl-nas/slowfast/SLOWFAST_32x2_R101_50_50.pkl 
  CHECKPOINT_TYPE: pytorch

DATA:
  NUM_FRAMES: 32
  SAMPLING_RATE: 2
  TRAIN_JITTER_SCALES: [256, 320]
  TRAIN_CROP_SIZE: 224
  TEST_CROP_SIZE: 256
  INPUT_CHANNEL_NUM: [3, 3]
  PATH_TO_DATA_DIR: '/root/autodl-tmp/SCB-ava-Dataset4'
DETECTION:
  ENABLE: True
  ALIGNED: False

AVA:
  FRAME_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/frames'
  FRAME_LIST_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/frame_lists'
  ANNOTATION_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/annotations'
  DETECTION_SCORE_THRESH: 0.8
  TRAIN_PREDICT_BOX_LISTS: [
    "ava_train_v2.2.csv",
    "person_box_67091280_iou90/ava_detection_train_boxes_and_labels_include_negative_v2.2.csv",
  ]
  TEST_PREDICT_BOX_LISTS: [
    "person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
  FULL_TEST_ON_VAL: True

SLOWFAST:
  ALPHA: 4
  BETA_INV: 8
  FUSION_CONV_CHANNEL_RATIO: 2
  FUSION_KERNEL_SZ: 5
RESNET:
  ZERO_INIT_FINAL_BN: True
  WIDTH_PER_GROUP: 64
  NUM_GROUPS: 1
  DEPTH: 101
  TRANS_FUNC: bottleneck_transform
  STRIDE_1X1: False
  NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
  SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]
  SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:
  LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]
  GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
  INSTANTIATION: dot_product
  POOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:
  USE_PRECISE_STATS: False
  NUM_BATCHES_PRECISE: 200
SOLVER:
  MOMENTUM: 0.9
  WEIGHT_DECAY: 1e-7
  OPTIMIZING_METHOD: sgd
MODEL:
  NUM_CLASSES: 80
  ARCH: slowfast
  MODEL_NAME: SlowFast
  LOSS_FUNC: bce
  DROPOUT_RATE: 0.5
  HEAD_ACT: sigmoid
TEST:
  ENABLE: True
  DATASET: ava
  BATCH_SIZE: 1
  CHECKPOINT_FILE_PATH: /root/autodl-tmp/slowfast_train/checkpoints/checkpoint_epoch_00034.pyth
DATA_LOADER:
  NUM_WORKERS: 2
  PIN_MEMORY: True
NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: '/root/autodl-tmp/'

4.2 测试

python tools/run_net.py \
  --cfg configs/AVA/c2/SLOWFAST_32x2_R101_50_50s2.yaml  \

在这里插入图片描述

错误：

Traceback (most recent call last):
  File "tools/run_net.py", line 52, in <module>
    main()
  File "tools/run_net.py", line 37, in main
    launch_job(cfg=cfg, init_method=args.init_method, func=test)
  File "/root/slowfast/slowfast/utils/misc.py", line 430, in launch_job
    func(cfg=cfg)
  File "/root/slowfast/tools/test_net.py", line 252, in test
    test_meter = perform_test(test_loader, model, test_meter, cfg, writer)
  File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/root/slowfast/tools/test_net.py", line 129, in perform_test
    test_meter.iter_toc()
  File "/root/slowfast/slowfast/utils/meters.py", line 159, in iter_toc
    self.iter_timer.pause()
  File "/root/miniconda3/lib/python3.8/site-packages/fvcore/common/timer.py", line 30, in pause
    raise ValueError("Trying to pause a Timer that is already paused!")
ValueError: Trying to pause a Timer that is already paused!

在这里插入图片描述

解决：
https://github.com/facebookresearch/SlowFast/issues/599#ref-issue-1441689338
https://github.com/facebookresearch/SlowFast/issues/623

修改test_net.py中的内容，加两组try except

在这里插入图片描述


    if cfg.NUM_GPUS > 1:
        preds, labels, video_idx = du.all_gather([preds, labels, video_idx])
    if cfg.NUM_GPUS:
        preds = preds.cpu()
        labels = labels.cpu()
        video_idx = video_idx.cpu()

    try:
        test_meter.iter_toc()
    except:
        pass
    if not cfg.VIS_MASK.ENABLE:
        # Update and log stats.
        test_meter.update_stats(
            preds.detach(), labels.detach(), video_idx.detach()
        )
    try:
        test_meter.log_iter_stats(cur_iter)
    except:
        test_meter.log_iter_stats(None,cur_iter)

    test_meter.iter_tic()

报错：

Traceback (most recent call last):
  File "tools/run_net.py", line 52, in <module>
    main()
  File "tools/run_net.py", line 37, in main
    launch_job(cfg=cfg, init_method=args.init_method, func=test)
  File "/root/slowfast/slowfast/utils/misc.py", line 430, in launch_job
    func(cfg=cfg)
  File "/root/slowfast/tools/test_net.py", line 257, in test
    test_meter = perform_test(test_loader, model, test_meter, cfg, writer)
  File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/root/slowfast/tools/test_net.py", line 166, in perform_test
    test_meter.finalize_metrics()
  File "/root/slowfast/slowfast/utils/meters.py", line 200, in finalize_metrics
    all_ori_boxes = torch.cat(self.all_ori_boxes, dim=0)
RuntimeError: Sizes of tensors must match except in dimension 0. Got 5 and 80 in dimension 1 (The offending index is 1)

在这里插入图片描述
这个错误是在运行PyTorch代码时出现的，它指的是你在进行张量（tensor）操作时，试图将两个维度（dimension）大小不匹配的张量进行相加或其他操作。

具体来说，错误消息"Sizes of tensors must match except in dimension 0. Got 5 and 80 in dimension 1 (The offending index is 1)"告诉我们，你正在试图将一个在第一个维度大小为5的张量和第一个维度大小为80的张量进行相加。这是不允许的，因为在除第一个维度之外的所有维度上，两个输入张量的大小必须完全相同。

为了解决这个问题，你需要确保你正在操作的两个张量在所有维度上的大小都相同，或者至少在需要进行操作的维度上相同。

例如，如果你有两个大小为(5, 80)和(80, 80)的张量，你不能直接将它们相加，因为它们的第二个维度大小不匹配。你需要确保它们的大小相同，比如通过调整你的数据或者改变你的操作。

如果你能提供更多关于你遇到这个错误的上下文信息（比如你的代码片段和你的数据），我可能能提供更具体的建议。


Traceback (most recent call last):
  File "tools/run_net.py", line 52, in <module>
    main()
  File "tools/run_net.py", line 37, in main
    launch_job(cfg=cfg, init_method=args.init_method, func=test)
  File "/data/SCB/slowfast/slowfast/utils/misc.py", line 430, in launch_job
    func(cfg=cfg)
  File "/data/SCB/slowfast/tools/test_net.py", line 262, in test
    test_meter = perform_test(test_loader, model, test_meter, cfg, writer)
  File "/root/miniconda3/envs/slowfast/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/data/SCB/slowfast/tools/test_net.py", line 171, in perform_test
    test_meter.finalize_metrics()
  File "/data/SCB/slowfast/slowfast/utils/meters.py", line 204, in finalize_metrics
    all_ori_boxes = torch.cat(self.all_ori_boxes, dim=0)
RuntimeError: Sizes of tensors must match except in dimension 0. Expected size 5 but got size 3 for tensor number 1 in the list.

解决：

报错的原因如下：
维度不匹配，
下面内容导致了维度不匹配（test_net.py）

        if not cfg.VIS_MASK.ENABLE:
            # Update and log stats.
            # print("labels.detach():",labels.detach())
            test_meter.update_stats(
                preds.detach(), labels.detach(), video_idx.detach()

在这里插入图片描述

需要添加配置文件SLOWFAST_32x2_R101_50_50s.yaml中添加如下东西
在这里插入图片描述

VIS_MASK:
  ENABLE: True

4.2 demo测试

4.2.1 配置文件

slowfast/demo/AVA/SLOWFAST_32x2_R101_50_50s2.yaml

TRAIN:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 1
  EVAL_PERIOD: 1
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  CHECKPOINT_FILE_PATH: /root/slowfast_train/checkpoints/checkpoint_epoch_00034.pyth  #path to pretrain model
  CHECKPOINT_TYPE: pytorch
DATA:
  NUM_FRAMES: 16
  SAMPLING_RATE: 2
  TRAIN_JITTER_SCALES: [256, 320]
  TRAIN_CROP_SIZE: 224
  TEST_CROP_SIZE: 256
  INPUT_CHANNEL_NUM: [3, 3]
DETECTION:
  ENABLE: True
  ALIGNED: False
AVA:
  BGR: False
  DETECTION_SCORE_THRESH: 0.8
  TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
SLOWFAST:
  ALPHA: 4
  BETA_INV: 8
  FUSION_CONV_CHANNEL_RATIO: 2
  FUSION_KERNEL_SZ: 5
RESNET:
  ZERO_INIT_FINAL_BN: True
  WIDTH_PER_GROUP: 64
  NUM_GROUPS: 1
  DEPTH: 101
  TRANS_FUNC: bottleneck_transform
  STRIDE_1X1: False
  NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
  SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]
  SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:
  LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]
  GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
  INSTANTIATION: dot_product
  POOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:
  USE_PRECISE_STATS: False
  NUM_BATCHES_PRECISE: 200
SOLVER:
  MOMENTUM: 0.9
  WEIGHT_DECAY: 1e-7
  OPTIMIZING_METHOD: sgd
MODEL:
  NUM_CLASSES: 80
  ARCH: slowfast
  MODEL_NAME: SlowFast
  LOSS_FUNC: bce
  DROPOUT_RATE: 0.5
  HEAD_ACT: sigmoid
TEST:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 1
DATA_LOADER:
  NUM_WORKERS: 1
  PIN_MEMORY: True
 
NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: .
#TENSORBOARD:
#  MODEL_VIS:
#    TOPK: 2
DEMO:
  ENABLE: True
  LABEL_FILE_PATH: "demo/AVA/student_behaviors.json" # Add local label file path here.
  INPUT_VIDEO: "/root/autodl-tmp/0_0_0.mp4"
  OUTPUT_FILE: "/root/autodl-tmp/0_1_1.mp4"
  DETECTRON2_CFG: "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"
  DETECTRON2_WEIGHTS: "/root/autodl-nas/slowfast/model_final_280758.pkl"

4.2.2 student_behaviors.json

slowfast/demo/AVA/student_behaviors.json

{"hand-raising": 0, "reading": 1, "writing": 2}

4.2.2 demo测试

cd /root/slowfast/
python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50s2.yaml

在这里插入图片描述

5 在非云平台安装SlowFast（本地机器上）

5.1 基础环境

torch版本：
2.0.0+cu117

torchvision版本
0.15.1+cu117

pip install 'git+https://gitee.com/YFwinston/fvcore'

pip install simplejson

conda install av -c conda-forge -y

conda install x264 ffmpeg -c conda-forge -y

pip install -U iopath

pip install psutil

pip install opencv-python

// pip install torchvision

pip install tensorboard

pip install moviepy

pip install pytorchvideo

pip install 'git+https://gitee.com/YFwinston/fairscale'

cd /data/SCB/

git clone https://gitee.com/YFwinston/pytorchvideo.git
cd pytorchvideo
pip install -e .
/data/SCB/

5.2 detectron2_repo 安装

// pip install -U torch torchvision cython
// pip install -U  cython
pip install cython

pip install -U 'git+https://gitee.com/YFwinston/fvcore.git' 'git+https://gitee.com/YFwinston/cocoapi.git#subdirectory=PythonAPI'

git clone https://gitee.com/YFwinston/detectron2 detectron2_repo

pip install typing-extensions==4.3.0

pip install -e detectron2_repo

pip install pillow
pip install pyyaml

pip install scipy
pip install pandas
pip install scikit-learn

5.3 slowfast 安装

git clone https://gitee.com/YFwinston/slowfast

export PYTHONPATH=/data/SCB/slowfast:$PYTHONPATH

cd slowfast
python setup.py build develop
cd ..

5.4 相关文件

ava.json

cd /data/SCB/slowfast/demo/AVA
vim ava.json


{"bend/bow (at the waist)": 0, "crawl": 1, "crouch/kneel": 2, "dance": 3, "fall down": 4, "get up": 5, "jump/leap": 6, "lie/sleep": 7, "martial art": 8, "run/jog": 9, "sit": 10, "stand": 11, "swim": 12, "walk": 13, "answer phone": 14, "brush teeth": 15, "carry/hold (an object)": 16, "catch (an object)": 17, "chop": 18, "climb (e.g., a mountain)": 19, "clink glass": 20, "close (e.g., a door, a box)": 21, "cook": 22, "cut": 23, "dig": 24, "dress/put on clothing": 25, "drink": 26, "drive (e.g., a car, a truck)": 27, "eat": 28, "enter": 29, "exit": 30, "extract": 31, "fishing": 32, "hit (an object)": 33, "kick (an object)": 34, "lift/pick up": 35, "listen (e.g., to music)": 36, "open (e.g., a window, a car door)": 37, "paint": 38, "play board game": 39, "play musical instrument": 40, "play with pets": 41, "point to (an object)": 42, "press": 43, "pull (an object)": 44, "push (an object)": 45, "put down": 46, "read": 47, "ride (e.g., a bike, a car, a horse)": 48, "row boat": 49, "sail boat": 50, "shoot": 51, "shovel": 52, "smoke": 53, "stir": 54, "take a photo": 55, "text on/look at a cellphone": 56, "throw": 57, "touch (an object)": 58, "turn (e.g., a screwdriver)": 59, "watch (e.g., TV)": 60, "work on a computer": 61, "write": 62, "fight/hit (a person)": 63, "give/serve (an object) to (a person)": 64, "grab (a person)": 65, "hand clap": 66, "hand shake": 67, "hand wave": 68, "hug (a person)": 69, "kick (a person)": 70, "kiss (a person)": 71, "lift (a person)": 72, "listen to (a person)": 73, "play with kids": 74, "push (another person)": 75, "sing to (e.g., self, a person, a group)": 76, "take (an object) from (a person)": 77, "talk to (e.g., self, a person, a group)": 78, "watch (a person)": 79}

SLOWFAST_32x2_R101_50_50s.yaml

cd /data/SCB/slowfast/demo/AVA
vim SLOWFAST_32x2_R101_50_50s.yaml

TRAIN:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 1
  EVAL_PERIOD: 1
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  CHECKPOINT_FILE_PATH: /data/SCB/slowfast/mywork/SLOWFAST_32x2_R101_50_50.pkl  #path to pretrain model
  CHECKPOINT_TYPE: pytorch
DATA:
  NUM_FRAMES: 16
  SAMPLING_RATE: 2
  TRAIN_JITTER_SCALES: [256, 320]
  TRAIN_CROP_SIZE: 224
  TEST_CROP_SIZE: 256
  INPUT_CHANNEL_NUM: [3, 3]
DETECTION:
  ENABLE: True
  ALIGNED: False
AVA:
  BGR: False
  DETECTION_SCORE_THRESH: 0.8
  TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
SLOWFAST:
  ALPHA: 4
  BETA_INV: 8
  FUSION_CONV_CHANNEL_RATIO: 2
  FUSION_KERNEL_SZ: 5
RESNET:
  ZERO_INIT_FINAL_BN: True
  WIDTH_PER_GROUP: 64
  NUM_GROUPS: 1
  DEPTH: 101
  TRANS_FUNC: bottleneck_transform
  STRIDE_1X1: False
  NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
  SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]
  SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:
  LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]
  GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
  INSTANTIATION: dot_product
  POOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:
  USE_PRECISE_STATS: False
  NUM_BATCHES_PRECISE: 200
SOLVER:
  MOMENTUM: 0.9
  WEIGHT_DECAY: 1e-7
  OPTIMIZING_METHOD: sgd
MODEL:
  NUM_CLASSES: 80
  ARCH: slowfast
  MODEL_NAME: SlowFast
  LOSS_FUNC: bce
  DROPOUT_RATE: 0.5
  HEAD_ACT: sigmoid
TEST:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 1
DATA_LOADER:
  NUM_WORKERS: 1
  PIN_MEMORY: True
 
NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: .
#TENSORBOARD:
#  MODEL_VIS:
#    TOPK: 2
DEMO:
  ENABLE: True
  LABEL_FILE_PATH: "demo/AVA/ava.json" # Add local label file path here.
  INPUT_VIDEO: "/data/SCB/slowfast/mywork/1.mp4"
  OUTPUT_FILE: "/data/SCB/slowfast/mywork/1_1.mp4"
  DETECTRON2_CFG: "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"
  DETECTRON2_WEIGHTS: "/data/SCB/slowfast/mywork/model_final_280758.pkl"

5.5 demo测试

在/data/SCB/slowfast/mywork/中传入一个视频：1.mp4
执行：

cd /data/SCB/slowfast/
python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50s.yaml

在这里插入图片描述

在这里插入图片描述

5.6 训练

训练的配置文件 SLOWFAST_32x2_R101_50_50s.yaml

slowfast/configs/AVA/c2/SLOWFAST_32x2_R101_50_50s.yaml

TRAIN:
  ENABLE: True
  DATASET: ava
  BATCH_SIZE: 8
  EVAL_PERIOD: 1
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  # CHECKPOINT_FILE_PATH: /data/SCB/slowfast/mywork/SLOWFAST_32x2_R101_50_50.pkl 
  CHECKPOINT_TYPE: pytorch
DATA:
  NUM_FRAMES: 32
  SAMPLING_RATE: 2
  TRAIN_JITTER_SCALES: [256, 320]
  TRAIN_CROP_SIZE: 224
  TEST_CROP_SIZE: 256
  INPUT_CHANNEL_NUM: [3, 3]
  PATH_TO_DATA_DIR: '/data/SCB/Dataset/SCB-ava-Dataset4'
DETECTION:
  ENABLE: True
  # ENABLE: False
  ALIGNED: False
VIS_MASK:
  ENABLE: True
AVA:
  FRAME_DIR: '/data/SCB/Dataset/SCB-ava-Dataset4/frames'
  FRAME_LIST_DIR: '/data/SCB/Dataset/SCB-ava-Dataset4/frame_lists'
  ANNOTATION_DIR: '/data/SCB/Dataset/SCB-ava-Dataset4/annotations'
  DETECTION_SCORE_THRESH: 0.8
  TRAIN_PREDICT_BOX_LISTS: [
    "ava_train_v2.2.csv",
    "person_box_67091280_iou90/ava_detection_train_boxes_and_labels_include_negative_v2.2.csv",
  ]
  TEST_PREDICT_BOX_LISTS: [
    "person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
  FULL_TEST_ON_VAL: True

SLOWFAST:
  ALPHA: 4
  BETA_INV: 8
  FUSION_CONV_CHANNEL_RATIO: 2
  FUSION_KERNEL_SZ: 5
RESNET:
  ZERO_INIT_FINAL_BN: True
  WIDTH_PER_GROUP: 64
  NUM_GROUPS: 1
  DEPTH: 101
  TRANS_FUNC: bottleneck_transform
  STRIDE_1X1: False
  NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
  SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]
  SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:
  LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]
  GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
  INSTANTIATION: dot_product
  POOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:
  USE_PRECISE_STATS: False
  NUM_BATCHES_PRECISE: 200
SOLVER:
  BASE_LR: 0.01
  MOMENTUM: 0.9
  WEIGHT_DECAY: 1e-7
  OPTIMIZING_METHOD: sgd
  MAX_EPOCH: 50
MODEL:
  # NUM_CLASSES: 80
  NUM_CLASSES: 3
  ARCH: slowfast
  MODEL_NAME: SlowFast
  # LOSS_FUNC: bce
  LOSS_FUNC: cross_entropy
  # LOSS_FUNC: F_BCE
  DROPOUT_RATE: 0.5
  # HEAD_ACT: sigmoid
TEST:
  ENABLE: True
  DATASET: ava
  BATCH_SIZE: 1
DATA_LOADER:
  NUM_WORKERS: 2
  PIN_MEMORY: True
NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: '/data/SCB/slowfast/mywork/out/'

注意 NUM_GPUS 和 TEST.BATCH_SIZE 要保持一致

训练

python tools/run_net.py \
  --cfg configs/AVA/c2/SLOWFAST_32x2_R101_50_50s.yaml

在这里插入图片描述

6 其他

6.1 日志文件修改

在日志文件中添加类似如下内容：

{ 'PascalBoxes_PerformanceByCategory/AP@0.5IOU/hand-raising': 0.9341844125742432,
  'PascalBoxes_PerformanceByCategory/AP@0.5IOU/reading': 0.29780414588177817,
  'PascalBoxes_PerformanceByCategory/AP@0.5IOU/writing': 0.375,
  'PascalBoxes_Precision/mAP@0.5IOU': 0.5356628528186738}

在"slowfast/slowfast/utils/ava_eval_helper.py"
添加：

    logger.info("AVA eval done in %f seconds." % (time.time() - eval_start))

    logger.info(results)
    
    return results["PascalBoxes_Precision/mAP@0.5IOU"]

在这里插入图片描述
结果如下：

在这里插入图片描述

相关参考

[深度学习][原创]slowfast的demo跑起来的正确方法

[深度学习][原创]使用labelImg+yolov5完成所有slowfast时空动作检测项目-开山篇

[深度学习][原创]使用labelImg+yolov5完成所有slowfast时空动作检测项目-流程篇

[深度学习][原创]使用labelImg+yolov5完成所有slowfast时空动作检测项目-配置文件篇

[深度学习][原创]使用labelImg+yolov5完成所有slowfast时空动作检测项目-训练测试篇

[深度学习][原创]mmaction2时空动作检测过滤成自己想要的类别

三分钟快速安装 facebookresearch SlowFast

PyTorchVideo SlowFast

CSPhD-winston-杨帆
关注
  • 12
    点赞
  • 踩
  • 49
    收藏
    觉得还不错? 一键收藏
  • 28
    评论
ava.json测试标签数据80种类
11-17
AVA数据集json标签文件ava.json测试标签数据80种类，google研究AVA人类行为数据集的标签文件
22-8-4 mmaction2 slowfast训练日志
08-05
22-8-4 mmaction2 slowfast训练日志 包含配置文件：my_slowfast_kinetics_pretrained_r50_4x16x1_20e_ava_rgb.py 训练结束后使用最好的checkpoint的参数进行测试，将测试结果存储在：part_0.pkl 训练过程的记录：20220804_185539.log.json
关于slowfast自建数据集的训练日志（失败，弃坑）
weixin_40459958的博客
08-15 292
2.如果发生行为的那一秒并没能检测到对象（自己用的检测跟踪算法不是官方那个detectron2，而是自己的yolo+deepsort）那就只好删掉这个视频对应的东西。3.对视频每秒做行为标注时留神每秒一张截图是否有包含动作发生的过程，比如动作发生太短，那也就只好pass了吧。提到行为的标注是按照每秒来的，而不是每帧来的，在ava_help里也是有设置一个秒的范围。之前有试过一个小型数据集按照每帧来标注，效果很混乱，所以这次记录整理一下。1.去除路人的检测标注，只留下自己要识别行为的对象的检测结果。
【深度学习】--slowfast视频理解数据集处理pipeline
最新发布
weixin_40293999的博客
05-10 323
缺点，这两个都没有 排除帧，所谓排除帧， 就是所有帧里面都是有目标的。这样的话，就会造成后面的配置文件有个别文件不起作用。谁知道哪两个文件有排除帧，欢迎评论区见。主要是采样目标后还要还原，还原的不对当下不知道，得训练的时候才知道，而且没法和原来的数据做对比。将配置文件需要的文件都裁切成只包含后面这两个文件，这里面坑很多，我是试出来的，所以记录一下。我要是现在linux上打开就好了，光挨个窜这几个文件窜了2天哎！我的这个文件，能够一键生成目标数据集的配套内容。打开视频汇总的文件，挑两个下载。
SlowFast训练自己的数据集
weixin_43720054的博客
08-15 5800
SlowFast训练自己的数据集
【slowfast 训练自己的数据集】自定义动作，制作自己的数据集，使用预训练模型进行训练，并检测其结果
热门推荐
CSPhD-winston的博客
04-11 2万+
前言 终于到了这一步了，看了很久很久的slowfast，这次终于用slowfast训练了自己的数据集（只是用了个非常小的，非常小的数据集跑了一下） 在看这篇训练自己的数据集博客之前，我觉得有必要了解一下之前我写一些博客： 的slowfast的训练，slowfast的减少数据集 1：【SlowFast复现】SlowFast Networks for Video Recognition复现代码 使用自己的视频进行demo检测 2:【Faster RCNN & detectron2】detectron2实
自定义AVA数据集流程及在SlowFast中训练
qq_45672807的博客
03-07 8062
自定义的AVA数据集在Linux系统下的训练
行为识别系列：Win11系统下使用SlowFast训练AVA数据集
Yayisheng的博客
12-20 1953
本文主要讲述如何在Win11系统使用MMaction2框架下SlowFast网络进行AVA数据集的训练。
抽烟检测数据集 分类识别训练 yolo格式
06-07
标题中的“抽烟检测数据集 分类识别训练 yolo格式”指的是一个用于训练计算机视觉模型，特别是针对抽烟行为检测的数据集。这个数据集采用的是YOLO（You Only Look Once）算法的标注格式，这是一种实时目标检测系统，...
快递包裹YOLO训练数据集
04-25
【标题】"快递包裹YOLO训练数据集"指的是一个专门针对快递包裹识别的深度学习模型训练数据集。YOLO（You Only Look Once）是一种实时目标检测系统，它在计算机视觉领域广泛应用，尤其在物体识别方面表现出色。这个...
鞋子分类训练数据集 yolo txt格式标注文件
06-07
主要涉及的是一个用于训练鞋子分类模型的数据集，该数据集采用了YOLO（You Only Look Once）算法的注释格式。 首先，我们要理解什么是“鞋子分类训练数据集”。在机器学习中，尤其是计算机视觉任务，训练数据集是...
FLIR训练集数据集yolo格式标签
10-12
FLIR训练集是一个专门针对热红外图像处理的数据集，它包含了丰富的热红外图像和对应的标签信息，适用于开发和优化目标检测算法，如YOLO（You Only Look Once）框架。YOLO是一种实时目标检测系统，以其高效和准确的...
YOLO格式的TinyPerson数据集
03-05
数据集样本数量为1532，所有图片已标注为VOC xml和YOLO txt两种格式，其中YOLO TXT 格式的已划分为训练集、验证集和测试集，能直接用于YOLO算法的训练。可用于YOLO小目标检测模型训练，机器学习，深度学习，人工智能...
slowfast训练自己数据---保姆级教程-看这一篇就够了
weixin_47453139的博客
01-09 6344
slowfast迅速跑通
【SlowFasr ava数据集制作】
JYD_0307的博客
11-22 2257
slowfast—ava数据集制作
slowfast安装并训练自己的数据集
qq_36072670的博客
06-01 7743
安装 slowfast主页：https://github.com/facebookresearch/SlowFast
Slow-Fast使用教程2023.5.30
weixin_45231460的博客
05-30 1224
配置跑通slow-fast项目，能够检测输入的视频输出检测结果
Win10 slowfast实现记录
souyan1991的博客
11-10 1865
Win10 slowfast实现记录 slow fast教程运行后报错 AttributeError: module 'torch.distributed' has no attribute 'is_initialized' 可能是pytorch版本问题，需要升级到1.7。 注意不要用官网的方法，用这个方法在虚拟环境下升级link 卸载当前版本的pytorch 进入虚拟环境，用pip卸载pytorch和torchvision pip uninstall torch pip uninstall to
安装pytorch3d
09-04
要安装pytorch3d，您可以按照以下步骤进行操作： 1. 确保您已经安装了Python和pip。您可以在命令行中运行以下命令来检查它们的版本： ``` python --version pip --version ``` 2. 安装PyTorch。根据您的操作系统和硬件配置，访问PyTorch官方网站（https://pytorch.org/）并按照指南安装适合您的版本。 3. 在命令行中运行以下命令来安装PyTorch3D： ``` pip install 'git+https://github.com/facebookresearch/pytorch3d.git@stable' ``` 4. 安装过程可能需要一些时间，请耐心等待。安装完成后，您就可以使用PyTorch3D啦！ 请注意，这是一种基本的安装方法。如果您遇到任何问题，建议您查阅PyTorch3D的官方文档或寻求相关资源的帮助。
评论 28
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值