SSN debug

SSN debug

一、下载SSN repo并安装dependencies

本人不习惯于用pip install -r requirements.txt故所有包都是用conda单独装的。

• 安装pytorch和torchvision

conda conda install pytorch=0.4.1 torchvision cuda100 -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/linux-64/

• 安装terminaltables

conda install -c conda-forge/label/gcc7 terminaltables

• 安装pandas

conda install pandas

• 安装 scipy

conda install scipy

• 下载repo

在这里插入代码片

• 安装yaml

conda install pyyaml

• 下载repo

git clone --recursive https://github.com/yjxiong/action-detection

二、提取光流

以THUMOS14数据集为例

1. 下载TSN repo

 git clone --recursive https://github.com/yjxiong/temporal-segment-networks

2. 将编译好的denseflow复制到TSN repo

路径为：/home/ubuntu/users/zhanghao/temporal-segment-networks/lib

cp -r /home/ubuntu/users/zhanghao/opencv_3.4.10/dense_flow /home/ubuntu/users/zhanghao/temporal-segment-networks/lib

3.(error) total number of videos found: 0

• 找不到数据集中的视频

(denseflow) 
# zhanghao @ Lab164 in ~/temporal-segment-networks on git:master x [17:35:42] 
$ bash scripts/extract_optical_flow.sh /mnt/DataDrive164/zhanghao/datasets/THUMOS14 /home/ubuntu/users/zhanghao/action-detection/OUT_FOLDER 3
Extracting optical flow from videos in folder: /mnt/DataDrive164/zhanghao/datasets/THUMOS14
reading videos from folder:  /mnt/DataDrive164/zhanghao/datasets/THUMOS14
selected extension of videos: avi
total number of videos found:  0

• solution
  更改/home/ubuntu/users/zhanghao/temporal-segment-networks/lib/dense_flow/bulid_of.py中提取视频文件的格式
  由于THUMOS14数据集中视频都为mp4格式，故需将build_of.py19行

parser.add_argument("--ext", type=str, default='avi', choices=['avi','mp4'], help='video file extensions')

改为

parser.add_argument("--ext", type=str, default='mp4', choices=['avi','mp4'], help='video file extensions')

4. (error)Check failed: [video_stream.isOpened()] Cannot open video stream “true” for optical flow extraction

• 找不到数据集的接口

(denseflow) 
# zhanghao @ Lab164 in ~/temporal-segment-networks on git:master x [17:41:27] 
$ bash scripts/extract_optical_flow.sh /mnt/DataDrive164/zhanghao/datasets/THUMOS14/validation  /home/ubuntu/users/zhanghao/action-detection/OUT_FOLDER 3
Extracting optical flow from videos in folder: /mnt/DataDrive164/zhanghao/datasets/THUMOS14/validation
reading videos from folder:  /mnt/DataDrive164/zhanghao/datasets/THUMOS14/validation
selected extension of videos: mp4
total number of videos found:  1010
2020-08-19 17:42:41,916 FATAL [default] Check failed: [video_stream.isOpened()] Cannot open video stream "true" for optical flow extraction.
2020-08-19 17:42:41,916 FATAL [default] Check failed: [video_stream.isOpened()] Cannot open video stream "true" for optical flow extraction.
2020-08-19 17:42:41,916 WARN  [default] Aborting application. Reason: Fatal log at [/home/ubuntu/users/zhanghao/opencv_3.4.10/dense_flow/src/dense_flow_gpu.cpp:19]
2020-08-19 17:42:41,916 FATAL [default] Check failed: [video_stream.isOpened()] Cannot open video stream "true" for optical flow extraction.
2020-08-19 17:42:41,916 WARN  [default] Aborting application. Reason: Fatal log at [/home/ubuntu/users/zhanghao/opencv_3.4.10/dense_flow/src/dense_flow_gpu.cpp:19]
2020-08-19 17:42:41,916 WARN  [default] Aborting application. Reason: Fatal log at [/home/ubuntu/users/zhanghao/opencv_3.4.10/dense_flow/src/dense_flow_gpu.cpp:19]
170 video_validation_0000948 done
0 video_validation_0000186 done
85 video_validation_0000970 done

• solution
  更改build_of.py第52行和76行分别为：

cmd = os.path.join(df_path + 'build/extract_gpu')+' -f={} -x={} -y={} -i={} -b=20 -t=1 -d={} -s=1 -o={} -w={} -h={}'.format(
        quote(vid_path), quote(flow_x_path), quote(flow_y_path), quote(image_path), dev_id, out_format, new_size[0], new_size[1])

&

cmd = os.path.join(df_path + 'build/extract_warp_gpu')+' -f={} -x={} -y={} -b=20 -t=1 -d={} -s=1 -o={}'.format(
        vid_path, flow_x_path, flow_y_path, dev_id, out_format)

结果还是报同样的错。
经过仔细研究发现，应该更改的是TSN中/home/ubuntu/users/zhanghao/temporal-segment-networks/tools而不是denseflow中/home/ubuntu/users/zhanghao/temporal-segment-networks/lib/dense_flow的build_of.py。

(denseflow) 
# zhanghao @ Lab164 in ~/temporal-segment-networks on git:master x [10:59:57] 
$ bash scripts/extract_optical_flow.sh /mnt/DataDrive164/zhanghao/datasets/THUMOS14/validation  /home/ubuntu/users/zhanghao/action-detection/OUT_FOLDER 3
Extracting optical flow from videos in folder: /mnt/DataDrive164/zhanghao/datasets/THUMOS14/validation
reading videos from folder:  /mnt/DataDrive164/zhanghao/datasets/THUMOS14/validation
selected extension of videos: mp4
total number of videos found:  1010
85 video_validation_0000970 done
170 video_validation_0000948 done
86 video_validation_0000740 done
0 video_validation_0000186 done

现在，不仅可以提取单个文件的光流，还能提取整个数据集中的光流。