3D ConvNet Demo运行

3D ConvNet1是由Du Tran等人在2015年提出的提取视频时间域和空间域特征的三维立体卷积神经网络。Demo运行参考了C3D User Guide。本文会提到一些运行demo的注意事项，以避免出现我个人遇到的问题。

Run Train 3D ConvNet Demo

Change directory to YOUR_C3D_HOME/examples/c3d_trian_ucf101/

• Compute volume mean from list
  
  • run sh create_volume_mean.sh to compute the volume mean file.
• Train your own network from scratch
  
  • run sh train_ucf101.sh to train, expect a couple days to finish.
  • when running this train_ucf101.sh file, I encountered following problem, while I change the data_dir to my own path and set the GPU device to #1, which is available:

I0921 21:17:04.632088 6032 video_data_layer.cpp:344] read video from /MY_DATA_PATH/v_JumpingJack_g25_c05/
F0921 21:17:04.632129 6032 video_data_layer.cpp:346] Check failed: ReadImageSequenceToVolumeDatum(file_list_[id].c_str(), 1, label_list_[id], new_length, new_height, new_width, sampling_rate, &datum)
*** Check failure stack trace: ***
 0x7efd01251c7d google::LogMessage::Fail()
 0x7efd01253b30 google::LogMessage::SendToLog()
@ 0x7efd01251842 google::LogMessage::Flush()
@ 0x7efd0125454e google::LogMessageFatal::~LogMessageFatal()
@ 0x4b01fe caffe::VideoDataLayer<>::SetUp()
@ 0x4458cb caffe::Net<>::Init()
@ 0x446b75 caffe::Net<>::Net()
@ 0x43412f caffe::Solver<>::Init()
@ 0x43783b caffe::Solver<>::Solver()
@ 0x40b826 main
@ 0x7efcfcfd9a40 (unknown)
@ 0x40e079 _start
Aborted (core dumped)0

• Solution: as I compared my extracted frame files with the one in demo, I found that they are different in frame image extension. The frame image extracted by author is in JPG while mine is in JPEG, so I change them in to JPG.
• With the problem above solved, I encountered a new problem:

sdy@sdy:~/Git/C3D/examples/c3d_train_ucf101$ sh train_ucf101.sh
I0108 11:25:15.496800  6760 train_net.cpp:26] Starting Optimization
I0108 11:25:15.496939  6760 solver.cpp:41] Creating training net.
I0108 11:25:15.505625  6760 net.cpp:76] Creating Layer data
I0108 11:25:15.505667  6760 net.cpp:112] data -> data
I0108 11:25:15.505692  6760 net.cpp:112] data -> label
I0108 11:25:15.512724  6760 video_data_layer.cpp:283] Opening file ../c3d_finetuning/train_01.lst
I0108 11:25:15.601454  6760 video_data_layer.cpp:312] Shuffling data
I0108 11:25:15.989821  6760 video_data_layer.cpp:317] A total of 107258 video chunks.
I0108 11:25:15.989881  6760 video_data_layer.cpp:344] read video from /home/sdy/Git/C3D/data/UCF101/frames/v_JumpingJack_g25_c05/
I0108 11:25:16.024459  6760 video_data_layer.cpp:365] output data size: 30,3,16,112,112
I0108 11:25:16.024513  6760 video_data_layer.cpp:387] Loading mean file from ucf101_train_mean.binaryproto
F0108 11:25:16.024785  6760 blob.cpp:98] Check failed: data_ 
*** Check failure stack trace: ***
    @     0x7fa6f45a8daa  (unknown)
    @     0x7fa6f45a8ce4  (unknown)
    @     0x7fa6f45a86e6  (unknown)
    @     0x7fa6f45ab687  (unknown)
    @           0x436159  caffe::Blob<>::mutable_cpu_data()
    @           0x4371e6  caffe::Blob<>::FromProto()
    @           0x4ce1c5  caffe::VideoDataLayer<>::SetUp()
    @           0x45ea1f  caffe::Net<>::Init()
    @           0x4602c0  caffe::Net<>::Net()
    @           0x43aaee  caffe::Solver<>::Init()
    @           0x43fb8a  caffe::Solver<>::Solver()
    @           0x40b15f  main
    @     0x7fa6f0a95f45  (unknown)
    @           0x40de1e  (unknown)
    @              (nil)  (unknown)
Aborted (core dumped)

• Solution:报错显示问题出在ucf101_train_mean.binaryproto文件上，于是重新运行create_volume_mean.sh，报错如下。参考github上相关问题答案，作者表示此类问题出在train list上，文件路径不正确或者帧文件起始编码不是从000001开始导致。因此，通过重新对比文件路径和list文件，发现有两个视频命名大小写与list中路径不同，如文件名是v_HandstandPushups_g01_c01但路径却是：/home/…/v_HandStandPushups_g01_c01/ 1 36，由此作相应修改。

sdy@sdy:~/Git/C3D/examples/c3d_train_ucf101$ sh create_volume_mean.sh
I0108 15:20:14.439486  8571 compute_volume_mean_from_list.cpp:53] using dropping rate 10
I0108 15:20:14.479281  8571 compute_volume_mean_from_list.cpp:80] Starting Iteration
F0108 15:20:54.331308  8571 compute_volume_mean_from_list.cpp:92] Check failed: size_in_datum == data_size (0 vs. 1050624) Incorrect data field size 0
*** Check failure stack trace: ***
    @     0x7f0e683a0daa  (unknown)
    @     0x7f0e683a0ce4  (unknown)
    @     0x7f0e683a06e6  (unknown)
    @     0x7f0e683a3687  (unknown)
    @           0x408d20  main
    @     0x7f0e64f91f45  (unknown)
    @           0x408edd  (unknown)
    @              (nil)  (unknown)
Aborted (core dumped)

• 修改后成功生成ucf101_trian_mean.binaryproto文件

sdy@sdy:~/Git/C3D/examples/c3d_train_ucf101$ sh create_volume_mean.sh
I0108 22:02:42.597549 22827 compute_volume_mean_from_list.cpp:53] using dropping rate 10
I0108 22:02:42.662868 22827 compute_volume_mean_from_list.cpp:80] Starting Iteration
E0108 22:05:40.792213 22827 compute_volume_mean_from_list.cpp:106] Processed 10000 files.
E0108 22:05:50.637400 22827 compute_volume_mean_from_list.cpp:112] Processed 10725 files.
I0108 22:05:50.639452 22827 compute_volume_mean_from_list.cpp:119] Write to ucf101_train_mean.binaryproto

• 运行train_ucf101.sh

sdy@sdy:~/Git/C3D/examples/c3d_train_ucf101$ sh train_ucf101.sh
I0108 22:10:21.996961 22888 train_net.cpp:26] Starting Optimization
I0108 22:10:21.997050 22888 solver.cpp:41] Creating training net.
I0108 22:10:21.997517 22888 net.cpp:76] Creating Layer data
I0108 22:10:21.997531 22888 net.cpp:112] data -> data
I0108 22:10:21.997541 22888 net.cpp:112] data -> label
I0108 22:10:21.997555 22888 video_data_layer.cpp:283] Opening file ../c3d_finetuning/train_01.lst
I0108 22:10:22.070096 22888 video_data_layer.cpp:312] Shuffling data
I0108 22:10:22.391844 22888 video_data_layer.cpp:317] A total of 107258 video chunks.
I0108 22:10:22.391902 22888 video_data_layer.cpp:344] read video from /home/sdy/Git/C3D/data/UCF101/frames/v_JumpingJack_g25_c05/
I0108 22:10:22.410243 22888 video_data_layer.cpp:365] output data size: 30,3,16,112,112
I0108 22:10:22.410284 22888 video_data_layer.cpp:387</