视频动作识别--Towards Good Practices for Very Deep Two-Stream ConvNets

Towards Good Practices for Very Deep Two-stream ConvNets
http://yjxiong.me/others/action_recog/
https://github.com/yjxiong/caffe/tree/action_recog

本文首先指出对于静态图像分类，CNN已经取得很大进步，但是对于视频动作分类，CNN网络表现的不是很好。这里我们分析了一下原因，主要有两个:1) 使用的CNN网络 relatively shallow，没有使用 VGGNet , GoogLeNet 这些高大上的网络；2）没有足够的训练数据。这里我们提出的解决方法是：一个是使用高大上的网络VGGNet , GoogLeNet；另一个是针对训练数据过少的问题： (i) pre-training for both spatial and temporal nets, (ii) smaller learningrates, (iii) more data augmentation techniques, (iv) high drop out ratio.

2 Very Deep Two-stream ConvNets
2.1. Network architectures
这里我们分别实现了 GoogLeNet and VGGNet

very Deep Two-stream ConvNets.
The spatial net is built on a single frame image (224 × 224 × 3)，这个和图像分类网络是一致的。

The input of temporal net is 10-frame stacking of optical flow fields (224 × 224 × 20), 这个网络的滤波器需要根据输入进行修改.

2.2. Network training
这里主要介绍怎么在 UCF101 dataset 训练我们的 very deep two-stream ConvNets

首先是预训练 Pre-training for Two-stream ConvNets， 预训练完之后，我们对两个网络分别使用 Smaller Learning Rate，

数据增强
More Data Augmentation Techniques. random cropping and horizontal flipping

two new data augmentation techniques：1） corner cropping strategy，2）multi-scale cropping method

High Dropout Ratio： In particular, we set 0.9 and 0.8 drop out ratios for the fully connected layers of temporal nets. For spatial nets, we set 0.9 and 0.9 drop out ratios for the fully connected layers

Multi-GPU training

3 Experiments