TSM: Temporal Shift Module for Efficient Video Understanding

TSM: Temporal Shift Module for Efficient Video Understanding[Website][arXiv][Demo]

@inproceedings{lin2019tsm,
  title={TSM: Temporal Shift Module for Efficient Video Understanding},
  author={Lin, Ji and Gan, Chuang and Han, Song},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision},
  year={2019}
} 

[NEW!]We have released the pre-trainedoptical flowmodel on Kinetics. We believe the pre-trained weight will help the training of two-stream models on other datasets.

[NEW!]We have released the code of online hand gesture recognition on NVIDIA Jeston Nano. It can achieve real-time recognition at only 8 watts. Seeonline_demofolder for the details.[Full Video]

 

Overview

We release the PyTorch code of theTemporal Shift Module.

 

Content

• Prerequisites
• Data Preparation
• Code
• Pretrained Models
  • Kinetics-400
    • Dense Sample
    • Unifrom Sampling
  • Something-Something
    • Something-Something-V1
    • Something-Something-V2
• Testing
• Training
• Live Demo on NVIDIA Jetson Nano

Prerequisites

The code is built with following libraries:

• PyTorch1.0 or higher
• TensorboardX
• tqdm
• scikit-learn

For video data pre-processing, you may needffmpeg.

Data Preparation

We need to first extract videos into frames for fast reading. Please refer toTSNrepo for the detailed guide of data pre-processing.

We have successfully trained onKinetics,UCF101,HMDB51,