视频研究（分类，行为识别，检测等）

视频研究入门经典

 

• Labor-Free Video Concept Learningby Jointly Exploiting Web Videos and Images

    intro: CVPR 2016
    intro: Lead–Exceed Neural Network (LENN), LSTM
    paper: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/06/CVPR16_webly_final.pdf

 

• Video Fill in the Blank with Merging LSTMs

    intro: for Large Scale Movie Description and Understanding Challenge (LSMDC) 2016, "Movie fill-in-the-blank" Challenge, UCF_CRCV
    intro: Video-Fill-in-the-Blank (ViFitB)
    arxiv: https://arxiv.org/abs/1610.04062

• Video Pixel Networks

    intro: Google DeepMind
    arxiv: https://arxiv.org/abs/1610.00527

• Robust Video Synchronization using Unsupervised Deep Learning

    arxiv: https://arxiv.org/abs/1610.05985

• Video Propagation Networks

    intro: CVPR 2017. Max Planck Institute for Intelligent Systems & Bernstein Center for Computational Neuroscience
    project page: https://varunjampani.github.io/vpn/
    arxiv: https://arxiv.org/abs/1612.05478
    github(Caffe): https://github.com/varunjampani/video_prop_networks

• Video Frame Synthesis using Deep Voxel Flow

    project page: https://liuziwei7.github.io/projects/VoxelFlow.html
    arxiv: https://arxiv.org/abs/1702.02463

• Optimizing Deep CNN-Based Queries over Video Streams at Scale

    intro: Stanford InfoLab
    keywords: NoScope. difference detectors, specialized models
    arxiv: https://arxiv.org/abs/1703.02529
    github: https://github.com/stanford-futuredata/noscope
    github: https://github.com/stanford-futuredata/tensorflow-noscope

• NoScope: 1000x Faster Deep Learning Queries over Video

http://dawn.cs.stanford.edu/2017/06/22/noscope/

• Unsupervised Visual-Linguistic Reference Resolution in Instructional Videos

    intro: CVPR 2017. Stanford University & University of Southern California
    arxiv: https://arxiv.org/abs/1703.02521

• ProcNets: Learning to Segment Procedures in Untrimmed and Unconstrained Videos

https://arxiv.org/abs/1703.09788

• Unsupervised Learning Layers for Video Analysis

    intro: Baidu Research
    intro: "The experiments demonstrated the potential applications of UL layers and online learning algorithm to head orientation estimation and moving object localization"
    arxiv: https://arxiv.org/abs/1705.08918

• Look, Listen and Learn

    intro: DeepMind
    intro: "Audio-Visual Correspondence" learning
    arxiv: https://arxiv.org/abs/1705.08168

• Video Imagination from a Single Image with Transformation Generation

    intro: Peking University
    arxiv: https://arxiv.org/abs/1706.04124
    github: https://github.com/gitpub327/VideoImagination

• Learning to Learn from Noisy Web Videos

    intro: CVPR 2017. Stanford University & CMU & Simon Fraser University
    arxiv: https://arxiv.org/abs/1706.02884

• Convolutional Long Short-Term Memory Networks for Recognizing First Person Interactions

    intro: Accepted on the second International Workshop on Egocentric Perception, Interaction and Computing(EPIC) at International Conference on Computer Vision(ICCV-17)
    arxiv: https://arxiv.org/abs/1709.06495

• Learning Binary Residual Representations for Domain-specific Video Streaming

    intro: AAAI 2018
    project page: http://research.nvidia.com/publication/2018-02_Learning-Binary-Residual
    arxiv: https://arxiv.org/abs/1712.05087

• Video Representation Learning Using Discriminative Pooling

    intro: CVPR 2018
    arxiv: https://arxiv.org/abs/1803.10628

• Rethinking the Faster R-CNN Architecture for Temporal Action Localization

    intro: CVPR 2018
    arxiv: https://arxiv.org/abs/1804.07667

• Deep Keyframe Detection in Human Action Videos

    intro: two-stream ConvNet
    arxiv: https://arxiv.org/abs/1804.10021

• FFNet: Video Fast-Forwarding via Reinforcement Learning

    intro: CVPR 2018
    arxiv: https://arxiv.org/abs/1805.02792

• Fast forwarding Egocentric Videos by Listening and Watching

https://arxiv.org/abs/1806.04620

• Scanner: Efficient Video Analysis at Scale

    intro: CMU
    arxiv: https://arxiv.org/abs/1805.07339

• Massively Parallel Video Networks

    intro: DeepMind & University of Oxford
    arxiv: https://arxiv.org/abs/1806.03863

• Object Level Visual Reasoning in Videos

    intro: LIRIS & Facebook AI Research
    arxiv: https://arxiv.org/abs/1806.06157

• Video Time: Properties, Encoders and Evaluation

    intro: BMVC 2018
    arxiv: https://arxiv.org/abs/1807.06980

 

视频行为识别检测综述 IDT TSN CNN-LSTM C3D CDC R-C3D

 

视频分类

 

• Large-scale Video Classification with Convolutional Neural Networks

    intro: CVPR 2014
    project page: http://cs.stanford.edu/people/karpathy/deepvideo/
    paper: www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Karpathy_Large-scale_Video_Classification_2014_CVPR_paper.pdf

• Exploiting Image-trained CNN Architectures for Unconstrained Video Classification

    intro: Video-level event detection. extracting deep features for each frame, averaging frame-level deep features
    arxiv: http://arxiv.org/abs/1503.04144

• Beyond Short Snippets: Deep Networks for Video Classification

    intro: CNN + LSTM
    arxiv: http://arxiv.org/abs/1503.08909
    demo: http://pan.baidu.com/s/1eQ9zLZk

• Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification

    intro: ACM Multimedia, 2015
    arxiv: http://arxiv.org/abs/1504.01561

Video Content Recognition with Deep Learning

    author: Zuxuan Wu, Fudan University
    slides: http://vision.ouc.edu.cn/valse/slides/20160420/Zuxuan%20Wu%20-%20Video%20Content%20Recognition%20with%20Deep%20Learning-Zuxuan%20Wu.pdf

• Video Content Recognition with Deep Learning

    author: Yu-Gang Jiang, Lab for Big Video Data Analytics (BigVid), Fudan University
    slides: http://www.yugangjiang.info/slides/DeepVideoTalk-2015.pdf

• Efficient Large Scale Video Classification

    intro: Google
    arxiv: http://arxiv.org/abs/1505.06250

• Fusing Multi-Stream Deep Networks for Video Classification

    arxiv: http://arxiv.org/abs/1509.06086

• Learning End-to-end Video Classification with Rank-Pooling

    paper: http://jmlr.org/proceedings/papers/v48/fernando16.html
    paper: http://jmlr.csail.mit.edu/proceedings/papers/v48/fernando16.pdf
    summary(by Hugo Larochelle): http://www.shortscience.org/paper?bibtexKey=conf/icml/FernandoG16#hlarochelle

• Deep Learning for Video Classification and Captioning

    arxiv: http://arxiv.org/abs/1609.06782

• Fast Video Classification via Adaptive Cascading of Deep Models

    arxiv: https://arxiv.org/abs/1611.06453

• Deep Feature Flow for Video Recognition

    intro: CVPR 2017
    intro: It provides a simple, fast, accurate, and end-to-end framework for video recognition (e.g., object detection and semantic segmentation in videos)
    arxiv: https://arxiv.org/abs/1611.07715
    github(official, MXNet): https://github.com/msracver/Deep-Feature-Flow
    youtube: https://www.youtube.com/watch?v=J0rMHE6ehGw

• Large-Scale YouTube-8M Video Understanding with Deep Neural Networks

https://arxiv.org/abs/1706.04488

• Deep Learning Methods for Efficient Large Scale Video Labeling

    intro: Solution to the Kaggle's competition Google Cloud & YouTube-8M Video Understanding Challenge
    arxiv: https://arxiv.org/abs/1706.04572
    github: https://github.com/mpekalski/Y8M

• Learnable pooling with Context Gating for video classification

    intro: CVPR17 Youtube 8M workshop. Kaggle 1st place
    arxiv: https://arxiv.org/abs/1706.06905
    github: https://github.com/antoine77340/LOUPE

• Aggregating Frame-level Features for Large-Scale Video Classification

    intro: Youtube-8M Challenge, 4th place
    arxiv: https://arxiv.org/abs/1707.00803

• Tensor-Train Recurrent Neural Networks for Video Classification

https://arxiv.org/abs/1707.01786

• Hierarchical Deep Recurrent Architecture for Video Understanding

    intro: Classification Challenge Track paper in CVPR 2017 Workshop on YouTube-8M Large-Scale Video Understanding
    arxiv: https://arxiv.org/abs/1707.03296

• Large-scale Video Classification guided by Batch Normalized LSTM Translator

    intro: CVPR2017 Workshop on Youtube-8M Large-scale Video Understanding
    arxiv: https://arxiv.org/abs/1707.04045

• UTS submission to Google YouTube-8M Challenge 2017

    intro: CVPR'17 Workshop on YouTube-8M
    arxiv: https://arxiv.org/abs/1707.04143
    github: https://github.com/ffmpbgrnn/yt8m

• A spatiotemporal model with visual attention for video classification

https://arxiv.org/abs/1707.02069

• Cultivating DNN Diversity for Large Scale Video Labelling

    intro: CVPR 2017 Youtube-8M Workshop
    arxiv: https://arxiv.org/abs/1707.04272

• Attention Transfer from Web Images for Video Recognition

    intro: ACM Multimedia, 2017
    arxiv: https://arxiv.org/abs/1708.00973

• Non-local Neural Networks

    intro: CVPR 2018. CMU & Facebook AI Research
    arxiv: https://arxiv.org/abs/1711.07971
    github(Caffe2): https://github.com/facebookresearch/video-nonlocal-net

• Temporal 3D ConvNets: New Architecture and Transfer Learning for Video Classification

https://arxiv.org/abs/1711.08200

• Appearance-and-Relation Networks for Video Classification

    arxiv: https://arxiv.org/abs/1711.09125
    github: https://github.com/wanglimin/ARTNet

• Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification

    intro: ECCV 2018. Google Research & University of California San Diego
    arxiv: https://arxiv.org/abs/1712.04851

• Long Activity Video Understanding using Functional Object-Oriented Network

https://arxiv.org/abs/1807.00983

Deep Architectures and Ensembles for Semantic Video Classification

https://arxiv.org/abs/1807.01026

• Deep Discriminative Model for Video Classification

    intro: ECCV 2018
    arxiv: https://arxiv.org/abs/1807.08259

• Deep Video Color Propagation

    intro: BMVC 2018
    arxuv: https://arxiv.org/abs/1808.03232

• Non-local NetVLAD Encoding for Video Classification

    intro: ECCV 2018 workshop on YouTube-8M Large-Scale Video Understanding
    intro: Tencent AI Lab & Fudan University
    arxiv: https://arxiv.org/abs/1810.00207

Learnable Pooling Methods for Video Classification

    intro: Youtube 8M ECCV18 Workshop
    arxiv: https://arxiv.org/abs/1810.00530
    github: https://github.com/pomonam/LearnablePoolingMethods

• NeXtVLAD: An Efficient Neural Network to Aggregate Frame-level Features for Large-scale Video Classification

    intro: ECCV 2018 workshop
    arxiv: https://arxiv.org/abs/1811.05014
    github: https://github.com/linrongc/youtube-8m

 

 

 

视频行为识别 / 行为检测

 

• 3d convolutional neural networks for human action recognition

    paper: http://www.cs.odu.edu/~sji/papers/pdf/Ji_ICML10.pdf

• Sequential Deep Learning for Human Action Recognition

    paper: http://liris.cnrs.fr/Documents/Liris-5228.pdf

• Two-stream convolutional networks for action recognition in videos

    arxiv: http://arxiv.org/abs/1406.2199

• Finding action tubes

    intro: "built action models from shape and motion cues. They start from the image proposals and select the motion salient subset of them and extract saptio-temporal features to represent the video using the CNNs."
    arxiv: http://arxiv.org/abs/1411.6031

• Hierarchical Recurrent Neural Network for Skeleton Based Action Recognition

    paper: http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Du_Hierarchical_Recurrent_Neural_2015_CVPR_paper.pdf

• Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors

    intro: CVPR 2015. TDD
    paper: www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Wang_Action_Recognition_With_2015_CVPR_paper.pdf
    ext: http://www.cv-foundation.org/openaccess/content_cvpr_2015/app/2B_105_ext.pdf
    poster: https://wanglimin.github.io/papers/WangQT_CVPR15_Poster.pdf
    github: https://github.com/wanglimin/TDD

• Action Recognition by Hierarchical Mid-level Action Elements

    paper: http://cvgl.stanford.edu/papers/tian2015.pdf

Contextual Action Recognition with R*CNN

    arxiv: http://arxiv.org/abs/1505.01197
    github: https://github.com/gkioxari/RstarCNN

• Towards Good Practices for Very Deep Two-Stream ConvNets

    arxiv: http://arxiv.org/abs/1507.02159
    github: https://github.com/yjxiong/caffe

• Action Recognition using Visual Attention

    intro: LSTM / RNN
    arxiv: http://arxiv.org/abs/1511.04119
    project page: http://shikharsharma.com/projects/action-recognition-attention/
    github(Python/Theano): https://github.com/kracwarlock/action-recognition-visual-attention

• End-to-end Learning of Action Detection from Frame Glimpses in Videos

    intro: CVPR 2016
    project page: http://ai.stanford.edu/~syyeung/frameglimpses.html
    arxiv: http://arxiv.org/abs/1511.06984
    paper: http://vision.stanford.edu/pdf/yeung2016cvpr.pdf

• Multi-velocity neural networks for gesture recognition in videos

    arxiv: http://arxiv.org/abs/1603.06829

Active Learning for Online Recognition of Human Activities from Streaming Videos

    arxiv: http://arxiv.org/abs/1604.02855

• Convolutional Two-Stream Network Fusion for Video Action Recognition

    arxiv: http://arxiv.org/abs/1604.06573
    github: https://github.com/feichtenhofer/twostreamfusion

• Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables

    arxiv: http://arxiv.org/abs/1604.08880

• Unsupervised Semantic Action Discovery from Video Collections

    arxiv: http://arxiv.org/abs/1605.03324

• Anticipating Visual Representations from Unlabeled Video

    paper: http://web.mit.edu/vondrick/prediction.pdf

VideoLSTM Convolves, Attends and Flows for Action Recognition

    arxiv: http://arxiv.org/abs/1607.01794

• Hierarchical Attention Network for Action Recognition in Videos (HAN)

    arxiv: http://arxiv.org/abs/1607.06416

• Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition

    arxiv: http://arxiv.org/abs/1607.07043

• Connectionist Temporal Modeling for Weakly Supervised Action Labeling

    arxiv: http://arxiv.org/abs/1607.08584

• CUHK & ETHZ & SIAT Submission to ActivityNet Challenge 2016

    intro: won the 1st place in the untrimmed video classification task of ActivityNet Challenge 2016. TSN
    arxiv: http://arxiv.org/abs/1608.00797
    github: https://github.com/yjxiong/anet2016-cuhk

• Actionness Estimation Using Hybrid FCNs

    intro: CVPR 2016. H-FCN
    project page: http://wanglimin.github.io/actionness_hfcn/index.html
    paper: http://wanglimin.github.io/papers/WangQTV_CVPR16.pdf
    github: https://github.com/wanglimin/actionness-estimation/

• Real-time Action Recognition with Enhanced Motion Vector CNNs

    intro: CVPR 2016
    project page: http://zbwglory.github.io/MV-CNN/index.html
    paper: http://wanglimin.github.io/papers/ZhangWWQW_CVPR16.pdf
    github: https://github.com/zbwglory/MV-release

• Temporal Segment Networks: Towards Good Practices for Deep Action Recognition

    intro: ECCV 2016. HMDB51: 69.4%, UCF101: 94.2%
    arxiv: http://arxiv.org/abs/1608.00859
    paper: http://wanglimin.github.io/papers/WangXWQLTV_ECCV16.pdf
    github: https://github.com/yjxiong/temporal-segment-networks

• Temporal Segment Networks for Action Recognition in Videos

    intro: An extension of submission http://arxiv.org/abs/1608.00859
    arxiv: https://arxiv.org/abs/1705.02953

• Hierarchical Attention Network for Action Recognition in Videos

    arxiv: http://arxiv.org/abs/1607.06416

• DeepCAMP: Deep Convolutional Action & Attribute Mid-Level Patterns

    intro: CVPR 2016
    arxiv: http://arxiv.org/abs/1608.03217

• Depth2Action: Exploring Embedded Depth for Large-Scale Action Recognition

    arxiv: http://arxiv.org/abs/1608.04339

• Dynamic Image Networks for Action Recognition

    intro: CVPR 2016
    arxiv: http://users.cecs.anu.edu.au/~sgould/papers/cvpr16-dynamic_images.pdf
    github: https://github.com/hbilen/dynamic-image-nets

• Human Action Recognition without Human

    arxiv: http://arxiv.org/abs/1608.07876

• Temporal Convolutional Networks: A Unified Approach to Action Segmentation

    arxiv: http://arxiv.org/abs/1608.08242
    ECCV 2016 workshop: http://bravenewmotion.github.io/

• Temporal Activity Detection in Untrimmed Videos with Recurrent Neural Networks

    intro: Bachelor Thesis Report at ETSETB TelecomBCN
    project page: https://imatge-upc.github.io/activitynet-2016-cvprw/
    arxiv: http://arxiv.org/abs/1608.08128
    github: https://github.com/imatge-upc/activitynet-2016-cvprw

• Sequential Deep Trajectory Descriptor for Action Recognition with Three-stream CNN

    arxiv: http://arxiv.org/abs/1609.03056

• Semi-Coupled Two-Stream Fusion ConvNets for Action Recognition at Extremely Low Resolutions

    arxiv: https://arxiv.org/abs/1610.03898

• Spatiotemporal Residual Networks for Video Action Recognition

    intro: NIPS 2016
    arxiv: https://arxiv.org/abs/1611.02155

• Action Recognition Based on Joint Trajectory Maps Using Convolutional Neural Networks

    arxiv: https://arxiv.org/abs/1611.02447

• Deep Recurrent Neural Network for Mobile Human Activity Recognition with High Throughput

    arxiv: https://arxiv.org/abs/1611.03607

• Joint Network based Attention for Action Recognition

    arxiv: https://arxiv.org/abs/1611.05215

• Temporal Convolutional Networks for Action Segmentation and Detection

    arxiv: https://arxiv.org/abs/1611.05267

• AdaScan: Adaptive Scan Pooling in Deep Convolutional Neural Networks for Human Action Recognition in Videos

    arxiv: https://arxiv.org/abs/1611.08240

• ActionFlowNet: Learning Motion Representation for Action Recognition

    arxiv: https://arxiv.org/abs/1612.03052

• Higher-order Pooling of CNN Features via Kernel Linearization for Action Recognition

    intro: Australian Center for Robotic Vision & Data61/CSIRO
    arxiv: https://arxiv.org/abs/1701.05432

• Tube Convolutional Neural Network (T-CNN) for Action Detection in Videos

https://arxiv.org/abs/1703.10664

• Temporal Action Detection with Structured Segment Networks

    project page: http://yjxiong.me/others/ssn/
    arxiv: https://arxiv.org/abs/1704.06228
    github: https://github.com/yjxiong/action-detection

• Recurrent Residual Learning for Action Recognition

https://arxiv.org/abs/1706.08807

• Hierarchical Multi-scale Attention Networks for Action Recognition

https://arxiv.org/abs/1708.07590

• Two-stream Flow-guided Convolutional Attention Networks for Action Recognition

    intro: International Conference of Computer Vision Workshop (ICCVW), 2017
    arxiv: https://arxiv.org/abs/1708.09268

• Action Classification and Highlighting in Videos

https://arxiv.org/abs/1708.09522

• Real-Time Action Detection in Video Surveillance using Sub-Action Descriptor with Multi-CNN

https://arxiv.org/abs/1710.03383

• End-to-end Video-level Representation Learning for Action Recognition

    keywords: Deep networks with Temporal Pyramid Pooling (DTPP)
    arxiv: https://arxiv.org/abs/1711.04161

• Fully-Coupled Two-Stream Spatiotemporal Networks for Extremely Low Resolution Action Recognition

    intro: WACV 2018
    arxiv: https://arxiv.org/abs/1801.03983

• DiscrimNet: Semi-Supervised Action Recognition from Videos using Generative Adversarial Networks

https://arxiv.org/abs/1801.07230

• A Fusion of Appearance based CNNs and Temporal evolution of Skeleton with LSTM for Daily Living Action Recognition

https://arxiv.org/abs/1802.00421

• Real-Time End-to-End Action Detection with Two-Stream Networks

https://arxiv.org/abs/1802.08362

• A Closer Look at Spatiotemporal Convolutions for Action Recognition

    intro: CVPR 2018. Facebook Research
    intro: R(2+1)D and Mixed-Convolutions for Action Recognition.
    project page: https://dutran.github.io/R2Plus1D/
    arxiv: https://arxiv.org/abs/1711.11248
    github: https://github.com/facebookresearch/R2Plus1D

• VideoCapsuleNet: A Simplified Network for Action Detection

https://arxiv.org/abs/1805.08162

• Where and When to Look? Spatio-temporal Attention for Action Recognition in Videos

https://arxiv.org/abs/1810.04511
Projects

• A Torch Library for Action Recognition and Detection Using CNNs and LSTMs

    intro: CS231n student project report
    paper: http://cs231n.stanford.edu/reports2016/221_Report.pdf
    github: https://github.com/garythung/torch-lrcn

• 2016 ActivityNet action recognition challenge. CNN + LSTM approach. Multi-threaded loading.

    github: https://github.com/jrbtaylor/ActivityNet

• LSTM for Human Activity Recognition

    github: https://github.com/guillaume-chevalier/LSTM-Human-Activity-Recognition/
    github(MXNet): https://github.com/Ldpe2G/DeepLearningForFun/tree/master/Mxnet-Scala/HumanActivityRecognition

• Scanner: Efficient Video Analysis at Scale

    intro: Locate and recognize faces in a video, Detect shots in a film, Search videos by image
    github: https://github.com/scanner-research/scanner

• Charades Starter Code for Activity Classification and Localization

    intro: Activity Recognition Algorithms for the Charades Dataset
    github: https://github.com/gsig/charades-algorithms

• NonLocalNetwork and Sequeeze-Excitation Network

    intro: MXNet implementation of Non-Local and Squeeze-Excitation network
    github: https://github.com/WillSuen/NonLocalandSEnet

 

 

 

事件识别

 

• TagBook: A Semantic Video Representation without Supervision for Event Detection

    arxiv: http://arxiv.org/abs/1510.02899

• AENet: Learning Deep Audio Features for Video Analysis

    arxiv: https://arxiv.org/abs/1701.00599
    github: https://github.com/znaoya/aenet

 

 

 

事件检测

 

• DevNet: A Deep Event Network for Multimedia Event Detection and Evidence Recounting

    paper: http://120.52.72.47/winsty.net/c3pr90ntcsf0/papers/devnet.pdf
    paper: http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Gan_DevNet_A_Deep_2015_CVPR_paper.pdf

• Detecting events and key actors in multi-person videos

    intro: CVPR 2016
    arxiv: http://arxiv.org/abs/1511.02917
    paper: www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Ramanathan_Detecting_Events_and_CVPR_2016_paper.pdf
    paper: http://vision.stanford.edu/pdf/johnson2016cvpr.pdf
    blog: http://www.leiphone.com/news/201606/l1TKIRFLO3DUFNNu.html

• Deep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection

    intro: INTERSPEECH 2016
    arxiv: https://arxiv.org/abs/1604.07160

• Efficient Action Detection in Untrimmed Videos via Multi-Task Learning

    arxiv: https://arxiv.org/abs/1612.07403

• Joint Event Detection and Description in Continuous Video Streams

    intro: Joint Event Detection and Description Network (JEDDi-Net)
    arxiv: https://arxiv.org/abs/1802.10250

---

---

---

---