Feature Detection and Description
General Libraries:
- VLFeat
– Implementation of various feature descriptors (including SIFT, HOG, and LBP) and covariant feature detectors (including DoG, Hessian, Harris Laplace, Hessian Laplace, Multiscale Hessian, Multiscale Harris). Easy-to-use Matlab interface. SeeModern features: Software – Slides providing a demonstration of VLFeat and also links to other software. Check also VLFeat hands-on session training - OpenCV
– Various implementations of modern feature detectors and descriptors (SIFT, SURF, FAST, BRIEF, ORB, FREAK, etc.)
Fast Keypoint Detectors for Real-time Applications:
- FAST
– High-speed corner detector implementation for a wide variety of platforms - AGAST
– Even faster than the FAST corner detector. A multi-scale version of this method is used for the BRISK descriptor (ECCV 2010).
Binary Descriptors for Real-Time Applications:
- BRIEF
– C++ code for a fast and accurate interest point descriptor (not invariant to rotations and scale) (ECCV 2010) - ORB
– OpenCV implementation of the Oriented-Brief (ORB) descriptor (invariant to rotations, but not scale) - BRISK
– Efficient Binary descriptor invariant to rotations and scale. It includes a Matlab mex interface. (ICCV 2011) - FREAK
– Faster than BRISK (invariant to rotations and scale) (CVPR 2012)
SIFT and SURF Implementations:
- SIFT:
VLFeat, OpenCV, Original code by David Lowe, GPU implementation, OpenSIFT - SURF:
Herbert Bay’s code, OpenCV, GPU-SURF
Other Local Feature Detectors and Descriptors:
- VGG Affine Covariant features
– Oxford code for various affine covariant feature detectors and descriptors. - LIOP descriptor
– Source code for the Local Intensity order Pattern (LIOP) descriptor (ICCV 2011). - Local Symmetry Features
– Source code for matching of local symmetry features under large variations in lighting, age, and rendering style (CVPR 2012).
Global Image Descriptors:
- GIST
– Matlab code for the GIST descriptor - CENTRIST
– Global visual descriptor for scene categorization and object detection (PAMI 2011)
Feature Coding and Pooling
- VGG Feature Encoding Toolkit
– Source code for various state-of-the-art feature encoding methods – including Standard hard encoding, Kernel codebook encoding, Locality-constrained linear encoding, and Fisher kernel encoding. - Spatial Pyramid Matching
– Source code for feature pooling based on spatial pyramid matching (widely used for image classification)
Convolutional Nets and Deep Learning
- EBLearn
– C++ Library for Energy-Based Learning. It includes several demos and step-by-step instructions to train classifiers based on convolutional neural networks. - Torch7
– Provides a matlab-like environment for state-of-the-art machine learning algorithms, including a fast implementation of convolutional neural networks. - Deep Learning
- Various links for deep learning software.
- Deformable Part-based Detector
– Library provided by the authors of the original paper (state-of-the-art in PASCAL VOC detection task) - Efficient Deformable Part-Based Detector
– Branch-and-Bound implementation for a deformable part-based detector. - Accelerated Deformable Part Model
– Efficient implementation of a method that achieves the exact same performance of deformable part-based detectors but with significant acceleration (ECCV 2012). - Coarse-to-Fine Deformable Part Model
– Fast approach for deformable object detection (CVPR 2011). - Poselets
– C++ and Matlab versions for object detection based on poselets. - Part-based Face Detector and Pose Estimation
– Implementation of a unified approach for face detection, pose estimation, and landmark localization (CVPR 2012).
Attributes and Semantic Features
- Relative Attributes
– Modified implementation of RankSVM to train Relative Attributes (ICCV 2011). - Object Bank
– Implementation of object bank semantic features (NIPS 2010). See also ActionBank - Classemes, Picodes, and Meta-class features
– Software for extracting high-level image descriptors (ECCV 2010, NIPS 2011, CVPR 2012).
Large-Scale Learning
- Additive Kernels
– Source code for fast additive kernel SVM classifiers (PAMI 2013). - LIBLINEAR
– Library for large-scale linear SVM classification. - VLFeat
– Implementation for Pegasos SVM and Homogeneous Kernel map.
Fast Indexing and Image Retrieval
- FLANN
– Library for performing fast approximate nearest neighbor. - Kernelized LSH
– Source code for Kernelized Locality-Sensitive Hashing (ICCV 2009). - ITQ Binary codes
– Code for generation of small binary codes using Iterative Quantization and other baselines such as Locality-Sensitive-Hashing (CVPR 2011). - INRIA Image Retrieval
– Efficient code for state-of-the-art large-scale image retrieval (CVPR 2011).
Object Detection
- See
Part-based Models and Convolutional Nets above. - Pedestrian Detection at 100fps
– Very fast and accurate pedestrian detector (CVPR 2012). - Caltech Pedestrian Detection Benchmark
– Excellent resource for pedestrian detection, with various links for state-of-the-art implementations. - OpenCV
– Enhanced implementation of Viola&Jones real-time object detector, with trained models for face detection. - Efficient Subwindow Search
– Source code for branch-and-bound optimization for efficient object localization (CVPR 2008).
3D Recognition
- Point-Cloud Library
– Library for 3D image and point cloud processing.
Action Recognition
- ActionBank
– Source code for action recognition based on the ActionBank representation (CVPR 2012). - STIP Features
– software for computing space-time interest point descriptors - Independent Subspace Analysis
– Look for Stacked ISA for Videos (CVPR 2011) - Velocity Histories of Tracked Keypoints
- C++ code for activity recognition using the velocity histories of tracked keypoints (ICCV 2009)
Datasets
Attributes
- Animals with Attributes
– 30,475 images of 50 animals classes with 6 pre-extracted feature representations for each image. - aYahoo and aPascal
– Attribute annotations for images collected from Yahoo and Pascal VOC 2008. - FaceTracer
– 15,000 faces annotated with 10 attributes and fiducial points. - PubFig
– 58,797 face images of 200 people with 73 attribute classifier outputs. - LFW
– 13,233 face images of 5,749 people with 73 attribute classifier outputs. - Human Attributes
– 8,000 people with annotated attributes. Check also this link for another dataset of human attributes. - SUN Attribute Database
– Large-scale scene attribute database with a taxonomy of 102 attributes. - ImageNet Attributes
– Variety of attribute labels for the ImageNet dataset. - Relative attributes
– Data for OSR and a subset of PubFig datasets. Check also this link for the WhittleSearch data. - Attribute Discovery Dataset
– Images of shopping categories associated with textual descriptions.
Fine-grained Visual Categorization
- Caltech-UCSD Birds Dataset
– Hundreds of bird categories with annotated parts and attributes. - Stanford Dogs Dataset
– 20,000 images of 120 breeds of dogs from around the world. - Oxford-IIIT Pet Dataset
– 37 category pet dataset with roughly 200 images for each class. Pixel level trimap segmentation is included. - Leeds Butterfly Dataset
– 832 images of 10 species of butterflies. - Oxford Flower Dataset
– Hundreds of flower categories.
Face Detection
- FDDB
– UMass face detection dataset and benchmark (5,000+ faces) - CMU/MIT
– Classical face detection dataset.
Face Recognition
- Face Recognition Homepage
– Large collection of face recognition datasets. - LFW
– UMass unconstrained face recognition dataset (13,000+ face images). - NIST Face Homepage
– includes face recognition grand challenge (FRGC), vendor tests (FRVT) and others. - CMU Multi-PIE
– contains more than 750,000 images of 337 people, with 15 different views and 19 lighting conditions. - FERET
– Classical face recognition dataset. - Deng Cai’s face dataset in Matlab Format
– Easy to use if you want play with simple face datasets including Yale, ORL, PIE, and Extended Yale B. - SCFace
– Low-resolution face dataset captured from surveillance cameras.
Handwritten Digits
- MNIST
– large dataset containing a training set of 60,000 examples, and a test set of 10,000 examples.
Pedestrian Detection
- Caltech Pedestrian Detection Benchmark
– 10 hours of video taken from a vehicle,350K bounding boxes for about 2.3K unique pedestrians. - INRIA Person Dataset
– Currently one of the most popular pedestrian detection datasets. - ETH Pedestrian Dataset
– Urban dataset captured from a stereo rig mounted on a stroller. - TUD-Brussels Pedestrian Dataset
– Dataset with image pairs recorded in an crowded urban setting with an onboard camera. - PASCAL Human Detection
– One of 20 categories in PASCAL VOC detection challenges. - USC Pedestrian Dataset
– Small dataset captured from surveillance cameras.
Generic Object Recognition
- ImageNet
– Currently the largest visual recognition dataset in terms of number of categories and images. - Tiny Images
– 80 million 32x32 low resolution images. - Pascal VOC
– One of the most influential visual recognition datasets. - Caltech 101
/ Caltech 256 – Popular image datasets containing 101 and 256 object categories, respectively. - MIT LabelMe
– Online annotation tool for building computer vision databases.
Scene Recognition
- MIT SUN Dataset
– MIT scene understanding dataset. - UIUC Fifteen Scene Categories
– Dataset of 15 natural scene categories.
Feature Detection and Description
- VGG Affine Dataset
– Widely used dataset for measuring performance of feature detection and description. CheckVLBenchmarks for an evaluation framework.
Action Recognition
- Benchmarking Activity Recognition
– CVPR 2012 tutorial covering various datasets for action recognition.
RGBD Recognition
- RGB-D Object Dataset
– Dataset containing 300 common household objects
Related Courses
- Visual Recognition
- Kristen Grauman, U. Texas, Fall 2012. - The Cutting Edge of Computer Vision
- Fei-Fei Li, Stanford, Spring 2011. - Learning-based Methods in Vision
- Alyosha Efros and Leonid Sigal, CMU, Spring 2012. - Grounding Object Recognition and Scene Understanding
- Antonio Torralba, MIT, Fall 2011.