Discriminatively Trained Part Based Models for Object Detection
The object detector described below has been initially proposed byP.F. Felzenszwalb in [Felzenszwalb2010]. It is based on aDalal-Triggs detector that uses a single filter on histogram oforiented gradients (HOG) features to represent an object category.This detector uses a sliding window approach, where a filter isapplied at all positions and scales of an image. The firstinnovation is enriching the Dalal-Triggs model using astar-structured part-based model defined by a “root” filter(analogous to the Dalal-Triggs filter) plus a set of parts filtersand associated deformation models. The score of one of star modelsat a particular position and scale within an image is the score ofthe root filter at the given location plus the sum over parts of themaximum, over placements of that part, of the part filter score onits location minus a deformation cost easuring the deviation of thepart from its ideal location relative to the root. Both root andpart filter scores are defined by the dot product between a filter(a set of weights) and a subwindow of a feature pyramid computedfrom the input image. Another improvement is a representation of theclass of models by a mixture of star models. The score of a mixturemodel at a particular position and scale is the maximum overcomponents, of the score of that component model at the givenlocation.
In OpenCV there are C implementation of Latent SVM and C++ wrapper of it.C version is the structure CvObjectDetection and a set of functionsworking with this structure (see cvLoadLatentSvmDetector(),cvReleaseLatentSvmDetector(), cvLatentSvmDetectObjects()).C++ version is the class LatentSvmDetector and has slightly differentfunctionality in contrast with C version - it supports loading and detectionof several models.
There are two examples of Latent SVM usage: samples/c/latentsvmdetect.cppand samples/cpp/latentsvm_multidetect.cpp.
CvLSVMFilterPosition
-
struct
CvLSVMFilterPosition
-
Structure describes the position of the filter in the feature pyramid.
-
unsigned int
l
-
level in the feature pyramid
-
unsigned int
x
-
x-coordinate in level l
-
unsigned int
y
-
y-coordinate in level l
-
CvLSVMFilterObject
-
struct
CvLSVMFilterObject
-
Description of the filter, which corresponds to the part of the object.
-
CvLSVMFilterPosition
V
-
ideal (penalty = 0) position of the partial filterfrom the root filter position (V_i in the paper)
-
float
fineFunction[4]
-
vector describes penalty function (d_i in the paper)pf[0] * x + pf[1] * y + pf[2] * x^2 + pf[3] * y^2
-
int
sizeX
-
-
int
sizeY
-
Rectangular map (sizeX x sizeY),every cell stores feature vector (dimension = p)
-
int
numFeatures
-
number of features
-
float*
H
-
matrix of feature vectors to set and getfeature vectors (i,j) used formula H[(j * sizeX + i) * p + k],where k - component of feature vector in cell (i, j)
-
CvLatentSvmDetector
-
struct
CvLatentSvmDetector
-
Structure contains internal representation of trained Latent SVM detector.
-
int
num_filters
-
total number of filters (root plus part) in model
-
int
num_components
-
number of components in model
-
int*
num_part_filters
-
array containing number of part filters for each component
-
CvLSVMFilterObject**
filters
-
root and part filters for all model components
-
float*
b
-
biases for all model components
-
float
score_threshold
-
confidence level threshold
-
CvObjectDetection
-
struct
CvObjectDetection
-
Structure contains the bounding box and confidence level for detected object.
-
CvRect
rect
-
bounding box for a detected object
-
float
score
-
confidence level
-
cvLoadLatentSvmDetector
Loads trained detector from a file.
-
C++:
CvLatentSvmDetector*
cvLoadLatentSvmDetector
(const char*
filename
)
-
Parameters: - filename – Name of the file containing the description of a trained detector
cvReleaseLatentSvmDetector
Release memory allocated for CvLatentSvmDetector structure.
-
C++:
void
cvReleaseLatentSvmDetector
(CvLatentSvmDetector**
detector
)
-
Parameters: - detector – CvLatentSvmDetector structure to be released
cvLatentSvmDetectObjects
Find rectangular regions in the given image that are likely to contain objectsand corresponding confidence levels.
-
C++:
CvSeq*
cvLatentSvmDetectObjects
(IplImage*
image, CvLatentSvmDetector*
detector, CvMemStorage*
storage, float
overlap_threshold=0.5f, int
numThreads=-1
)
-
Parameters: - image – image
- detector – LatentSVM detector in internal representation
- storage – Memory storage to store the resultant sequence of the object candidate rectangles
- overlap_threshold – Threshold for the non-maximum suppression algorithm
- numThreads – Number of threads used in parallel version of the algorithm
LatentSvmDetector
-
class
LatentSvmDetector
-
This is a C++ wrapping class of Latent SVM. It contains internal representation of severaltrained Latent SVM detectors (models) and a set of methods to load the detectors and detect objectsusing them.
LatentSvmDetector::ObjectDetection
-
struct
LatentSvmDetector::
ObjectDetection
-
Structure contains the detection information.
-
Rect
rect
-
bounding box for a detected object
-
float
score
-
confidence level
-
int
classID
-
class (model or detector) ID that detect an object
-
LatentSvmDetector::LatentSvmDetector
Two types of constructors.
-
C++:
LatentSvmDetector::
LatentSvmDetector
(
)
-
-
C++:
LatentSvmDetector::
LatentSvmDetector
(const vector<String>&
filenames, const vector<String>&
classNames=vector<String>()
)
-
Parameters: - filenames – A set of filenames storing the trained detectors (models). Each file contains one model. See examples of such files here /opencv_extra/testdata/cv/latentsvmdetector/models_VOC2007/.
- classNames – A set of trained models names. If it’s empty then the name of each model will be constructed from the name of file containing the model. E.g. the model stored in “/home/user/cat.xml” will get the name “cat”.
LatentSvmDetector::~LatentSvmDetector
Destructor.
-
C++:
LatentSvmDetector::
~LatentSvmDetector
(
)
-
LatentSvmDetector::~clear
Clear all trained models and their names stored in an class object.
-
C++:
void
LatentSvmDetector::
clear
(
)
-
LatentSvmDetector::load
Load the trained models from given .xml files and return true if at least one model was loaded.
-
C++:
bool
LatentSvmDetector::
load
(const vector<String>&
filenames, const vector<String>&
classNames=vector<String>()
)
-
Parameters: - filenames – A set of filenames storing the trained detectors (models). Each file contains one model. See examples of such files here /opencv_extra/testdata/cv/latentsvmdetector/models_VOC2007/.
- classNames – A set of trained models names. If it’s empty then the name of each model will be constructed from the name of file containing the model. E.g. the model stored in “/home/user/cat.xml” will get the name “cat”.
LatentSvmDetector::detect
Find rectangular regions in the given image that are likely to contain objects of loaded classes (models)and corresponding confidence levels.
-
C++:
void
LatentSvmDetector::
detect
(const Mat&
image, vector<ObjectDetection>&
objectDetections, float
overlapThreshold=0.5f, int
numThreads=-1
)
-
Parameters: - image – An image.
- objectDetections – The detections: rectangulars, scores and class IDs.
- overlapThreshold – Threshold for the non-maximum suppression algorithm.
- numThreads – Number of threads used in parallel version of the algorithm.
LatentSvmDetector::getClassNames
Return the class (model) names that were passed in constructor or method load or extracted from models filenames in those methods.
-
C++:
const vector<String>&
LatentSvmDetector::
getClassNames
(
)
const
-
LatentSvmDetector::getClassCount
Return a count of loaded models (classes).
-
C++:
size_t
LatentSvmDetector::
getClassCount
(
)
const
-
[Felzenszwalb2010] | Felzenszwalb, P. F. and Girshick, R. B. and McAllester, D. and Ramanan, D. Object Detection with Discriminatively Trained Part Based Models. PAMI, vol. 32, no. 9, pp. 1627-1645, September 2010 |