tracking_KCF

Kernelized Correlation Filters

João F. Henriques, Rui Caseiro, Pedro Martins, Jorge Batista

ECCV 2012, TPAMI 2015

Qualitative comparison of the proposed KCF tracker with other state-of-the-art trackers, TLD and Struck, on a benchmark of 50 videos. The proposed method is much faster and admits a considerably simpler implementation.

The core component of most modern trackers is a discriminative classifier, tasked with distinguishing between the targetand the surrounding environment. To cope with natural image changes, this classifier is typically trained with translated and scaled sample patches.

Such sets of samples are riddled with redundancies — any overlapping pixels are constrained to be the same. Based on this simple observation, we propose an analytic model for datasets of thousands of translated patches.

By showing that the resulting data matrix is circulant, we can diagonalize it with the Discrete Fourier Transform, reducing both storage and computation by several orders of magnitude. Interestingly, for linear regression our formulation is equivalent to a correlation filter, used by some of the fastest competitive trackers. For kernel regression, however, we derive a new Kernelized Correlation Filter (KCF), that unlike other kernel algorithms has the exact same complexity as its linear counterpart. Building on it, we also propose a fast multi-channel extension of linear correlation filters, via a linear kernel, which we call Dual Correlation Filter (DCF).

Both KCF and DCF outperform top-ranking trackers such as Struck or TLD on a 50 videos benchmark, despite running at hundreds of frames-per-second, and being implemented in a few lines of code. To encourage further developments, our tracking framework was made open-source.

Video. You can view an example video on YouTube here. It is for the ECCV'12 version of the tracker, which only supported a single channel. In the precision plot below, it corresponds to the "KCF on raw pixels" curve. The new version ("KCF/DCF on HOG") is significantly improved.

Source code. Below you can download the Matlab code (both the ECCV'12 and the new version), along with ports to Python (by Rodrigo Benenson), Java (by Peter Abeles), and C++ (by João Faro, Christian Bailer and me; and another port by Tomáš Vojíř).

Precision at different thresholds for 4 variants of the proposed method (in bold) on a 50 videos benchmark.

High-Speed Tracking with Kernelized Correlation Filters
J. F. Henriques, R. Caseiro, P. Martins, J. Batista
TPAMI - IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015

Paper ( PDF)

Preprint on arXiv (2014)

Code version 2 ( Matlab)

Code, multiscale ( C++)

Code, streamlined ( C++)

Reference ( Bibtex)

Exploiting the Circulant Structure of Tracking-by-detection with Kernels
J. F. Henriques, R. Caseiro, P. Martins, J. Batista
ECCV - European Conference on Computer Vision, 2012

Paper ( PDF)

Code version 1 ( Matlab)

Code ( Python)

Code ( Java / doc)

Poster ( PDF)

Supplement ( PDF)

More properties ( PDF)

Reference ( Bibtex)