Ａre sparse representations really relevant for image classification?

Roberto Rigamonti, Matthew A.Brown, Vincent Lepetit

CVLab, EPFL

The result:

no advantages is gained by imposing sparsity at run-time, at least when this sparsity is not tailored in a discriminative fashion.

Sparsity constaints: <- evidence for sparse representations in th mammal brain

Predictive Sparse Decomposition:an endeavor to avoid the sparsity optimization by learning a regressor that   approximates the

optimal reconstruction.

1. extract features by using filters which are either learned or handcrafted: Olshausen and Field's algorihm(OLS)

Ｍ　ｄｉｃｔｉｏｎａｒｙ, t 坐标

上式只能用在ｓｍａｌｌ patches of images，　在large image 上时需要对Ｍ同时进行优化。因此改成卷积形式。

ｆ:filters　　t：　ｉｍａｇｅｓ with the same size of x

using stochastic gradient descent with clipping to optimize the filters and t

get the filters-> for each image, caculate t:

 three methods:two optimizing method and one convolutional method

2. apply a non-linear operation to the features

ABS or Relu

3.pool the features to obtain robustness to small translations and deformations

Gaussian pooling(Gauss)/average pooling(Boxcaｒ)/Maximum value pooling (MAX)

4. Classification

Nearest neighbor classification(NN)

support vector machines(SVM)

5. Result:

5.1 Recognition rate drops dramatically as sparsity increases when computing features.

5.2A strong sparsity is important in learning the feature extractors, but harmful during classification.

5.3 Increasing sparsity helps  in the denoising of  Gaussian noise, but fails in the case of structured noise.As in latter case , it focuses on the noisy area, and ignores the area with meaningful information

5.4 In architectures that employ pooling stages, sparsity is therefore a temporary condition only. The results after pooling  are dense.