特征选择Feature Selection

1. Definition

      In machine learning and statistics, feature selection, also known as variable selection, attribute selection or variable subset selection, is the process of selecting a subset of relevant features for use in model construction, along with anevaluation measure which scores the different feature subsets.

     The choice of evaluation metric heavily influences the algorithm, and it is these evaluation metrics which distinguish between the three main categories of feature selection algorithms:

• wrappers
• filters
• embedded methods

     There are 5 main type of evaluation functions:

• distance(euclidean distance measure)                             -------------->filter
• information(entropy, information gain, etc.)                       -------------->filter
• dependency(Mutual Information,correlation coefficient)  -------------->filter
• consistency(min-features bias)                                           -------------->filter
• classifier error rate(the classifier themselves)                 -------------->wrapper

2. Machine Learning Methods of Feature Selection for Classification

• online boosting (online boosting and vision)
• decision tree/random forest

3. Feature Selection for Retrieval/Match

• maximun conditional entropy[3]
• Conditional Mutual Information Maximization[4]
• binary feature like BRIEF, Brisk, ORB, D-Brief等等

 

4. Reference

[1] http://en.wikipedia.org/wiki/Feature_selection

[2] Feature Selection for Classification (google)

[3] Real-time Large Scale Near-duplicate Web Video Retrieval

[4]  Fast Binary Feature Selection withConditional Mutual Information