scikit-learn：3.3. Model evaluation: quantifying the quality of predictions

参考：http://scikit-learn.org/stable/modules/model_evaluation.html#scoring-parameter

三种方法评估模型的预测质量：

• Estimator score method: Estimators都有 score method作为默认的评估标准，不属于本节内容，具体参考不同estimators的文档。
• Scoring parameter: Model-evaluation toolsusing cross-validation (such as cross_validation.cross_val_score and grid_search.GridSearchCV) rely on an internal scoring strategy. 本节讨论The scoring parameter: defining model evaluation rules.（参考第一小节）
• Metric functions: The metrics module 能较全面评价预测质量，本节讨论Classification metrics, Multilabel ranking metrics, Regression metrics and Clustering metrics.（参考二、三、四、五小节）

最后介绍 Dummy estimators ，提供随机猜测的策略，可以作为预测质量评价的baseline。（参考第六小节）

See also

 

For “pairwise” metrics, between samples and not estimators or predictions, see the Pairwise metrics, Affinities and Kernels section.

具体内容有时间再写。。。

1、

The  scoring parameter: defining model evaluation rules

Model selection and evaluation using tools, such as grid_search.GridSearchCV and cross_validation.cross_val_score, take a scoring parameter that controls what metric they apply to the estimators evaluated.

1）预定义的标准

所有的scorer都是越大越好，因此mean_absolute_error and mean_squared_error（测量预测点离模型的距离）是负值。

Scoring	Function	Comment
Classification
‘accuracy’	metrics.accuracy_score
‘average_precision’	metrics.average_precision_score
‘f1’	metrics.f1_score	for binary targets
‘f1_micro’	metrics.f1_score	micro-averaged
‘f1_macro’	metrics.f1_score	macro-averaged
‘f1_weighted’	metrics.f1_score	weighted average
‘f1_samples’	metrics.f1_score	by multilabel sample
‘log_loss’	metrics.log_loss	requires predict_proba support
‘precision’ etc.	metrics.precision_score	suffixes apply as with ‘f1’
‘recall’ etc.	metrics.recall_score	suffixes apply as with ‘f1’
‘roc_auc’	metrics.roc_auc_score
Clustering
‘adjusted_rand_score’	metrics.adjusted_rand_score
Regression
‘mean_absolute_error’	metrics.mean_absolute_error
‘mean_squared_error’	metrics.mean_squared_error
‘median_absolute_error’	metrics.median_absolute_error