模型评估(预测的质量):存在三种方式来评估预测结果的质量
1、Estimator score method:每个估计模型都有自己的评价方式,可以直接使用
2、Scoring parameter:模型评价工具使用cross-validation(
cross_validation.cross_val_score和grid_search.GridSearch)
3、Metric functions: 实现功能评估为特定目的的功能
交叉验证(cross-validation)
1)切分数据:使用train_test_split函数很容易的实现随机的切分形成training和test数据集。
x_train,x_test,y_train,y_test=cross_validation.train_test_split(iris.data,iris.target,test_size=0.4,random_state=0)
clf.svm.SVC(kernel='linear',C=1).fit(x_train,y_train)
clf.score(x_test,y_test)
2)cv:最简单的方式是使用函数cross_val_score,当cv为数字时,默认采用的是KFold或者stratifiedKFold
clf=svm.SVC(kernel='linear',C=1)
scores=cross_validation.cross_val_score(clf,iris.data,iris.target,cv=5)
scores==>得到array([1. ..., 0.96..., 0.9 ..., 0.96..., 1. ])
平均分数和标准偏差估计分数:scores.mean() scores.std()
上面的方式cv的每次迭代都是计算的score,我们可以通过加入scoring参数来计算不同的指标
from sklearn import metrics
cross_validation.cross_val_score(clf,iris.data,iris.target,cv=5,scoring='f1')
==>得到
array([ 1. ..., 0.96..., 0.89..., 0.96..., 1. ])
注:scoring的默认取值以及对应的函数名称
Classification
‘accuracy’
sklearn.metrics.accuracy_score
‘average_precision’
sklearn.metrics.average_precision_score
‘f1’
sklearn.metrics.f1_score f1就是F-measure
‘precision’
sklearn.metrics.precision_score
‘recall’
sklearn.metrics.recall_score
‘roc_auc’
sklearn.metrics.roc_auc_score
Clustering
‘adjusted_rand_score’
sklearn.metrics.adjusted_rand_score
Regression
‘mean_squared_error’
sklearn.metrics.mean_squared_error
>>> from sklearn.metrics import classification_report >>> y_true = [0, 1, 2, 2, 0] >>> y_pred = [0, 0, 2, 2, 0] >>> target_names = ['class 0', 'class 1', 'class 2'] >>> print(classification_report(y_true, y_pred, target_names=target_names)) precision recall f1-score support class 0 0.67 1.00 0.80 2 class 1 0.00 0.00 0.00 1 class 2 1.00 1.00 1.00 2 avg / total 0.67 0.80 0.72 5