写在前面
scikit-learn 官网的Ensemble methods 文档部分只介绍了/bagging / boosting / voting / 三种模型组合方式;但是通过查找学习,受周志华《机器学习》集成学习部分的学习法启发,了解并学习了 stacking,在此以作记录。
概述
Stacking 是一种集合学习技术,通过元分类器组合多个分类模型。基于完整训练集训练各个分类模型; 然后,基于整体中的各个分类模型的输出 - 元特征来拟合元分类器。元分类器可以根据预测类标签或来自集合的概率进行训练。
流程图:
OR
算法总结:
下面直接上实现过程
环境
- ubantu 16.04 + jupyter + python2.7
- scikit-learn + mlxtend + anconda
示例1.基础StackingClassifier
from sklearn import model_selection
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.ensemble import RandomForestClassifier
from mlxtend.classifier import StackingClassifier
import numpy as np
clf1 = KNeighborsClassifier(n_neighbors=1)
clf2 = RandomForestClassifier(random_state=1)
clf3 = GaussianNB()
lr = LogisticRegression()
sclf = StackingClassifier(classifiers=[clf1, clf2, clf3],
meta_classifier=lr)
print('3-fold cross validation:\n')
for clf, label in zip([clf1, clf2, clf3, sclf],
['KNN',
'Random Forest',
'Naive Bayes',
'StackingClassifier']):
scores = model_selection.cross_val_score(clf, X, y,
cv=3, scoring='accuracy')
print("Accuracy: %0.2f (+/- %0.2f) [%s]"
% (scores.mean(), scores.std(), label))
3-fold cross validation:
Accuracy: 0.91 (+/- 0.01) [KNN]