DW_集成学习_task7

最新推荐文章于 2022-11-10 16:44:21 发布

weixin_46121800

最新推荐文章于 2022-11-10 16:44:21 发布

阅读量100

点赞数

本文链接：https://blog.csdn.net/weixin_46121800/article/details/115663666

版权

该博客介绍了投票法在回归和分类模型中的使用，包括硬投票和软投票策略。硬投票选择出现最多的类别作为预测结果，而软投票则依据概率总和。通过对比不同基模型（如KNN）的效果，展示投票法如何提高模型稳定性，特别是在基模型效果相近且具有不同特性时。代码示例展示了如何在sklearn中实现投票法。

摘要由CSDN通过智能技术生成

投票法思路

本次学习主要讲述了投票法，应用于回归模型以及分类模型，对于回归模型事对所有模型预测结果的平均，分类投票法被划分为硬投票和软投票
- 硬投票：预测结果是所有投票结果最多出现的类
- 软投票：预测结果是所有投票结果中概率加和最大的类
  投票法需要满足以下两个条件:
基模型之间效果不能差别过大
基模型之间应该有较小的同质性；举例：在基模型预测效果近似情况下，基于树模型与线性模型的投票，往往优于两个树模型或两个线性模型
代码：

from sklearn.datasets import make_classification
from sklearn.ensemble import VotingClassifier
from sklearn.model_selection import RepeatedStratifiedKFold, cross_val_score
from sklearn.neighbors import KNeighborsClassifier
from matplotlib import pyplot
import numpy as np
# define dataset
def get_dataset():
	X, y = make_classification(n_samples=1000, n_features=20, n_informative=15, n_redundant=5, random_state=2)
	return X,y
#summarize the dataset
#print(X.shape, y.shape)

# get a voting ensemble of models
def get_voting():
	# define the base models
	models = list()
	models.append(('knn1', KNeighborsClassifier(n_neighbors=1)))
	models.append(('knn3', KNeighborsClassifier(n_neighbors=3)))
	models.append(('knn5', KNeighborsClassifier(n_neighbors=5)))
	models.append(('knn7', KNeighborsClassifier(n_neighbors=7)))
	models.append(('knn9', KNeighborsClassifier(n_neighbors=9)))
	# define the voting ensemble
	ensemble = VotingClassifier(estimators=models, voting='hard')
	return ensemble

def get_models():
	models = dict()
	models['knn1'] = KNeighborsClassifier(n_neighbors=1)
	models['knn3'] = KNeighborsClassifier(n_neighbors=3)
	models['knn5'] = KNeighborsClassifier(n_neighbors=5)
	models['knn7'] = KNeighborsClassifier(n_neighbors=7)
	models['knn9'] = KNeighborsClassifier(n_neighbors=9)
	models['hard_voting'] = get_voting()
	return models

def evaluate_model(model, X, y):
	cv = RepeatedStratifiedKFold(n_splits=10, n_repeats=3, random_state=1)
	scores = cross_val_score(model, X, y, scoring='accuracy', cv=cv, n_jobs=-1, error_score='raise')
	return scores

# define dataset
X, y = get_dataset()
# get the models to evaluate
models = get_models()
# evaluate the models and store results
results, names = list(), list()
for name, model in models.items():
	scores = evaluate_model(model, X, y)
	results.append(scores)
	names.append(name)
	print('>%s %.3f (%.3f)' % (name, np.mean(scores), np.std(scores)))
# plot model performance for comparison
pyplot.boxplot(results, labels=names, showmeans=True)
pyplot.show()