利用树的集成回归模型RandomForestRegressor/ExtraTreesRegressor/GradientBoostingRegressor进行回归预测(复习11)

本文是个人学习笔记，内容主要基于树的三种集成回归模型RandomForestRegressor，ExtraTreesRegressor以及GradientBoostingRegressor对boston数据集学习回归模型和利用模型预测。

普通的随机森林模型构建的分裂节点时是随机选取特征的，极端随机森林在构建每一棵树的分裂节点时，不会任意的选取特征，而是先随机收集一部分特征，然后利用信息熵/基尼指数挑选最佳的节点特征。

from sklearn.datasets import load_boston
boston=load_boston()
print(boston.DESCR)   #打印数据描述

from sklearn.cross_validation import train_test_split
import numpy as np
X=boston.data
y=boston.target
X_train,X_test,y_train,y_test=train_test_split(X,y,random_state=33,test_size=0.25)

print('The max target value is',np.max(boston.target))
print('The min target value is',np.min(boston.target))
print('The average target value is',np.mean(boston.target))

from sklearn.preprocessing import StandardScaler

ss_X=StandardScaler()
ss_y=StandardScaler()

X_train=ss_X.fit_transform(X_train)
X_test=ss_X.transform(X_test)
y_train=ss_y.fit_transform(y_train.reshape(-1, 1))
y_test=ss_y.transform(y_test.reshape(-1, 1))

from sklearn.ensemble import RandomForestRegressor,ExtraTreesRegressor,GradientBoostingRegressor

rfr=RandomForestRegressor()
rfr.fit(X_train,y_train)
rfr_y_predict=rfr.predict(X_test)

etr=ExtraTreesRegressor()
etr.fit(X_train,y_train)
etr_y_predict=etr.predict(X_test)

gbr=GradientBoostingRegressor()
gbr.fit(X_train,y_train)
gbr_y_predict=gbr.predict(X_test)

from sklearn.metrics import r2_score,mean_absolute_error,mean_squared_error
print('R-squared value of RandomForestRegressor:',rfr.score(X_test,y_test))
print('The mean squared error of RandomForestRegressor:',mean_squared_error(ss_y.inverse_transform(y_test),
                                                                            ss_y.inverse_transform(rfr_y_predict)))
print('The mean absolute error of RandomForestRegressor:',mean_absolute_error(ss_y.inverse_transform(y_test),
                                                                              ss_y.inverse_transform(rfr_y_predict)))
print(' ')
print('R-squared value of ExtraTreesRegressor:',etr.score(X_test,y_test))
print('The mean squared error of ExtraTreesRegressor:',mean_squared_error(ss_y.inverse_transform(y_test),
                                                                          ss_y.inverse_transform(etr_y_predict)))
print('The mean absolute error of ExtraTreesRegressor:',mean_absolute_error(ss_y.inverse_transform(y_test),
                                                                            ss_y.inverse_transform(etr_y_predict)))
print(' ')
print('R-squared value of GradientBoostingRegressor:',gbr.score(X_test,y_test))
print('The mean squared error of GradientBoostingRegressor:',mean_squared_error(ss_y.inverse_transform(y_test),
                                                                                ss_y.inverse_transform(gbr_y_predict)))
print('The mean absolute error of GradientBoostingRegressor:',mean_absolute_error(ss_y.inverse_transform(y_test),
                                                                                  ss_y.inverse_transform(gbr_y_predict)))

很多在工业界的工作者更加青睐集成模型，经常以集成模型的性能表现为基准，与新设计的其他模型性能进行比对。