利用树的集成模型分类器RandomForestClassifier/GradientBoostingClassifier进行二类分类(复习6)

本文是个人学习笔记，内容主要涉及树的集成模型随机森林(RandomForest)和梯度提升树(GradientBoostingDecisionTree)对titanic数据集进行二类分类。

集成模型就是综合考量多个分类器的预测结果，再作出决策。“综合考量”的方式大体分2种：（1）按一定次序搭建多个分类模型，后续模型的加入要对现有集成模型的性能有所贡献，从而不断提升更新后的集成模型性能。在每一棵数生成过程尽可能降低整体集成模型在训练集上的拟合误差。（2）对相同数据进行随机样本采样和随机特征采样，同时搭建多个独立的分类模型，最后根据多数投票表决分类。工业界为了追求更加强劲的预测性能，经常使用随机森林分类模型作为Baseline System。

import pandas as pd
titanic=pd.read_csv('http://biostat.mc.vanderbilt.edu/wiki/pub/Main/DataSets/titanic.txt')
titanic.head()

titanic.info()

X=titanic[['pclass','age','sex']]
y=titanic['survived']

X['age'].fillna(X['age'].mean(),inplace=True)

from distutils.version import LooseVersion as Version  
from sklearn import __version__ as sklearn_version  
from sklearn import datasets  
if Version(sklearn_version) < '0.18':  
    from sklearn.cross_validation import train_test_split  
else:  
    from sklearn.model_selection import train_test_split  
X_train,X_test,y_train,y_test = train_test_split(X, y, test_size=0.25, random_state=33)

from sklearn.feature_extraction import DictVectorizer   #特征转换器:将类别型特征独热编码
vec=DictVectorizer(sparse=False)
X_train=vec.fit_transform(pd.DataFrame(X_train).to_dict(orient='record'))   
X_test=vec.transform(pd.DataFrame(X_test).to_dict(orient='record'))

from sklearn.tree import DecisionTreeClassifier
dtc=DecisionTreeClassifier()
dtc.fit(X_train,y_train)
dtc_y_pred=dtc.predict(X_test)

from sklearn.ensemble import RandomForestClassifier
rfc=RandomForestClassifier()
rfc.fit(X_train,y_train)
rfc_y_pred=rfc.predict(X_test)

from sklearn.ensemble import GradientBoostingClassifier
gbc=GradientBoostingClassifier()
gbc.fit(X_train,y_train)
gbc_y_pred=gbc.predict(X_test)

from sklearn.metrics import classification_report
print('The accuracy of decision tree is',dtc.score(X_test,y_test))
print(classification_report(dtc_y_pred,y_test))

print('The accuracy of random forest classifier is',rfc.score(X_test,y_test))
print(classification_report(rfc_y_pred,y_test))

print('The accuracy of gradient tree boosting is',gbc.score(X_test,y_test))
print(classification_report(gbc_y_pred,y_test))