集成学习——XGBoost

集成学习——XGBoost

1. 导包

from xgboost import XGBRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn import datasets
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.metrics import mean_squared_error

2. 原始数据

# 使用datasets生成数据
data = datasets.load_boston()
X = data.data
y = data.target

3. 数据预处理

# 拆分训练集、测试集
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

4. 构建XGBoost模型

• 构建决策树模型

model = XGBRegressor(n_estimators=100)
model = model.fit(X_train, y_train)

• 结果预测与评分

# 评分
print(model.score(X_test, y_test))
# 结果预测
y_predict = model.predict(X_test)
print(y_predict[0:10])
# 均方差
print(mean_squared_error(y_test, y_predict))

# 评分
0.9427838117247079
# 均方差
5.2137720695838485

• 使用交叉验证

model = XGBRegressor(n_estimators=100)
cvs = cross_val_score(model, X_train, y_train, cv=5)
cvs.mean()

# 评分均值
0.825226867638434

5. 随机森林效果对比

• 随机森林模型构建

model = RandomForestRegressor(n_estimators=100)
model = model.fit(X_train, y_train)

• 结果预测与评分

# 评分
print(model.score(X_test, y_test))
# 结果预测
y_predict = model.predict(X_test)
print(y_predict[0:10])

# 评分
0.9259072259316912