XGBoost ——机器学习竞赛平台（介绍及代码实现）

最新推荐文章于 2023-09-28 14:16:47 发布

五角钱的程序员

最新推荐文章于 2023-09-28 14:16:47 发布

阅读量1.2k

点赞数 1

分类专栏： python系列文章标签：机器学习决策树算法深度学习 python

本文链接：https://blog.csdn.net/youif/article/details/104869960

版权

python系列专栏收录该内容

26 篇文章 5 订阅

订阅专栏

文章目录

XGBoost简介
安装XGBoost
代码

XGBoost简介

XGBoost号称“比赛夺冠的必备大杀器”，横扫机器学习竞赛罕逢敌手，堪称机器学习算法中的新女王！

在涉及非结构化数据（图像、文本等）的预测问题中，人工神经网络显著优于所有其他算法或框架。但当涉及到中小型结构/表格数据时，基于决策树的算法现在被认为是最佳方法。而基于决策树算法中最惊艳的，非XGBoost莫属了。

打过Kaggle、天池、DataCastle、Kesci等国内外数据竞赛平台之后，一定对XGBoost的威力印象深刻。XGBoost号称“比赛夺冠的必备大杀器”，横扫机器学习竞赛罕逢敌手。最近甚至有一位大数据/机器学习主管被XGBoost在项目中的表现惊艳到，盛赞其为“机器学习算法中的新女王”！

XGBoost最初由陈天奇开发。陈天奇是华盛顿大学计算机系博士生，研究方向为大规模机器学习。他曾获得KDD CUP 2012 Track 1第一名，并开发了SVDFeature，XGBoost，cxxnet等著名机器学习工具，是Distributed (Deep) Machine Learning Common的发起人之一。

XGBoost实现了高效、跨平台、分布式gradient boosting (GBDT, GBRT or GBM) 算法的一个库，可以下载安装并应用于C++，Python，R，Julia，Java，Scala，Hadoop等。目前Github上超过15700星、6500个fork。

安装XGBoost

pip install xgboost - i https://pypi.douban.com/simple

使用豆瓣云速度超快的

代码

原始

导入xgboost

import xgboost

导入相关库及数据，数据后面附地址

# First XGBoost model for Pima Indians dataset
from numpy import loadtxt
from xgboost import XGBClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
# load data
dataset = loadtxt('pima-indians-diabetes.csv', delimiter=",")
dataset

在这里插入图片描述
分割数据集，安装属性和标签分别分割

# split data into X and y
X = dataset[:,0:8]
Y = dataset[:,8]
Y

在这里插入图片描述
训练，预测，评估

# split data into train and test sets
seed = 7
test_size = 0.33
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=test_size, random_state=seed)
# fit model no training data
model = XGBClassifier()
model.fit(X_train, y_train)
# make predictions for test data
y_pred = model.predict(X_test)
predictions = [round(value) for value in y_pred]
# evaluate predictions
accuracy = accuracy_score(y_test, predictions)
print("Accuracy: %.2f%%" % (accuracy * 100.0))

在这里插入图片描述

查看训练过程

from numpy import loadtxt
from xgboost import XGBClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
# load data
dataset = loadtxt('pima-indians-diabetes.csv', delimiter=",")
# split data into X and y
X = dataset[:,0:8]
Y = dataset[:,8]
# split data into train and test sets
seed = 7
test_size = 0.33
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=test_size, random_state=seed)
# fit model no training data
model = XGBClassifier()
eval_set = [(X_test, y_test)]
model.fit(X_train, y_train, early_stopping_rounds=10, eval_metric="logloss", eval_set=eval_set, verbose=True)
# make predictions for test data
y_pred = model.predict(X_test)
predictions = [round(value) for value in y_pred]
# evaluate predictions
accuracy = accuracy_score(y_test, predictions)
print("Accuracy: %.2f%%" % (accuracy * 100.0))

在这里插入图片描述

查看特征重要性

from numpy import loadtxt
from xgboost import XGBClassifier
from xgboost import plot_importance
from matplotlib import pyplot
# load data
dataset = loadtxt('pima-indians-diabetes.csv', delimiter=",")
# split data into X and y
X = dataset[:,0:8]
y = dataset[:,8]
# fit model no training data
model = XGBClassifier()
model.fit(X, y)
# plot feature importance
plot_importance(model)
pyplot.show()

在这里插入图片描述

调参查找最好的学习率

# Tune learning_rate
from numpy import loadtxt
from xgboost import XGBClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import StratifiedKFold
# load data
dataset = loadtxt('pima-indians-diabetes.csv', delimiter=",")
# split data into X and y
X = dataset[:,0:8]
Y = dataset[:,8]
# grid search
model = XGBClassifier()
learning_rate = [0.0001, 0.001, 0.01, 0.1, 0.2, 0.3]
param_grid = dict(learning_rate=learning_rate)
kfold = StratifiedKFold(n_splits=10, shuffle=True, random_state=7)
grid_search = GridSearchCV(model, param_grid, scoring="neg_log_loss", n_jobs=-1, cv=kfold)
grid_result = grid_search.fit(X, Y)
# summarize results
print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_))
means = grid_result.cv_results_['mean_test_score']
params = grid_result.cv_results_['params']
for mean, param in zip(means, params):
    print("%f  with: %r" % (mean, param))