【xgboost】XGBoost 调参

已于 2023-08-13 00:35:12 修改
阅读量368 收藏 4
点赞数
分类专栏： Python 文章标签： sklearn xgboost python 机器学习
于 2023-08-12 18:58:30 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_45249685/article/details/132252200
版权

1. 原生api 训练、网格搜索

1.1 数据

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

data = load_breast_cancer()
print(data['data'].shape, data['target'].shape)
x_data = data.data
y_data = data.target
x_train, x_test, y_train, y_test = train_test_split(x_data, y_data, test_size=0.2, random_state=666)

dtrain = xgb.DMatrix(x_train, y_train)
dtest = xgb.DMatrix(x_test, y_test)
'''
(569, 30) (569,)
'''

1.2 设置参数 & 训练

import xgboost as xgb


params = {'objective': 'binary:logistic',
          'max_depth': 5,
          'min_child_weight': 0.8,
          'verbosity': 0,
          'subsample': 0.8,
          'colsample_bytree': 0.8,
          'gamma': 0.1,
          'lambda': 0.8,
          'eta': 1}

num_round = 50
watch_list = [(dtrain, 'train'), (dtest, 'test')]
evals_result = {}

xgb_model = xgb.train(
    params=params,
    dtrain=dtrain,
    num_boost_round=num_round,
    evals=watch_list,
    early_stopping_rounds=10,
    evals_result=evals_result
)
'''
[0]	train-logloss:0.19172	test-logloss:0.30370
[1]	train-logloss:0.09322	test-logloss:0.22359
[2]	train-logloss:0.05241	test-logloss:0.19594
[3]	train-logloss:0.03034	test-logloss:0.21439
[4]	train-logloss:0.02134	test-logloss:0.20752
[5]	train-logloss:0.01631	test-logloss:0.20046
[6]	train-logloss:0.01476	test-logloss:0.21509
[7]	train-logloss:0.01407	test-logloss:0.20678
[8]	train-logloss:0.01231	test-logloss:0.20136
[9]	train-logloss:0.01197	test-logloss:0.20550
[10]	train-logloss:0.01053	test-logloss:0.19675
[11]	train-logloss:0.00992	test-logloss:0.20746
'''

preds = xgb_model.predict(dtest)
# 也可直接用不带标签的特征数据
# preds = xgb_model.predict(xgb.DMatrix(x_test))
labels = dtest.get_label()
labels, preds
'''
array([0., 1., 1., 1., 0., 0., 0., 1., 0., 1., 1., 1., 1., 1., 1., 1., 1.,
       1., 1., 1., 0., 0., 1., 1., 1., 1., 1., 0., 1., 0., 0., 1., 1., 0.,
       0., 1., 1., 0., 0., 1., 0., 1., 0., 1., 1., 0., 1., 1., 0., 1., 0.,
       0., 0., 1., 1., 0., 1., 1., 1., 0., 1., 0., 0., 0., 1., 1., 1., 1.,
       0., 0., 1., 1., 0., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 1., 0.,
       1., 1., 0., 1., 1., 1., 1., 1., 0., 1., 1., 0., 1., 0., 1., 1., 0.,
       0., 1., 0., 1., 0., 0., 1., 1., 0., 1., 1., 0.], dtype=float32)
array([2.3674986e-03, 9.9861741e-01, 9.9975663e-01, 2.1583840e-01,
       1.1854486e-03, 6.3544436e-04, 1.2252343e-04, 9.9900925e-01,
       6.5811968e-05, 9.9750012e-01, 9.9970168e-01, 9.9982613e-01,
       1.4022045e-01, 9.8603803e-01, 9.6359092e-01, 9.9929202e-01,
       9.9949872e-01, 9.9832839e-01, 9.9978691e-01, 9.9982613e-01,
       6.3544436e-04, 6.2098140e-03, 9.9957401e-01, 9.9969757e-01,
       9.9982423e-01, 9.9957675e-01, 9.9975401e-01, 1.4727229e-04,
       9.9970168e-01, 1.3158246e-04, 1.0680066e-03, 9.9991202e-01,
       9.9991202e-01, 1.0036851e-01, 1.4891458e-04, 6.8655396e-01,
       9.9900925e-01, 6.5541302e-04, 8.1419773e-02, 9.9970526e-01,
       6.5962277e-02, 9.9671751e-01, 1.5763033e-04, 9.9976224e-01,
       1.8256794e-01, 1.4891458e-04, 9.9982613e-01, 9.9935108e-01,
       6.6271797e-04, 9.9606222e-01, 9.9767417e-01, 6.6271797e-04,
       6.6271797e-04, 9.9754131e-01, 9.9975401e-01, 7.4481402e-05,
       9.0036017e-01, 3.5478315e-01, 9.9987698e-01, 1.1854486e-03,
       9.9930811e-01, 3.8913928e-02, 9.9082899e-01, 7.0148520e-04,
       9.9900925e-01, 9.9870503e-01, 9.9982613e-01, 9.6617037e-01,
       1.5763033e-04, 1.8701397e-01, 9.9964190e-01, 9.9982613e-01,
       2.0063692e-03, 9.9939549e-01, 8.6724478e-01, 9.9991202e-01,
       9.9935108e-01, 9.7720438e-01, 2.3587022e-03, 1.5763033e-04,
       5.4037949e-04, 4.3376945e-03, 7.8841185e-05, 9.9982423e-01,
       3.1165761e-04, 9.6175414e-01, 9.9949872e-01, 1.4727229e-04,
       9.9987698e-01, 9.9982613e-01, 9.9797589e-01, 9.9478018e-01,
       9.4439185e-01, 1.3158246e-04, 1.5761836e-01, 9.9597138e-01,
       9.6866369e-01, 9.9949872e-01, 7.1160225e-03, 9.9847311e-01,
       9.9786395e-01, 5.4037949e-04, 1.1854486e-03, 9.9964190e-01,
       1.8933583e-02, 9.9413067e-01, 7.8841185e-05, 1.3096399e-03,
       9.9991202e-01, 9.9705601e-01, 8.6442098e-02, 9.3267983e-01,
       9.9922729e-01, 1.6699207e-04], dtype=float32)

'''

error = sum([1 for i in range(len(preds)) if int(preds[i] > 0.5) != labels[i]]) / float(len(preds))

print(f'error={error: .2f}')
'''
error= 0.07
'''

''' 
回归
'objective': 'reg:squarederror', # 平方误差目标函数
np.sqrt(mean_squared_error(preds, labels)) # rmse
'''

1.3 网格搜索调参

1.3.1 外套sklearn api⭐⭐⭐

import xgboost as xgb


# 自定义类，实现外套sklearn api
class MyXGBoost:
    def __init__(self, **params):
        self.params = params
        if 'num_boost_round' in self.params:
            self.num_boost_round = self.params['num_boost_round']

        # 默认回归，分类需要修改objective
        self.params.update({'verbosity': 1,
                            'objective':'reg:squarederror',
                            'seed':0})
        self.bst = None
    
    def fit(self, x_train, y_train):
        dtrain = xgb.DMatrix(x_train, y_train)
        self.bst = xgb.train(params=self.params
                             ,dtrain=dtrain
                             ,num_boost_round=self.num_boost_round
                             )
        
    def predict(self, x_pred):
        dpred = xgb.DMatrix(x_pred)
        return self.bst.predict(dpred)

    def kfold(self, x_train, y_train, nfold=5):
        dtrain = xgb.DMatrix(x_train, y_train)
        cv_round = xgb.cv(params=self.params
                          ,dtrain=dtrain
                          ,num_boost_round=self.num_boost_round
                          ,nfold=nfold
                          ,early_stopping_rounds=10
                          )
        
        return cv_round.iloc[-1, :]
    
    def get_params(self, deep=True):
        return self.params
    
    def set_params(self, **params):
        self.params.update(params)
        return self

1.3.2 自定义目标函数

from sklearn.metrics import mean_squared_error
from sklearn.metrics import make_scorer
import numpy as np

def score_fn(y_pred, y_true):
    return np.sqrt(mean_squared_error(y_pred, y_true))

# rmse 值越小拟合越好
cv_score_fn = make_scorer(score_func=score_fn
                          ,greater_is_better=False)

1.3.3 设置搜索参数 & 初次搜索

from sklearn.model_selection import GridSearchCV, RandomizedSearchCV


params_grid = {
    'max_depth': list([4, 5, 6, 7])
    ,'min_child_weight': list([1, 5, 9])
    ,'subsample': list([0.6, 0.8, 1])
    ,'colsample_bytree': list([0.6, 0.8, 1])
}

xgb_model = MyXGBoost(num_boost_round=20)

grid = GridSearchCV(estimator=xgb_model
                    ,param_grid=params_grid
                    ,scoring=cv_score_fn
                    ,cv=3
                    ,n_jobs=-1)
grid.fit(x_train, y_train)

print(grid.best_params_)

不知道为什么 Parameters: { "num_boost_round" } are not used.

'''
[17:04:14] WARNING: C:\Users\dev-admin\croot2\xgboost-split_1675461376218\work\src\learner.cc:767: 
Parameters: { "num_boost_round" } are not used.

{'colsample_bytree': 1, 'max_depth': 5, 'min_child_weight': 9, 'subsample': 1}
'''

1.3.4 确定部分参数 & 进一步搜索

params = {
    'max_depth': 5
    ,'min_child_weight': 9
    ,'subsample': 1
    ,'colsample_bytree':1
    ,'num_boost_round': 20
}

xgb_model = MyXGBoost(**params)

params_grid = {
    'gamma': list([0, 0.1, 0.5])
    ,'lambda': list([1, 1.5])
    ,'eta': list([0.3, 0.5, 1])
}
grid = GridSearchCV(estimator=xgb_model
                    ,param_grid=params_grid
                    ,scoring=cv_score_fn
                    ,cv=3
                    ,n_jobs=-1)
grid.fit(x_train, y_train)

print(grid.best_params_)

'''
[17:04:15] WARNING: C:\Users\dev-admin\croot2\xgboost-split_1675461376218\work\src\learner.cc:767: 
Parameters: { "num_boost_round" } are not used.

{'eta': 0.3, 'gamma': 0, 'lambda': 1}
'''

1.4 循环、交叉验证调参

import xgboost as xgb

params = {
    'colsample_bytree': 1,
    'max_depth': 7,
    'min_child_weight': 5,
    'subsample': 0.8
}
etas = [0.3, 0.5, 1]
num_boost_rounds = [10, 15, 20]
nfold = 3

best_eta, best_round = 0, 0
best_score = float('inf')

for eta in etas:
    for num_boost_round in num_boost_rounds:
        params.update({'eta': eta})
        dtrain = xgb.DMatrix(x_train, y_train)

        cv_rounds = xgb.cv(params=params
                           ,dtrain=dtrain
                           ,num_boost_round=num_boost_round
                           ,nfold=nfold
                           ,early_stopping_rounds=10
                           )
        # 返回num_boost_round轮评估指标的train-rmse-mean,train-rmse-std,test-rmse-mean,test-rmse-std
        # 选取最后一轮的test-rmse-mean
        score = cv_rounds.iloc[-1, -2]

        if score < best_score:
            best_score = score
            best_eta = eta
            best_round = num_boost_round

print(f'eta: {best_eta}, num_boost_round: {best_round}')
'''
eta: 0.5, num_boost_round: 15
'''

2. sklearn api 调参

2.1 数据

from sklearn.datasets import load_breast_cancer


data = load_breast_cancer()
X = data.data
y = data.target
Xtrain, Xtest, ytrain, ytest = train_test_split(X, y, test_size=0.2, random_state=98)
print(Xtrain.shape, Xtest.shape)
'''
(455, 30) (114, 30)
'''

2.2 网格搜索

from sklearn.model_selection import train_test_split
from sklearn.model_selection import KFold

from xgboost import XGBClassifier

from sklearn.metrics import mean_squared_error
from sklearn.metrics import make_scorer
import numpy as np

def score_fn(y_pred, y_true):
    return np.sqrt(mean_squared_error(y_pred, y_true))

# rmse 值越小拟合越好
cv_score_fn = make_scorer(score_func=score_fn
                          ,greater_is_better=False)

from sklearn.model_selection import GridSearchCV

param_grid = {
    'learning_rate': list([0.1, 0.2, 0.3])
    ,'n_estimators': list([50, 60, 70])
    ,'max_depth': list([4, 5, 6, 7])
    ,'min_child_weight': list([0.6, 0.8, 1])
    ,'subsample': list([0.8])
    ,'colsample_bytree': [0.8, 0.9]
}

xgbc = XGBClassifier(objective='binary:logistic')
cv = KFold(n_splits=5, shuffle=True, random_state=98)
grid = GridSearchCV(xgbc, 
                    param_grid=param_grid,
                    cv=cv,
                    scoring=cv_score_fn,
                    n_jobs=-1)
grid.fit(Xtrain, ytrain)

grid.best_params_
'''
{'colsample_bytree': 0.8,
 'learning_rate': 0.3,
 'max_depth': 6,
 'min_child_weight': 0.6,
 'n_estimators': 70,
 'subsample': 0.8}
'''

xgb_model = grid.best_estimator_
ypreds = xgb_model.predict(Xtest)

error = sum([1 for i in range(len(ypreds)) if int(ypreds[i] > 0.5) != ytest[i]]) / float(len(ypreds))
error
'''
0.03508771929824561
'''

2.3 optuna

from sklearn.model_selection import KFold
from xgboost import XGBRegressor
import optuna

from sklearn.metrics import mean_squared_error
from sklearn.metrics import make_scorer
import numpy as np

def score_fn(y_pred, y_true):
    return np.sqrt(mean_squared_error(y_pred, y_true))

# rmse 值越小拟合越好
cv_score_fn = make_scorer(score_func=score_fn
                          ,greater_is_better=False)

def objective(trial):

    params = {
        'n_estimators': trial.suggest_categorical('n_estimators',[5, 20, 50, 100])
        ,'max_depth': trial.suggest_categorical('max_depth',[3, 5, 10, 20])
        ,'learning_rate': trial.suggest_categorical('learning_rate',[0.01, 0.1, 0.3])
        ,'subsample': trial.suggest_categorical('subsample',[0.5, 0.8, 1])
        ,'colsample_bytree': trial.suggest_categorical('colsample_bytree',[0.5, 0.8, 1])
    }

    xgbr.set_params(**params)
    
    kf = KFold(n_splits = 10, shuffle = True, random_state = 42)
    scores = []
    for train_idx, test_idx in kf.split(Xtrain, ytrain):

        X_train_round, y_train_round = Xtrain[train_idx], ytrain[train_idx]
        X_test_round, y_test_round = Xtrain[test_idx], ytrain[test_idx]
        xgbr.fit(X_train_round, y_train_round)

        score = score_fn(y_test_round, xgbr.predict(X_test_round))
        scores.append(score)
    
    return np.mean(scores)

xgbr = XGBRegressor(random_state=42)

study = optuna.create_study(direction = 'minimize')
study.optimize(objective, n_trials = 50)

best_params = study.best_params
xgbr.set_params(**best_params)
xgbr.fit(Xtrain, ytrain)

ypreds = xgbr.predict(Xtest)
error = sum([1 for i in range(len(ypreds)) if int(ypreds[i] > 0.5) != ytest[i]]) / float(len(ypreds))
'''
[I 2023-08-13 00:24:31,687] A new study created in memory with name: no-name-55579fda-c6eb-44dd-af61-e942096f9e76
[I 2023-08-13 00:24:31,779] Trial 0 finished with value: 0.47942750589521343 and parameters: {'n_estimators': 5, 'max_depth': 5, 'learning_rate': 0.01, 'subsample': 0.8, 'colsample_bytree': 1}. Best is trial 0 with value: 0.47942750589521343.
[I 2023-08-13 00:24:32,029] Trial 1 finished with value: 0.19397620629291293 and parameters: {'n_estimators': 20, 'max_depth': 20, 'learning_rate': 0.1, 'subsample': 0.8, 'colsample_bytree': 0.8}. Best is trial 1 with value: 0.19397620629291293.
[I 2023-08-13 00:24:32,101] Trial 2 finished with value: 0.19759044486258567 and parameters: {'n_estimators': 5, 'max_depth': 5, 'learning_rate': 0.3, 'subsample': 0.8, 'colsample_bytree': 0.8}. Best is trial 1 with value: 0.19397620629291293.
[I 2023-08-13 00:24:32,685] Trial 3 finished with value: 0.25242426092514947 and parameters: {'n_estimators': 100, 'max_depth': 5, 'learning_rate': 0.01, 'subsample': 0.5, 'colsample_bytree': 0.8}. Best is trial 1 with value: 0.19397620629291293.
[I 2023-08-13 00:24:32,749] Trial 4 finished with value: 0.20931217440968805 and parameters: {'n_estimators': 5, 'max_depth': 20, 'learning_rate': 0.3, 'subsample': 1, 'colsample_bytree': 0.5}. Best is trial 1 with value: 0.19397620629291293.
[I 2023-08-13 00:24:32,897] Trial 5 finished with value: 0.2148347864283057 and parameters: {'n_estimators': 20, 'max_depth': 5, 'learning_rate': 0.3, 'subsample': 1, 'colsample_bytree': 1}. Best is trial 1 with value: 0.19397620629291293.
[I 2023-08-13 00:24:33,150] Trial 6 finished with value: 0.33977384268310445 and parameters: {'n_estimators': 50, 'max_depth': 10, 'learning_rate': 0.01, 'subsample': 0.5, 'colsample_bytree': 0.5}. Best is trial 1 with value: 0.19397620629291293.
[I 2023-08-13 00:24:33,770] Trial 7 finished with value: 0.183987741509823 and parameters: {'n_estimators': 100, 'max_depth': 20, 'learning_rate': 0.3, 'subsample': 0.8, 'colsample_bytree': 0.8}. Best is trial 7 with value: 0.183987741509823.
[I 2023-08-13 00:24:33,834] Trial 8 finished with value: 0.33880373171511113 and parameters: {'n_estimators': 5, 'max_depth': 5, 'learning_rate': 0.1, 'subsample': 1, 'colsample_bytree': 1}. Best is trial 7 with value: 0.183987741509823.
[I 2023-08-13 00:24:34,177] Trial 9 finished with value: 0.18608609798364395 and parameters: {'n_estimators': 100, 'max_depth': 3, 'learning_rate': 0.3, 'subsample': 1, 'colsample_bytree': 0.5}. Best is trial 7 with value: 0.183987741509823.
[I 2023-08-13 00:24:34,809] Trial 10 finished with value: 0.183987741509823 and parameters: {'n_estimators': 100, 'max_depth': 20, 'learning_rate': 0.3, 'subsample': 0.8, 'colsample_bytree': 0.8}. Best is trial 7 with value: 0.183987741509823.
[I 2023-08-13 00:24:35,432] Trial 11 finished with value: 0.183987741509823 and parameters: {'n_estimators': 100, 'max_depth': 20, 'learning_rate': 0.3, 'subsample': 0.8, 'colsample_bytree': 0.8}. Best is trial 7 with value: 0.183987741509823.
[I 2023-08-13 00:24:36,064] Trial 12 finished with value: 0.183987741509823 and parameters: {'n_estimators': 100, 'max_depth': 20, 'learning_rate': 0.3, 'subsample': 0.8, 'colsample_bytree': 0.8}. Best is trial 7 with value: 0.183987741509823.
[I 2023-08-13 00:24:36,696] Trial 13 finished with value: 0.183987741509823 and parameters: {'n_estimators': 100, 'max_depth': 20, 'learning_rate': 0.3, 'subsample': 0.8, 'colsample_bytree': 0.8}. Best is trial 7 with value: 0.183987741509823.
[I 2023-08-13 00:24:37,128] Trial 14 finished with value: 0.18313284638690286 and parameters: {'n_estimators': 50, 'max_depth': 10, 'learning_rate': 0.3, 'subsample': 0.8, 'colsample_bytree': 0.8}. Best is trial 14 with value: 0.18313284638690286.
[I 2023-08-13 00:24:37,540] Trial 15 finished with value: 0.1817555147315062 and parameters: {'n_estimators': 50, 'max_depth': 10, 'learning_rate': 0.1, 'subsample': 0.8, 'colsample_bytree': 0.8}. Best is trial 15 with value: 0.1817555147315062.
[I 2023-08-13 00:24:37,916] Trial 16 finished with value: 0.17797282645596724 and parameters: {'n_estimators': 50, 'max_depth': 10, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 0.8}. Best is trial 16 with value: 0.17797282645596724.
[I 2023-08-13 00:24:38,277] Trial 17 finished with value: 0.17797282645596724 and parameters: {'n_estimators': 50, 'max_depth': 10, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 0.8}. Best is trial 16 with value: 0.17797282645596724.
[I 2023-08-13 00:24:38,654] Trial 18 finished with value: 0.1796387060606775 and parameters: {'n_estimators': 50, 'max_depth': 10, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 1}. Best is trial 16 with value: 0.17797282645596724.
[I 2023-08-13 00:24:38,994] Trial 19 finished with value: 0.1726048235818686 and parameters: {'n_estimators': 50, 'max_depth': 10, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 0.5}. Best is trial 19 with value: 0.1726048235818686.
[I 2023-08-13 00:24:39,194] Trial 20 finished with value: 0.17321528816321324 and parameters: {'n_estimators': 50, 'max_depth': 3, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 0.5}. Best is trial 19 with value: 0.1726048235818686.
[I 2023-08-13 00:24:39,390] Trial 21 finished with value: 0.17321528816321324 and parameters: {'n_estimators': 50, 'max_depth': 3, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 0.5}. Best is trial 19 with value: 0.1726048235818686.
[I 2023-08-13 00:24:39,590] Trial 22 finished with value: 0.17321528816321324 and parameters: {'n_estimators': 50, 'max_depth': 3, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 0.5}. Best is trial 19 with value: 0.1726048235818686.
[I 2023-08-13 00:24:39,793] Trial 23 finished with value: 0.17321528816321324 and parameters: {'n_estimators': 50, 'max_depth': 3, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 0.5}. Best is trial 19 with value: 0.1726048235818686.
...
[I 2023-08-13 00:24:44,512] Trial 46 finished with value: 0.17738508610056414 and parameters: {'n_estimators': 50, 'max_depth': 5, 'learning_rate': 0.1, 'subsample': 1, 'colsample_bytree': 0.5}. Best is trial 19 with value: 0.1726048235818686.
[I 2023-08-13 00:24:44,671] Trial 47 finished with value: 0.18466580983642228 and parameters: {'n_estimators': 20, 'max_depth': 5, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 0.5}. Best is trial 19 with value: 0.1726048235818686.
[I 2023-08-13 00:24:45,027] Trial 48 finished with value: 0.17881540967668666 and parameters: {'n_estimators': 50, 'max_depth': 5, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 1}. Best is trial 19 with value: 0.1726048235818686.
[I 2023-08-13 00:24:45,607] Trial 49 finished with value: 0.17358566430619748 and parameters: {'n_estimators': 100, 'max_depth': 5, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 0.5}. Best is trial 19 with value: 0.1726048235818686.
'''

print(best_params)
print(error)
print(study.best_trial)
'''
{'n_estimators': 50, 'max_depth': 10, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 0.5}
0.07017543859649122
FrozenTrial(number=19, state=TrialState.COMPLETE, values=[0.1726048235818686], datetime_start=datetime.datetime(2023, 8, 13, 0, 24, 38, 654382), datetime_complete=datetime.datetime(2023, 8, 13, 0, 24, 38, 994747), params={'n_estimators': 50, 'max_depth': 10, 'learning_rate': 0.1, 'subsample': 0.5, 'colsample_bytree': 0.5}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'n_estimators': CategoricalDistribution(choices=(5, 20, 50, 100)), 'max_depth': CategoricalDistribution(choices=(3, 5, 10, 20)), 'learning_rate': CategoricalDistribution(choices=(0.01, 0.1, 0.3)), 'subsample': CategoricalDistribution(choices=(0.5, 0.8, 1)), 'colsample_bytree': CategoricalDistribution(choices=(0.5, 0.8, 1))}, trial_id=19, value=None)
'''
rejudge
关注
  • 0
    点赞
  • 踩
  • 4
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
xgboost 调参经验
liulina603的专栏
12-01 6596
1.xgboost 基本方法和默认参数 在训练过程中主要用到两个方法：xgboost.train()和xgboost.cv(). #xgboost.train()API xgboost.train(params,dtrain,num_boost_round=10,evals=(),obj=None,feval=None,maximize=False,early_stopping_r
XGboost-网格调参法
zllnau66的博客
08-23 1万+
一、原理 个人认为写的通俗易懂的一篇好文： https://www.jianshu.com/p/7467e616f227 二、网格调参 用xgboost既可以用来做二分类、多分类，也可以用来做回归预测数值，除了特征之外，影响模型的是如何调参了，一般是按一定的步骤、网格搜索最优参数，如下两篇文章一个是用来分类，一个是用来预测数值的案例，并且详细给出了调参的步骤和代码： https://bl...
【模型参数优化】网格搜索对XGBoost分类模型进行参数寻优【附python实现代码】
学无止境、积少成多、厚积薄发
05-05 1487
网格搜索对XGBoost分类模型进行参数寻优【附python实现代码】
【模型参数优化】网格搜索对随机森林分类模型进行参数寻优【附python实现代码】
学无止境、积少成多、厚积薄发
05-04 1248
网格搜索对随机森林分类模型进行参数寻优【附python实现代码】
XGBoost实战与调优
法相的博客
07-29 4892
首先，python和Anaconda都没有自带xgboost。windows下安装xgboost非常方便。在前面的文章中，提供了下载地址和详细的安装步骤。 你可以在python中，输入import xgboost来检查是否安装成功。 下面，我们正式进入xgboost实战与效果检验。在前面的文章《决策树模型调优》中，我们通过PCA，特征选择，简单的特征组合将决策树模型 预测广告的准确率从最初的93.3%提高到95.6%。那么，是否还能进一步提高呢，答案的是肯定的。接下来就来看看xg
【MATLAB第37期】 #保姆级教程 XGBOOST模型参数完整且详细介绍，调参范围、思路及具体步骤介绍
随风飘摇的土木狗的博客
05-26 9568
1.XGBoost模型[default=gbtree]有两种模型可以选择gbtree和gblinear。gbtree使用基于树的模型进行提升计算，gblinear使用线性模型进行提升计算。其中gbtree的效果要远好于gblinear。2.objective目标函数 [ default=reg:linear ]定义学习任务及相应的学习目标，可选的目标函数如下：“reg:linear” –线性回归。“reg:logistic” –逻辑回归。
XGBoost模型调参：GridSearchCV方法网格搜索优化参数
qq_44949041的博客
12-31 1万+
GridSearchCV是XGBoost模型最常用的调参方法。本文主要介绍了如何使用GridSearchCV寻找XGBoost的最优参数，有完整的代码和数据文件。文中详细介绍了GridSearchCV的工作原理，param_grid等常用参数；常见的learning_rate和max_depth等可调参数及调参顺序；最后总结了GridSearchCV的缺点及对应的解决方法。
xgboost调参指南
11-07
xgboost调参过程，介绍xgboost调参的详细步骤，以及代码示例。
xgboost调参
08-15
文件是整理的xgboost的调参手册，主要是一般情况会调整的参数。参数的默认值和解释意义都在备注中解释。
xgboost导读与实战
01-27
适用初学者，快速入门xgboost。主要内容包含传统GBDT理论和调参训练
XGBoost模型调参、训练、保存、评估和预测
LMTX069的博客
04-20 3万+
xgboost模型调参、训练、保存、评估、预测
使用GridSearch对xgboost进行调参（全部流程）
jho9o5的博客
12-03 3万+
本篇博客的大纲： 导入相关的库/模块 初始化xgb.XGBRegressor模型的默认参数 按照顺序，每次选取一个或多个相关参数，给出取值区间，进行GridSearch 针对网格搜索给出的最佳参数，再划定更详细的区间，将最优参数精细化 找到所有属性的最佳参数，与原始模型进行结果对比，验证是否提高了模型的精度 ...
XGBoost 调参指南: 如何找到最佳参数组合
禅与计算机程序设计艺术
12-31 826
1.背景介绍 XGBoost(eXtreme Gradient Boosting)是一种强大的梯度提升决策树(GBDT)算法，它在许多机器学习任务中表现出色，如分类、回归和排序等。XGBoost 通过迭代地构建多个决策树，以最小化损失函数来预测目标变量。这种方法在许多应用中得到了广泛采用，如信用风险评估、医疗诊断、推荐系统等。 在实际应用中，选择合适的参数是非常重要的。不同的参数组合可能会导致...
XGboost数据比赛实战之调参篇(完整流程)
HuangQinJian
03-26 3万+
这一篇博客的内容是在上一篇博客Scikit中的特征选择，XGboost进行回归预测，模型优化的实战的基础上进行调参优化的，所以在阅读本篇博客之前，请先移步看一下上一篇文章。我前面所做的工作基本都是关于特征选择的，这里我想写的是关于XGBoost参数调整的一些小经验。之前我在网站上也看到很多相关的内容，基本是翻译自一篇英文的博客，更坑的是很多文章步骤讲的不完整，新人看了很容易一头雾水。由于本人也是一个
xgboost网格搜索算法调参+多标签分类预测项目实战
qq_46000115的博客
02-28 1782
xgboost模型，网格搜索算法
机器学习Xgboost网格调参类别预测 200w数据 完整代码可直接运行
06-03 132
机器学习Xgboost网格调参类别预测 200w数据 完整代码可直接运行
xgboost参数-调参
Vulpes corsac
09-27 2万+
目录 一、xgboost 原生接口 重要参数 训练参数 预测函数 绘制特征重要性 回归例子 二、xgboost 的 sklearn 风格接口 XGBClassifier基本使用 XGBRegressor基本使用 三、xgboost 调参思路 四、参考文章 xgboost 包含原生接口和 sklearn 风格接口两种，并且二者都实现了分类和回归的功能。如果想了解一些理论性的内容，可以看看之前的文章： XGBoost算法的相关知识 一、xgboost 原生接口 重要参数 ...
XGBoost参数调优完全指南（附Python代码）
热门推荐
工药叉的经文
07-19 7万+
XGBoost参数调优完全指南（附Python代码） 原文地址：Complete Guide to Parameter Tuning in XGBoost (with codes in Python) 译注：文内提供的代码和运行结果有一定差异，可以从这里下载完整代码对照参考。另外，我自己跟着教程做的时候，发现我的库无法解析字符串类型的特征，所以只用其中一部分特征做的，具体数值跟文章中不一样
XGBoost调参笔记
浅笑古今的博客
11-02 2万+
将之前XGBoost的笔记整理在CSDN上。 一、通用参数 一、这些参数用来控制XGBoost的宏观功能。 1、booster[默认gbtree] 选择每次迭代的模型，有两种选择： gbtree：基于树的模型 gbliner：线性模型 2、silent[默认0] 当这个参数值为1时，静默模式开启，不会输出任何信息。 一般这个参数就保持默认的0，因为这样能帮我们更好地理解模型。 3、nth...
xgboost网格调参
最新发布
06-05
XGBoost是一种常用的机器学习算法，它的网格搜索调参方法是为了找到最佳的超参数。网格搜索调参通过遍历所有超参数的组合，选择最优组合来最小化模型的误差。在XGBoost中，有一些关键的超参数需要调整，例如学习率、...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值