提升杂谈

提升杂谈

概念

推导

学习率搜索

GBDT

XGBOOST

注：XGBoost着重于改变分类器的权重

数据集格式

为稀疏矩阵的表示方式

含义如下

代码

# XGBoost
    data_train = xgb.DMatrix(x_train, label=y_train)
    data_test = xgb.DMatrix(x_test, label=y_test)
    watch_list = [(data_test, 'eval'), (data_train, 'train')]
    param = {'max_depth': 6, 'eta': 0.8, 'silent': 1, 'objective': 'binary:logistic'}
             # 'subsample': 1, 'alpha': 0, 'lambda': 0, 'min_child_weight': 1}
    bst = xgb.train(param, data_train, num_boost_round=100, evals=watch_list)
    y_hat = bst.predict(data_test)
    # write_result(bst, 3)
    y_hat[y_hat > 0.5] = 1
    y_hat[~(y_hat > 0.5)] = 0
    xgb_acc = accuracy_score(y_test, y_hat)

ADABOOST

注：Adaboost改变样本权重的特性使之在训练集能得到较好结果，测试集则不一定

例子

误差上限和界

后话