梯度提升Gradient Boosting进行正弦函数回归&浅析损失函数和学习率影响

最新推荐文章于 2022-07-01 17:14:02 发布

Architect_0

最新推荐文章于 2022-07-01 17:14:02 发布

阅读量615

点赞数

分类专栏：机器学习文章标签：机器学习 python

本文链接：https://blog.csdn.net/weixin_42388833/article/details/118106415

版权

本文探讨了使用梯度提升进行正弦函数回归时，不同损失函数（最小二乘、最小绝对偏差、huber、quantile）的影响，并分析了学习率对模型性能的效应。指出小学习率需要更多迭代次数以达到稳定MSE，推荐采用早停法（Early Stopping）来选择合适的迭代次数，以减少过拟合并提高测试集准确性。

摘要由CSDN通过智能技术生成

1. 正弦函数回归

# coding: utf-8
# - https://scikit-learn.org/stable/modules/ensemble.html#gradient-boosting
# [1] J. Friedman, Greedy Function Approximation: A Gradient Boosting Machine, The Annals of Statistics, Vol. 29, No. 5, 2001.
# [2] T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning, Second Edition, Section 10.13.2, Springer, 2009.
import numpy as np
from matplotlib import pyplot as plt
from sklearn.ensemble import GradientBoostingRegressor
from tqdm import tqdm
import utils


def fit_gradient_boosting_regressor(X: np.ndarray, n_samples: int, y: np.ndarray,
                                    learning_rate: float, max_depth: int, random_state: int, loss: str):
    """Fits gradient tree boosting regressor and returns the learned model and training errors."""
    gb = GradientBoostingRegressor(n_estimators=n_samples, learning_rate=learning_rate, max_depth=max_depth, random_state=random_state,loss=loss)
    gb.fit(X,y)
    train_errors = gb.train_score_
    return gb, train_errors

def plot_sine(X, y):
    plt.figure()
    plt.plot(X, y)
    plt.xlabel("$x$")
    plt.ylabel("$sin(x)$")
    plt.title("Sinus Data")
    plt.show()


def plot_loss(errors):
    plt.figure()
    plt.plot(errors)
    plt.xlabel("Iteration")
    plt.ylabel("Least Squares Loss")
    plt.title("Gradient Boosting: Loss over Iterations")
    plt.show()


def visualize_gradient_boosting_sine(X, y, axs, staged_y_preds, i, step):
    # Plot sine curve fitting
    ax = axs[i//step]
    ax.set_xlim(0, X.max())
    ax.set_ylim(-1, 1)
    ax.set_xlabel("$x$")
    ax.set_ylabel("$sin(x) / Prediction$")

    # Obtain lines
    error_lines = []
    for _ in range(X.shape[<

最低0.47元/天解锁文章