第十二章随机梯度下降(Stochastic_Gradient_Descent)

最新推荐文章于 2024-07-31 14:47:45 发布

IntelligentRS

最新推荐文章于 2024-07-31 14:47:45 发布

阅读量411

点赞数

分类专栏： Python入门数据科学机器学习算法（学习笔记）文章标签：线性回归算法回归

本文链接：https://blog.csdn.net/qq_36171491/article/details/124621848

版权

Python入门数据科学机器学习算法（学习笔记）专栏收录该内容

21 篇文章 13 订阅

订阅专栏

12.1 原始线性模型

接下来用用代码建立一个：

$y = 0.4 + 0.8*x$

def make_prediction(input_row,coefficients):
    # 预测值初始化为第一个系数值
    out_put_y_hat = coefficients[0]
    for i in range(len(input_row)-1):
        out_put_y_hat += coefficients[i+1]*input_row[i]
    return out_put_y_hat

# 主函数
if '__main__' == __name__:
    test_dataset = [[1,1],[2,3],[4,3],[3,2],[5,5]]
    test_coefficients = [0.4,0.8]
    
    for row in test_dataset:
        y_hat = make_prediction(row, test_coefficients)
        print("True_Y_value = %.3f, our_prediction = %.3f"%(row[-1],y_hat))

12.2 梯度下降优化

epochs:次数，我们这个模型不断通过学习training data,不断进行迭代更新coefficient

$b = b-learningrate*error*x$

（1）把所有的epochs进行Loop

（2）在每一个epoch里面训练集row进行循环

（3）每次循环都会进行系数（coefficient）调优

$b_0(t+1) = b_0(t)-learning\;rate*error(t)$

$b_1(t+1) = b_1(t)-learning\;rate*error(t)*x_1(t)$

$error = prediction-true\;value$

# -*- coding: utf-8 -*-
"""
Created on Mon May  9 09:02:31 2022

@author: xiaofeng
"""

def make_prediction(input_row, coefficients):
    out_put_y_hat = coefficients[0]
    for i in range(len(input_row) - 1):
        out_put_y_hat += coefficients[i + 1] * input_row[i]
    return out_put_y_hat


def using_sgd_method_to_calculate_coefficients(training_dataset, learning_rate, n_times_epoch):
    coefficients = [0.0 for i in range(len(training_dataset[0]))]
    for epoch in range(n_times_epoch):
        the_sum_of_error = 0
        for row in training_dataset:
            y_hat = make_prediction(row, coefficients)
            error = y_hat - row[-1]
            the_sum_of_error += error ** 2
            coefficients[0] = coefficients[0] - learning_rate * error
            for i in range(len(row) - 1):
                coefficients[i + 1] = coefficients[i + 1] - learning_rate * error * row[i]
        print("This is epoch [%d],the learning rate we are using is [%.3f],the error is [%.3f]" % (
        epoch, learning_rate, the_sum_of_error))
    return coefficients


# 主函数
if '__main__' == __name__:
    your_trainging_dataset = [[1, 1], [2, 3], [4, 3], [3, 2], [5, 5]]
    your_model_learning_rate = 0.001
    your_n_epoch = 500
    your_coefficients = using_sgd_method_to_calculate_coefficients(your_trainging_dataset,
                                                                   your_model_learning_rate,
                                                                   your_n_epoch)
    print(your_coefficients)