《PyTorch深度学习实践》-刘二大人-第三讲-梯度下降算法

B站传送门刘二大人PyTorch第三讲

一、梯度下降算法

代码解读：
1、计算全局损失cost，对全局损失求导求偏置值，再求预测值
 

x_data=[1.0,2.0,3.0]
y_data=[2.0,4.0,6.0]

w=1.0
def forward(x):
    return w*x

def cost(xs,ys):
    cost=0
    for x,y in zip(xs,ys):
        y_pred=forward(x)
        cost+=(y_pred-y)**2
    return cost/len(xs)


def gradient(xs,ys):
    grad=0
    for x,y in zip(xs,ys):
        grad+=2*x*(x*w-y)
    return grad/len(xs)

print('Predict (before training)',4,forward(4))

for epoch in range(100):
    cost_val=cost(x_data,y_data) # 计算每一次的损失
    grad_val=gradient(x_data,y_data) # 计算损失的倒数
    w-=0.01*grad_val # α为0.01，求最低点
    print('Predict(after training)',4,forward(4))

二、随机梯度下降算法

随机梯度下降算法与梯度下降算法主要的区别在于将全局损失cost改为单个损失loss，用单个损失来计算梯度偏置值，再进行梯度更新

x_data=[1.0,2.0,3.0]
y_data=[2.0,4.0,6.0]

w=1.0
def forward(x):
    return w*x

def loss(x,y):
    y_pred=forward(x)
    return (y_pred-y)**2


def gradient(x,y):
    return 2*x*(w*x-y)


print('Predict (before training)',4,forward(4))

for epoch in range(100):
    for x,y in zip(x_data,y_data):
        grad=gradient(x,y)
        w-=0.01*grad
        print('\tgrad:',x,y,grad)


print('Predict(after training)', 4, forward(4))