刘二大人 《Pyorch深度学习实践》第4讲 反向传播

指路☞ 《PyTorch深度学习实践》完结合集_哔哩哔哩_bilibili

知识补充：

1、从左到右是前向，从右到左是反向传播

 ********************************************************************************************

import torch

x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]

#tensor张量
w = torch.Tensor([1.0])
#计算梯度，默认不计算
w.requires_grad = True

def forward(x):
    return x*w
def loss(x, y):
    y_p = forward(x)
    return (y_p - y)**2
print("before training", 4, forward(4).item())

for epoch in range(100):
    for x, y in zip(x_data, y_data):
        l = loss(x, y)
        l.backward()
        """
        backward把这个链路上之前所有的梯度都求出来，
        把梯度存到w里，计算图就没有了，下一次会出现新的计算图
        """
        print('\tgrad:', x, y, w.grad.item())
        #item把值取出来，变成一个标量
        w.data = w.data - 0.01*w.grad.data
        """
        grad也是Tensor,如果直接用grad，是重新建立计算图
        """
        w.grad.data.zero_()
        #清零
    print('progress:', epoch, l.item())
print("after training", 4, forward(4).item())

部分运行结果：

progress: 89 9.094947017729282e-13
	grad: 1.0 2.0 -7.152557373046875e-07
	grad: 2.0 4.0 -2.86102294921875e-06
	grad: 3.0 6.0 -5.7220458984375e-06
progress: 90 9.094947017729282e-13
	grad: 1.0 2.0 -7.152557373046875e-07
	grad: 2.0 4.0 -2.86102294921875e-06
	grad: 3.0 6.0 -5.7220458984375e-06
progress: 91 9.094947017729282e-13
	grad: 1.0 2.0 -7.152557373046875e-07
	grad: 2.0 4.0 -2.86102294921875e-06
	grad: 3.0 6.0 -5.7220458984375e-06
progress: 92 9.094947017729282e-13
	grad: 1.0 2.0 -7.152557373046875e-07
	grad: 2.0 4.0 -2.86102294921875e-06
	grad: 3.0 6.0 -5.7220458984375e-06
progress: 93 9.094947017729282e-13
	grad: 1.0 2.0 -7.152557373046875e-07
	grad: 2.0 4.0 -2.86102294921875e-06
	grad: 3.0 6.0 -5.7220458984375e-06
progress: 94 9.094947017729282e-13
	grad: 1.0 2.0 -7.152557373046875e-07
	grad: 2.0 4.0 -2.86102294921875e-06
	grad: 3.0 6.0 -5.7220458984375e-06
progress: 95 9.094947017729282e-13
	grad: 1.0 2.0 -7.152557373046875e-07
	grad: 2.0 4.0 -2.86102294921875e-06
	grad: 3.0 6.0 -5.7220458984375e-06
progress: 96 9.094947017729282e-13
	grad: 1.0 2.0 -7.152557373046875e-07
	grad: 2.0 4.0 -2.86102294921875e-06
	grad: 3.0 6.0 -5.7220458984375e-06
progress: 97 9.094947017729282e-13
	grad: 1.0 2.0 -7.152557373046875e-07
	grad: 2.0 4.0 -2.86102294921875e-06
	grad: 3.0 6.0 -5.7220458984375e-06
progress: 98 9.094947017729282e-13
	grad: 1.0 2.0 -7.152557373046875e-07
	grad: 2.0 4.0 -2.86102294921875e-06
	grad: 3.0 6.0 -5.7220458984375e-06
progress: 99 9.094947017729282e-13
after training 4 7.999998569488525

Process finished with exit code 0

 ****************************************************************************************************

设函数为y=x^2+x+1

import torch

x_data = [1.0, 2.0, 3.0]
y_data = [4.0, 9.0, 16.0]

w1 = torch.tensor([1.0])
w1.requires_grad = True
w2 = torch.tensor([1.0])
w2.requires_grad = True
b = torch.tensor([1.0])
b.requires_grad = True

def forward(x):
    return x * x * w1 + x * w2 + b
def loss(x, y):
    y_p = forward(x)
    return (y_p - y)**2
print("before training", 4, forward(4).item())
for epoch in range(6000):
    for x, y in zip(x_data, y_data):
        l = loss(x, y)
        l.backward()
        print("\tgrad", x, y, '{:.6f}'.format(w1.grad.item()), '{:.6f}'.format(w2.grad.item()), '{:.6f}'.format(b.grad.item()))
        w1.data = w1.data - 0.01 * w1.grad.data
        w2.data = w2.data - 0.01 * w2.grad.data
        b.data = b.data - 0.01 * b.grad.data
        w1.grad.data.zero_()
        w2.grad.data.zero_()
        b.grad.data.zero_()
    print("progress:", epoch, '{:.6f}'.format(l.item()))
print("after training", 4, forward(4).item())

部分运行结果：