B站刘二大人老师的《PyTorch深度学习实践》Lecture_04 重点回顾+手记+代码复现+知识补充
Lecture_04 反向传播 Back Propagation
一、重点回顾
(一)计算图 Computational Graph
1. 简单网络的梯度计算
(1)线性模型的计算图实例
2. 复杂网络的梯度计算
(1) 两层神经网路
(2)存在问题
(3)问题解决
(二)链式法则 Chain Rule
(1)使用链式法则进行梯度计算
(2)使用链式法则进行梯度计算实例
二、手记
三、代码复现
(一)线性模型 Linear Model
import torch
# 已知数据:
x_data = [1.0,2.0,3.0]
y_data = [2.0,4.0,6.0]
# 线性模型为y = wx, 预测x = 4时, y的值
# 假设 w = 1
w = torch.Tensor([1.0])
w.requires_grad = True
# 定义模型:
def forward(x):
return x*w
# 定义损失函数:
def loss(x,y):
y_pred = forward(x)
return (y_pred - y)**2
print("Prediction before training:",4,'%.2f'%(forward(4)))
for epoch in range(100):
for x, y in zip(x_data,y_data):
l = loss(x,y)
l.backward() # 对requires_grad = True的Tensor(w)计算其梯度并进行反向传播,并且会释放计算图进行下一次计算
print("\tgrad:%.1f %.1f %.2f" % (x,y,w.grad.item()))
w.data = w.data - 0.01 * w.grad.data # 通过梯度对w进行更新
w.grad.data.zero_() #梯度清零
print("Epoch:%d, w = %.2f, loss = %.2f" % (epoch,w,l.item()))
print("Prediction after training:",4,'%.2f'%(forward(4)))
(二)二次模型 Quadratic Model
import torch
# 已知数据:
x_data = [1.0,2.0,3.0]
y_data = [6.0,11.0,18.0]
# 线性模型为y = w1x²+w2x+b时, 预测x = 4时, y的值
# 假设 w = 1, b = 1
w1 = torch.Tensor([1.0])
w1.requires_grad = True
w2 = torch.Tensor([1.0])
w2.requires_grad = True
b = torch.Tensor([1.0])
b.requires_grad = True
# 定义模型:
def forward(x):
return x*x*w1+x*w2+b
# 定义损失函数:
def loss(x,y):
y_pred = forward(x)
return (y_pred - y)**2
print("Prediction before training:",4,'%.2f'%(forward(4)))
for epoch in range(1000):
for x, y in zip(x_data,y_data):
l = loss(x,y)
l.backward() # 对requires_grad = True的Tensor(w)计算其梯度并进行反向传播,并且会释放计算图进行下一次计算
w1.data = w1.data - 0.02 * w1.grad.data # 通过梯度对w进行更新
w2.data = w2.data - 0.02 * w2.grad.data
b.data = b.data - 0.02 * b.grad.data
w1.grad.data.zero_()
w2.grad.data.zero_()
b.grad.data.zero_() # 梯度清零
print("Epoch:%d, w1 = %.4f,w2 = %.4f,b = %.4f, loss = %.4f" % (epoch,w1,w2,b,l.item()))
print("Prediction after training:",4,'%.4f'%(forward(4)))