前馈计算
权重维度增加,层数增加,模型变得复杂
但是化简后仍是线性,因此增加层数意义不大
引入激活函数,从而增加非线性
反向传播计算梯度,使用链式法则
反向传播过程
Tensor in PyTorch
Tenso(张量):PyTorch中存储数据的基本元素。
Tensor两个重要的成员,data和grad。(grad也是个张量)
课堂练习:线性模型 Linear Model
实现代码
import torch
# 已知数据:
x_data = [1.0,2.0,3.0]
y_data = [2.0,4.0,6.0]
# 线性模型为y = wx, 预测x = 4时, y的值
# 假设 w = 1
w = torch.Tensor([1.0])
w.requires_grad = True
# 定义模型:
def forward(x):
return x*w
# 定义损失函数:
def loss(x,y):
y_pred = forward(x)
return (y_pred - y)**2
print("Prediction before training:",4,'%.2f'%(forward(4)))
for epoch in range(100):
for x, y in zip(x_data,y_data):
l = loss(x,y)
l.backward() # 对requires_grad = True的Tensor(w)计算其梯度并进行反向传播,并且会释放计算图进行下一次计算
print("\tgrad:%.1f %.1f %.2f" % (x,y,w.grad.item()))
w.data = w.data - 0.01 * w.grad.data # 通过梯度对w进行更新
w.grad.data.zero_() #梯度清零
print("Epoch:%d, w = %.2f, loss = %.2f" % (epoch,w,l.item()))
print("Prediction after training:",4,'%.2f'%(forward(4)))
- 本算法中反向传播主要体现在,l.backward()。调用该方法后w.grad由None更新为Tensor类型,且w.grad.data的值用于后续w.data的更新。
- l.backward()会把计算图中所有需要梯度(grad)的地方都会求出来,然后把梯度都存在对应的待求的参数中,最终计算图被释放。
- 取tensor中的data是不会构建计算图的。
结果
课后练习
1. 计算y=xw的梯度
2. 计算仿射模型y=xw+b的梯度
3. 使用计算图计算y=w1x^2+w2x+b的梯度
4. 使用Pytorch计算y=w1x^2+w2x+b的梯度
二次模型 Quadratic Model
代码如下:
import torch
# 已知数据:
x_data = [1.0,2.0,3.0]
y_data = [6.0,11.0,18.0]
# 线性模型为y = w1x²+w2x+b时, 预测x = 4时, y的值
# 假设 w = 1, b = 1
w1 = torch.Tensor([1.0])
w1.requires_grad = True
w2 = torch.Tensor([1.0])
w2.requires_grad = True
b = torch.Tensor([1.0])
b.requires_grad = True
# 定义模型:
def forward(x):
return x*x*w1+x*w2+b
# 定义损失函数:
def loss(x,y):
y_pred = forward(x)
return (y_pred - y)**2
print("Prediction before training:",4,'%.2f'%(forward(4)))
for epoch in range(1000):
for x, y in zip(x_data,y_data):
l = loss(x,y)
l.backward() # 对requires_grad = True的Tensor(w)计算其梯度并进行反向传播,并且会释放计算图进行下一次计算
w1.data = w1.data - 0.02 * w1.grad.data # 通过梯度对w进行更新
w2.data = w2.data - 0.02 * w2.grad.data
b.data = b.data - 0.02 * b.grad.data
# 梯度清零
w1.grad.data.zero_()
w2.grad.data.zero_()
b.grad.data.zero_()
print("Epoch:%d, w1 = %.4f,w2 = %.4f,b = %.4f, loss = %.4f" % (epoch,w1,w2,b,l.item()))
print("Prediction after training:",4,'%.4f'%(forward(4)))
结果:
学习资料
- https://blog.csdn.net/weixin_43786637/article/details/126117060
- https://blog.csdn.net/Lilo_/article/details/113522485?utm_medium=distribute.pc_relevant.none-task-blog-2defaultbaidujs_title~default-9-113522485-blog-126117060.pc_relevant_aa&spm=1001.2101.3001.4242.6&utm_relevant_index=12
- https://blog.csdn.net/lizhuangabby/article/details/125548170?app_version=5.7.0&code=app_1562916241&csdn_share_tail=%7B%22type%22%3A%22blog%22%2C%22rType%22%3A%22article%22%2C%22rId%22%3A%22125548170%22%2C%22source%22%3A%22qq_43800119%22%7D&ctrtid=0pZiz&uLinkId=usr1mkqgl919blen&utm_source=app
系列文章索引
教程指路:【《PyTorch深度学习实践》完结合集】 https://www.bilibili.com/video/BV1Y7411d7Ys?share_source=copy_web&vd_source=3d4224b4fa4af57813fe954f52f8fbe7