跳坑一
RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling .backward() or autograd.grad() the first time.
查阅资料发现是在调用backward时发生的问题,也就是很重要的反向传播计算出错了!用个形象的例子
for i in range(1000):
# 清零
opt.zero_grad()
# 计算损失
cri=nn.MSELoss()
loss=cri(out1,target.T)
# 反向传播
loss.backward()
# 参数更新
opt.step()
if i% 100==0:
print("loss=",loss.data)
pytorch构建的计算图是动态图,所以每次迭代完之后计算图就被内存释放。很显然,代码中迭代了1000次,为了让它不被立即释放,需要加入标识retain_graph=True
改写
loss.backward(retain_graph=True)
跳坑二
所以代码已经完成了吗?NO!NO!NO!,又出错了!
错了很长一大段
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [128, 4]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
这段说的就是梯度能求不了了,出现了nan?仔细瞅瞅这个代码是不是缺了点啥?是的,缺了脑子。。。。。
整个的迭代过程,输入特征没有参与,所以能迭代个毛啊!可不就是在第一次迭代之后出现nan嘛!
正确代码奉上!
for i in range(1000):
# 重新执行一次网络
out1 = net(x)
# 清零
opt.zero_grad()
# 计算损失
cri=nn.MSELoss()
loss=cri(out1,target.T)
# 反向传播
loss.backward(retain_graph=True)
# 参数更新
opt.step()
if i% 100==0:
print("loss=",loss.data)
彩蛋!
不用自带的优化器,自己搞!
# 构建优化: 损失——梯度清零——反向传播
#
# for i in range(1000):
# cri=nn.MSELoss()
# loss=cri(out,target.T)
# # 清零
# net.zero_grad()
# # 反向传播
# loss.backward(retain_graph=True)
# # 更新权重
# l_rate=0.3
# for f in list(net.parameters()):
# f.data.sub_(f.grad.data*l_rate)
# if i%100==0:
# print("loss=",loss.data,i)
参考链接
pytorch基础