pytorch 进行训练时报错:
RuntimeError:Trying to backward throughthe graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call.backward() or autograd.grad(). Specify retain graph=True if ou need to backward throug h the graph a secon time or if you need to access saved tensors after calling backward.
出现这个问题主要是梯度发生了变化,可以检查一下在loss.backwalk()执行前是否对loss进行了其它操作。
经过检查,我对loss进行了累加,如下:
loss += loss_func(outputs, labels).to(DEVICE) # loss_func是损失函数
optimizer.zero_grad() # gradient to zero
loss.backward() # backward
optimizer.step() # renew weight
因为我想计算训练误差,通过把各个批量的误差累加起来求平均,来表示本轮epoch的训练误差。
为了解决以上问题,不要对loss进行任何操作,只需添加一个新变量来保存即可,如下:
loss = loss_func(outputs, labels).to(DEVICE) # loss_func是损失函数
loss_totla += loss # loss_total用来保存批量误差
optimizer.zero_grad() # gradient to zero
loss.backward() # backward
optimizer.step() # renew weight
以上操作,便可解决错误!