01 Pytorch代码实践之【autograd模块】

模块介绍参考：https://pytorch.org/docs/stable/

autograd模块

导入库

import torch
from torch.autograd import Variable

requires_grad = True表示此Tensor变量需要自动微分，默认为False

# use Tensor new a Variable
x = Variable(torch.ones(2),requires_grad = True)
x

tensor([1., 1.], requires_grad=True)

前向传播，x.grad存储Tensor的微分值，x.grad_fn存储Tensor的微分函数

print(x.grad) 
print(x.grad_fn)
z = 4 * x
z

None
None

tensor([4., 4.], grad_fn=<MulBackward0>)

y = z.norm()
y

tensor(5.6569, grad_fn=<NormBackward0>)

在使用backward进行反向传播之后，x.grad和x.grad_fn不再为None

z = 4 * x
y = z.norm()
y.backward(retain_graph=True)
print(x.grad)
print(y.grad_fn)

tensor([8.4853, 8.4853])
<NormBackward0 object at 0x000002A25DF29108>

在jupyter notebook中可能会遇到如下错误：

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

即在前向传播结束之后，内存已经释放，第二次调用失败，解决方法就是将前向传播和后向传播放在一个block中运行，保证一次调用结束：

z = torch.ones(2,1)
x = torch.Tensor([[2,3],[1,2]])
x.requires_grad = True
y = x.mm(z)
y.backward(torch.ones(2,1))
x.grad

tensor([[1., 1.],
        [1., 1.]])