pytorch tensor 梯度

在使用backward()之前，默认为None，使用之后累加grad

1.创建可求导的tensor

import torch
x = torch.randn(3,4,requires_grad=True)
# x = torch.randn(3,4)
# x.requires_grad=True

2.可grad的tensor计算得来的tensor也可grad

import torch
x = torch.randn(3,4,requires_grad=True)
b = torch.randn(3,4,requires_grad=True)
y = x + b
z = y.sum()
x.requires_grad, b.requires_grad, y.requires_grad, z.requires_grad
z.backward()
x.grad, b.grad, y.grad, z.grad

3.多次求导，grad会累加
所以在实际中要每次迭代清空grad

import torch
x = torch.randn(3,4,requires_grad=True)
b = torch.randn(3,4,requires_grad=True)
y = x + b
z = y.sum()
x.requires_grad, b.requires_grad, y.requires_grad, z.requires_grad
z.backward(retain_graph=True)
x.grad, b.grad, y.grad, z.grad
-------------------------------
(tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]]), tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]]), None, None)
-------------------------------
z.backward(retain_graph=True)
x.grad, b.grad, y.grad, z.grad
--------------------------------
(tensor([[2., 2., 2., 2.],
        [2., 2., 2., 2.],
        [2., 2., 2., 2.]]), tensor([[2., 2., 2., 2.],
        [2., 2., 2., 2.],
        [2., 2., 2., 2.]]), None, None)
---------------------------------

参考：
https://pytorch.org/docs/stable/autograd.html#torch.Tensor.grad