第一周、autograd-自动求导系统

autograd注意

1.梯度不自动清零
2.依赖于叶子结点的结点，requires_ grad默认为True
3.叶子结点不可执行in-place（不能用+=）

torch.autograd.backward

功能:自动求取梯度
tensors:用于求导的张量，如loss
retain_ graph :保存计算图
create_ graph :创建导数计算图，用于高阶求导
grad_ tensors:多梯度权重
函数:torch. autograd. backward( tensors, 
                   grad_ tensors =None ,
                   retain_ .graph=None,
                   create_ .graph=False)

1.1 retain_graph

功能：保存计算图，执行多次反向传播

flag = True
# flag = False
if flag:
    w = torch.tensor([1.], requires_grad=True)
    x = torch.tensor([2.], requires_grad=True)

    a = torch.add(w, x)
    b = torch.add(w, 1)
    y = torch.mul(a, b)

    y.backward(retain_graph=True)
    print(w.grad)
    y.backward()
    print(w.grad)

  tensor([5.])
  tensor([10.])

1.2 grad_tensors

功能：多个梯度之间权重的设置

flag = True
# flag = False
flag = True
# flag = False
if flag:
    w = torch.tensor([1.], requires_grad=True)
    x = torch.tensor([2.], requires_grad=True)

    a = torch.add(w, x)     # retain_grad()
    b = torch.add(w, 1)

    y0 = torch.mul(a, b)    # y0 = (x+w) * (w+1)    dy0/dw = 5
    y1 = torch.add(a, b)    # y1 = (x+w) + (w+1)    dy1/dw = 2

    loss = torch.cat([y0, y1], dim=0)       # [y0, y1]
    grad_tensors = torch.tensor([1., 2.])

    loss.backward(gradient=grad_tensors)    # gradient 传入 torch.autograd.backward()中的grad_tensors

    print(w.grad)

  tensor([9.])

torch.autograd.grad

功能:求取梯度
outputs:用于求导的张量，如loss
inputs:需要梯度的张量
retain_ graph :保存计算图
create_ graph :创建导数计算图，用于高阶求导
grad_ outputs:多梯度权重
函数:torch. autograd. grad( outputs, 
                   inputs,
                   grad_ outputs=None ,
                   retain_.graph=None,
                   create_.graph=False)

flag = True
# flag = False
if flag:

    x = torch.tensor([3.], requires_grad=True)
    y = torch.pow(x, 2)     # y = x**2

    grad_1 = torch.autograd.grad(y, x, create_graph=True)   # grad_1 = dy/dx = 2x = 2 * 3 = 6
    print(grad_1)

    grad_2 = torch.autograd.grad(grad_1[0], x)              # grad_2 = d(dy/dx)/dx = d(2x)/dx = 2
    print(grad_2)

  (tensor([6.], grad_fn=<MulBackward0>),)
  (tensor([2.]),)

三级目录