2 PyTorch 官网教材之 autograd 自动微分

AUTOGRAD：自动微分

官网链接：

https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html

1. 设置自动微分

# 创建有梯度的张量
x = torch.ones(2, 2, requires_grad=True)

# 张量计算
y = x + 2
z = y * y * 3
out = z.mean()

2. 改变微分状态

# .requires_grad_( ... ) 改变flag
print(x.requires_grad)  #  True
x.requires_grad_(False)  # 改变

print(x.requires_grad)  # False
print(x.grad_fn)  # grad_fn 的类型，加法、乘法、均值grad_fn=<MulBackward0>， grad_fn=<MeanBackward0>

3. backprop

• 简单的反向传播可以直接计算。复杂的就不能，需要指定输入的值。

# backprop
out.backward()
print(x.grad)  # 反向计算出的 grad 的值

• Tensor 没法对 Tensor 求导（RuntimeError: grad can be implicitly created only for scalar outputs） 。Now in this case y is no longer a scalar（标量）. torch.autograd could not compute the full Jacobian directly, but if we just want the vector-Jacobian product, simply pass the vector to backward as argument:
• CSDN链接：autograd与backward()及相关参数的理解

x = torch.randn(3, requires_grad=True)

y = x * 2
while y.data.norm() < 1000:
    y = y * 2

print(y)
v = torch.tensor([0.1, 1.0, 0.0001], dtype=torch.float)
y.backward(v)  #  必须添加v,没有会报错：RuntimeError: grad can be implicitly created only for scalar outputs。可以仅为标量输出隐式创建grad

print(x.grad)