2020.5.22 PyTorch从零开始笔记（3） ——autograd_part2（有问题未解决）

1、torch.autograd.backward和grad

(1)为什么x.grad为none？ x.grad本应该为x.grad: tensor([[1., 3.]])才对
参考链接：https://www.cnblogs.com/marsggbo/p/11549631.html

x = torch.tensor([2., 1.], requires_grad=True).view(1, 2)
y = torch.tensor([[1., 2.], [3., 4.]], requires_grad=True)
z = torch.mm(x, y)
print(f"z:{z}")
z.backward(torch.Tensor([[1., 0]]), retain_graph=True)
print(f"x.grad: {x.grad}")
print(f"y.grad: {y.grad}")
###
z:tensor([[5., 8.]], grad_fn=<MmBackward>)
x.grad: None
y.grad: tensor([[2., 0.],
        [1., 0.]])

(2) 为什么x.grad和y.grad全为none？

x = torch.arange(1, 3, dtype=torch.float, requires_grad=True).view(1, 2)
y = torch.arange(3, 7, dtype=torch.float, requires_grad=True).view(2, 2)
print(x, x.requires_grad, x.grad_fn, '\n', y, y.requires_grad, y.grad_fn, y.is_leaf)
z = torch.mm(x, y)
print(f"{z, z.requires_grad, z.grad_fn}")
z.backward(torch.ones_like(z))
print(x.grad, y.grad)

####
tensor([[1., 2.]], grad_fn=<ViewBackward>) True <ViewBackward object at 0x000002105416B518> 
 tensor([[3., 4.],
        [5., 6.]], grad_fn=<ViewBackward>) True <ViewBackward object at 0x000002105416B550> False
(tensor([[ 3.,  8.],
        [ 5., 12.]], grad_fn=<MulBackward0>), True, <MulBackward0 object at 0x000002105416B518>)
None None

x张量需要加view（1，2）使shape为torch.Size([1, 2])，才能进行torch.mm,否则shape为torch.Size([2])

x = torch.arange(1, 3, dtype=torch.float, requires_grad=True) # 尺度不对
y = torch.arange(3, 7, dtype=torch.float, requires_grad=True).view(2, 2)
print(x, x.shape, x.requires_grad, x.grad_fn, '\n', y, y.requires_grad, y.grad_fn, y.is_leaf)
z = torch.mm(x, y)
print(f"{z, z.requires_grad, z.grad_fn}")
z.backward(torch.ones_like(z))
print(x.grad, y.grad)

tensor([1., 2.], requires_grad=True) torch.Size([2]) True None 
 tensor([[3., 4.],
        [5., 6.]], grad_fn=<ViewBackward>) True <ViewBackward object at 0x00000210540D2C88> False
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-50-9d36e1faaa71> in <module>
      2 y = torch.arange(3, 7, dtype=torch.float, requires_grad=True).view(2, 2)
      3 print(x, x.shape, x.requires_grad, x.grad_fn, '\n', y, y.requires_grad, y.grad_fn, y.is_leaf)
----> 4 z = torch.mm(x, y)
      5 print(f"{z, z.requires_grad, z.grad_fn}")
      6 z.backward(torch.ones_like(z))

RuntimeError: matrices expected, got 1D, 2D tensors at C:\w\1\s\tmp_conda_3.6_045031\conda\conda-bld\pytorch_1565412750030\work\aten\src\TH/generic/THTensorMath.cpp:747

2、size和shape在tensor中的作用

w1 = torch.tensor(2.0, requires_grad=True)

print(w1.data, w1.shape, w1.grad, w1.grad_fn)  # torch.Size([])
# tensor(2.) torch.Size([]) None None

print(w1.data, w1.size(), w1.grad, w1.grad_fn)
w1 = torch.tensor([2.0], requires_grad=True) # torch.Size([1])
# tensor([2.]) torch.Size([1]) None None

size()和shape()在Numpy中的作用：
参考链接：https://www.jianshu.com/p/3fab1e0beaed
size 用来计算数组和矩阵中所有元素的个数

a = np.array([[1, 2, 3], [4, 5, 6]])
np.size(a)    # 返回值为6
np.size(a,1)    # 返回值为3

shape 用来计算矩阵每维的大小

np.shape(a)    # 返回值为(2, 3)

3、获取中间变量的导数a1.retain_grad()