2021-08-17

Pytorch学习中遇到的bug

使用torch.rand()随机化张量，并设置自动微分True，采用GPU加速时出现的bug
正确写法：
w = torch.rand(2,requires_grad=True,device=“cuda”)#随机化w，设置自动微分为true
或者:
w = torch.rand(2)#随机化w
w = w.cuda()
w.requires_grad = True
错误写法:
w = torch.rand(2,requires_grad=True)#随机化w，设置自动微分为true
w = w.cuda()
这种写法似乎会导致w被当作非叶节点，后面求微分值grad时会报错
在这里摘抄一下查问题时翻到的东西
###############################################
你好，

叶变量是位于图形开头的变量。这意味着 autograd 引擎跟踪的任何操作都没有创建它。
这就是您优化神经网络时想要的，因为它通常是您的权重或输入。

所以为了能够给优化器赋予权重，他们应该遵循上面叶子变量的定义。

a = torch.rand(10, requires_grad=True) # a is a leaf variable
a = torch.rand(10, requires_grad=True).double() # a is NOT a leaf variable as it was created by the operation that cast a float tensor into a double tensor
a = torch.rand(10).requires_grad_().double() # equivalent to the formulation just above: not a leaf variable
a = torch.rand(10).double() # a does not require gradients and has not operation creating it (tracked by the autograd engine).
a = torch.rand(10).doube().requires_grad_() # a requires gradients and has no operations creating it: it's a leaf variable and can be given to an optimizer.
a = torch.rand(10, requires_grad=True, device="cuda") # a requires grad, has not operation creating it: it's a leaf variable as well and can be given to an optimizer

所以在你的情况下，你想使用最后一行。
#####################
虽然上面的内容里说那种写法也是叶节点，如果不使用GPU加速，也没问题，可能就是pytorch的bug吧