RuntimeError: one of the variables needed for gradient computation has been modified by an inplace o

错误如下
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.IntTensor [12, 1, 10]] is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

只报loss.backward()错误

1-bug调试

调试bug时，可以采用.backward()方式

torch.sum(Tensor).backward()

2-可能的原因

1）Tensor中inplace参数使用，在原地修改了Tensor。

1、存在+=、-=、/=、*=符号的使用

input+=3
# 修改为
input= input +3

2、torch中的函数，有inplace参数的设置为False。例如ReLU。
上述两种情况易于发现，一般情况下影响不大。

2）Tensor有部分赋值操作

原始代码如下

sentence_masks = torch.rand((12, 1, 10))
# sentence_masks维度为[12, 1, 10]
sentence_masks[:, :, 0] = 0

解决方法

sentence_masks = torch.rand((12, 1, 10))
# sentence_masks维度为[12, 1, 10]
zeros_masks = torch.zeros((sentence_masks.size(0), sentence_masks.size(1)))
sentence_masks = torch.cat([zeros_masks.unsqueeze(-1), sentence_masks[:, :, 1:]], dim=-1)