PyTorch Bug 记录：one of the variables needed for gradient computation has been modified by an inplace

有一段代码在 pytorch 1.2 上没有问题，但是移植到 pytorch 1.8 就会报如下错误：

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [3136, 10]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

经过检查，代码里并没有用到 inplace 操作。
后来发现这是 pytorch 版本更新造成的，对于 pytorch 1.4 之前的版本，如下代码是不会出错的：

opt1.zero_grad()
loss1.backward()
opt1.step()

opt2.zero_grad()
loss2.backward()
opt2.step()

但是更新到 pytorch 1.5 之后，这种操作就会报错，应该用下面代码代替：

opt1.zero_grad()
loss1.backward()

opt2.zero_grad()
loss2.backward()

opt1.step()
opt2.step()

这个BUG难了我一下午，特此记录一下。