RuntimeError: one of the variables needed for gradient computation has been modified by an inplace o

问题描述

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2, 256, 1, 1]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

问题分析

在原模型定义中增加了一个 SE 块，有以下代码：

self.headpose_feature = nn.Sequential(
	nn.ReLU(inplace=True),
	SELayer(256, 6)
)

问题解决

去掉 inplace=True，修改后代码如下：

self.headpose_feature = nn.Sequential(
	nn.ReLU(inplace=False),
	SELayer(256, 6)
)

参考链接：https://discuss.pytorch.org/t/runtimeerror-one-of-the-variables-needed-for-gradient-computation-has-been-modified-by-an-inplace-operation-torch-cuda-floattensor-1-1-256-256-which-is-output-0-of-relubackward0-is-at-version-1-expected-version-0-instead/143490
https://github.com/NVlabs/FUNIT/issues/23