解决pytorch DDP 报错This error indicates that your module has parameters that were not used

使用pytorch DDP训练的时候，有的时候会报错This error indicates that your module has parameters that were not used。

解决办法:

现在单GPU下做一个iteration，并在loss.backward()和optimizer.step()之间插入以下代码:

for name, param in model.named_parameters():
    if param.grad is None:
        print(name)

这时未使用的参数就会未参与梯度的计算，因此可以获取未使用参数，并注释掉消除错误。完整代码如下:

model = model().cuda()
x = torch.rand((...)).cuda()

criterion = torch.nn.MSELoss(reduction='sum')
optimizer = torch.optim.SGD(model.parameters(), lr=1e-4)

y_pred = model(x)

y = y_pred

loss = criterion(y_pred, y)

# Zero gradients, perform a backward pass, and update the weights.
optimizer.zero_grad()
loss.backward()

for name, param in model.named_parameters():
    if param.grad is None:
        print(name)

optimizer.step()