错误信息
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`, and by
making sure all `forward` function outputs participate in calculating loss.
出现原因
在nn.Module下__init__有定义的变量未在forward中使用,可自己逐步排查
我的解决办法
找到代码中那一部分的参数没有接收到回传梯度,然后改正即可。可以在梯度回传之后、参数更新之前加入
for name, param in model.named_parameters():
if param.grad is None:
print(name)
参考:https://blog.csdn.net/shaojie_45/article/details/123029735