Pytorch报错 “RuntimeError: Expected to have finished reduction in the prior iteration ... ” 的解决方案

错误信息

RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`, and by 
making sure all `forward` function outputs participate in calculating loss. 

出现原因

在nn.Module下__init__有定义的变量未在forward中使用，可自己逐步排查

我的解决办法

找到代码中那一部分的参数没有接收到回传梯度，然后改正即可。可以在梯度回传之后、参数更新之前加入

for name, param in model.named_parameters():
    if param.grad is None:
        print(name)

参考：https://blog.csdn.net/shaojie_45/article/details/123029735