解决问题：....but found at least two devices, cuda:0 and cpu!

出现的问题：

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument running_mean in method wrapper__cudnn_batch_norm)

困扰了好几天，设置model.to(device)没有问题，但是出现在运行后就出现有多个设备的问题。

参考了

Batch_norm causes RuntimeError: Expected all tensors to be on the same device , but found at least two devices, cuda:0 and cpu! - vision - PyTorch Forums

在这个文件夹中：........site-packages/torch/nn/functional.py添加了

添加

if not running_mean.is_cuda:
    running_mean = running_mean.to('cuda')

之后会得到running_mean在cuda上运行了

但是又会出现

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument running_var in method wrapper__cudnn_batch_norm)

继续添加

if not running_var.is_cuda:
    running_var = running_var.to('cuda')

记得取消掉添加的print running_mean以及running_var的内容！