【已解决】RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILEDYou can try to repro this exception us

问题描述

Traceback (most recent call last):
  File "Wtest.py", line 209, in <module>
    main()
  File "Wtest.py", line 167, in main
    output_clean_poisoned = test(poisoned_loader, clean_model, use_cuda)
  File "Wtest.py", line 202, in test
    outputs = model(inputs)
  File "/home/visionx/anaconda3/envs/dvbw/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/visionx/anaconda3/envs/dvbw/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 165, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/home/visionx/anaconda3/envs/dvbw/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/media/visionx/monica/project/DVBW/CIFAR/model.py", line 93, in forward
    out = self.layer2(out)
  File "/home/visionx/anaconda3/envs/dvbw/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/visionx/anaconda3/envs/dvbw/lib/python3.8/site-packages/torch/nn/modules/container.py", line 119, in forward
    input = module(input)
  File "/home/visionx/anaconda3/envs/dvbw/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/media/visionx/monica/project/DVBW/CIFAR/model.py", line 35, in forward
    out = self.bn2(self.conv2(out))
  File "/home/visionx/anaconda3/envs/dvbw/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/visionx/anaconda3/envs/dvbw/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 399, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/visionx/anaconda3/envs/dvbw/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 395, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED
You can try to repro this exception using the following code snippet. If that doesn't trigger the error, please include your original repro script when reporting this issue.

 

解决办法

        1、改小NUM-WORKS 

        由于在这里面我们采样了100张图片，也就是可以认为numworks=100，这是比较多的，那把这个值改小就可以了

        改之前命令

python3 Wtest.py --gpu-id 1 --model 'resnet' --trigger './triggers/Trigger_cross.png' --alpha './triggers/Alpha_cross.png'  --model-path './checkpoint/infected/resnet_badnets_cross_0_010/checkpoint.pth.tar' --target-label 0 --num-img 100

        改之后

python3 Wtest.py --gpu-id 1 --model 'resnet' --trigger './triggers/Trigger_cross.png' --alpha './triggers/Alpha_cross.png'  --model-path './checkpoint/infected/resnet_badnets_cross_0_010/checkpoint.pth.tar' --target-label 0 --num-img 10

        其实就是gpu显存不够，减小点工作量就可以了 

        修改之后，再次运行，成功！ 

         2、手动使用cudnn

import torch
torch.backends.cudnn.enabled = False

        网上有很多人也遇到这个问题，有人说是 CUDA 和 cudnn 的版本匹配问题，有人说需要重装 Pytorch，CUDA，cudnn。我看了官网，版本是匹配的，试着重装了也不管用，而且我按照另一个系统的版本装也不行。所以在报错代码里引入上面代码就可以了

        3、GPU，CUDA，cudnn 的关系

• CUDA 是 NVIDIA 推出的用于自家 GPU 的并行计算框架，只能在 NVIDIA 的GPU 上运行，而且只有当要解决的计算问题是可以大量并行计算的时候才能发挥 CUDA 的作用。

• cuDNN 是 NVIDIA 打造的针对深度神经网络的加速库，是一个用于深层神经网络的 GPU 加速库。如果你要用 GPU 训练模型，cuDNN 不是必须的，但是一般会采用这个加速库。

完结撒花

        有能力去爱别人的前提是你要有能力爱自己