0.pip install grad-cam
1.首先是跑了其他一些博主的帖子,都没跑通,最后在知乎上找到的这篇帖子,修改了图片的输入设置后跑通了,实例代码如下
import cv2
import numpy as np
import torch
from pytorch_grad_cam import GradCAM, \
ScoreCAM, \
GradCAMPlusPlus, \
AblationCAM, \
XGradCAM, \
EigenCAM, \
EigenGradCAM, \
LayerCAM, \
FullGrad
from pytorch_grad_cam import GuidedBackpropReLUModel
from pytorch_grad_cam.utils.image import show_cam_on_image, preprocess_image
# 加载预训练的 ViT 模型
model = torch.hub.load('facebookresearch/deit:main','deit_tiny_patch16_224', pretrained=True)
model.eval()
# 判断是否使用 GPU 加速
use_cuda = torch.cuda.is_available()
if use_cuda:
model = model.cuda()
def reshape_transform(tensor, height=14, width=14):
# 去掉cls token
result = tensor[:, 1:, :].reshape(tensor.size(0),
height, width, tensor.size(2))
# 将通道维度放到第一个位置
result = result.transpose(2, 3).transpose(1, 2)
return result
# 创建 GradCAM 对象
cam = GradCAM(model=model,
target_layers=[model.blocks[-1].norm1],
# 这里的target_layer要看模型情况,
# 比如还有可能是:target_layers = [model.blocks[-1].ffn.norm]
use_cuda=use_cuda,
reshape_transform=reshape_transform)
# 读取输入图像
image_path = "2.png"
rgb_img = cv2.imread(image_path, 1)[:, :, ::-1]
rgb_img = cv2.resize(rgb_img, (224, 224))
rgb_img = np.float32(rgb_img) / 255.0
# 预处理图像
input_tensor = preprocess_image(rgb_img,
mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
# 看情况将图像转换为批量形式
# input_tensor = input_tensor.unsqueeze(0)
if use_cuda:
input_tensor = input_tensor.cuda()
# 计算 grad-cam
target_category = None # 可以指定一个类别,或者使用 None 表示最高概率的类别
grayscale_cam = cam(input_tensor=input_tensor, targets=target_category)
grayscale_cam = grayscale_cam[0, :]
# 将 grad-cam 的输出叠加到原始图像上
visualization = show_cam_on_image(rgb_img, grayscale_cam)
# 保存可视化结果
cv2.cvtColor(visualization, cv2.COLOR_RGB2BGR, visualization)
cv2.imwrite('cam3.jpg', visualization)
参考链接:
https://zhuanlan.zhihu.com/p/640450435
2.接下来参考了这位大佬的博客,针对我的网络进行修改,一些遇到的错误在大佬在里面说了
https://blog.csdn.net/holly_Z_P_F/article/details/130011296
3.新的问题
AttributeError: ‘NoneType’ object has no attribute ‘shape’
Traceback (most recent call last):
File "D:\yanjiusheng\ZSE-SBIR\kk.py", line 68, in <module>
test()
File "D:\yanjiusheng\ZSE-SBIR\kk.py", line 56, in test
grayscale_cam = cam(input_tensor=input_tensor, targets=target_category)
File "D:\anaconda3\envs\zse-sbir\lib\site-packages\pytorch_grad_cam\base_cam.py", line 186, in __call__
return self.forward(input_tensor, targets, eigen_smooth)
File "D:\anaconda3\envs\zse-sbir\lib\site-packages\pytorch_grad_cam\base_cam.py", line 110, in forward
cam_per_layer = self.compute_cam_per_layer(input_tensor, targets, eigen_smooth)
File "D:\anaconda3\envs\zse-sbir\lib\site-packages\pytorch_grad_cam\base_cam.py", line 141, in compute_cam_per_layer
cam = self.get_cam_image(input_tensor, target_layer, targets, layer_activations, layer_grads, eigen_smooth)
File "D:\anaconda3\envs\zse-sbir\lib\site-packages\pytorch_grad_cam\base_cam.py", line 66, in get_cam_image
weights = self.get_cam_weights(input_tensor, target_layer, targets, activations, grads)
File "D:\anaconda3\envs\zse-sbir\lib\site-packages\pytorch_grad_cam\grad_cam.py", line 23, in get_cam_weights
if len(grads.shape) == 4:
AttributeError: 'NoneType' object has no attribute 'shape'
这个问题我最后解决也是一知半解做出来了,目标层的问题:GradCAM 需要一个可以计算梯度的层作为 target_layer,通常是卷积层或自注意力机制中的特定部分,我原来选择的是model.sa.model.transformer.layers[-1] 作为目标层,但是这可能并不是一个适合用于 GradCAM 的层。我选择了原来目标层下更具体的一层,即model.sa.model.transformer.layers[-1][0].layer_norm_input,兄弟们可以 print(model) 看下结构,多试几层看看哪层能用,我太菜了只能试。。。
参考的另外一些帖子:
https://ask.csdn.net/questions/8051598
https://github.com/open-mmlab/mmdetection/issues/1809