Gradcam可视化yolov5检测结果

参考源码

pytorch-grad-cam-book

步骤

1.将参考源码拷贝到pycharm中进行修改，如果加载的是yolov5官方文件，不做任何修改，可以直接使用，如果想加载自己的模型文件需要查看第二步
2.torch.hub.conf加载本地源码

2.1 torch.hub.conf加载本地的yolov5模型

问题1：
改模型结构的情况下 ，可以通过以下方法加载模型，只需改第1，3个参数
其中：第一个参数是项目根目录，主要是用来找hubconf.py文件
第二个参数custom，
第三个是自己的权重参数，
第四个表示加载的是本地模型

model=torch.hub.load('G:/zongwenjian/code/yolov5-20220907T133254Z-001/yolov5','custom','G:/zongwenjian/code/yolov5-20220907T133254Z-001/yolov5/runs/train/exp4/weights/best.pt',source='local')

现象：通过pycharm的debug查看model的参数，可以发现
模型的结构是对的（采用的是yolov5s,改模型结构）
所检测目标的类别数量、类别名字是对的（类别数量、类别名字在yolov5s训练自己数据集时进行更改）
先验框的尺寸是不对的（先验框尺寸在yolov5s.yaml中进行了修改，但是加载进来的model先验框尺寸仍然不对），这个不改不影响检测，没关系
进行debug可以发现 torch.hub.load使用了下面这个函数，意思是说 torch.load.hub会从hubconf.py中进行加载文件，因此我们需要改原来的yolov5根目录下的hubconf.py文件

	def _load_local(hubconf_dir, model, *args, **kwargs):
    r"""
    Load a model from a local directory with a ``hubconf.py``.

    Args:
        hubconf_dir (string): path to a local directory that contains a
            ``hubconf.py``.
        model (string): name of an entrypoint defined in the directory's
            ``hubconf.py``.
        *args (optional): the corresponding args for callable ``model``.
        **kwargs (optional): the corresponding kwargs for callable ``model``.

    Returns:
        a single model with corresponding pretrained weights.

    Example:
        >>> path = '/some/local/path/pytorch/vision'
        >>> model = _load_local(path, 'resnet50', pretrained=True)
    """
    sys.path.insert(0, hubconf_dir)

2.2 更改hubconf.py文件

1.在yolov5文件下有hubconf.py文件 打开后找到以下代码 ，需要根据自己的目标检测类别个数修改classes数目

修改前

def _create(name, pretrained=True, channels=3, classes=80, autoshape=True, verbose=True, device=None):

修改后

def _create(name, pretrained=True, channels=3, classes=8, autoshape=True, verbose=True, device=None):

修改后的这部分代码：

import torch


def _create(name, pretrained=True, channels=3, classes=8, autoshape=True, verbose=True, device=None):
    """Creates or loads a YOLOv5 model

    Arguments:
        name (str): model name 'yolov5s' or path 'path/to/best.pt'
        pretrained (bool): load pretrained weights into the model
        channels (int): number of input channels
        classes (int): number of model classes
        autoshape (bool): apply YOLOv5 .autoshape() wrapper to model
        verbose (bool): print all information to screen
        device (str, torch.device, None): device to use for model parameters

    Returns:
        YOLOv5 model
    """

2 还需要修改def_create中的try: 部分里的类别个数
修改的语句为：

if pretrained and channels == 3 and classes == 80:

改为

if pretrained and channels == 3 and classes == 8:

3更改后的这部分代码为：

    try:
        device = select_device(('0' if torch.cuda.is_available() else 'cpu') if device is None else device)

        if pretrained and channels == 3 and classes == 8:
            model = DetectMultiBackend(path, device=device)  # download/load FP32 model
            # model = models.experimental.attempt_load(path, map_location=device)  # download/load FP32 model
            print('111')
        else:

hubconf.py相关代码更改完毕

3 从参考源码复制代码到pycharm,新建.py文件 输入以下代码即可

import torch
import cv2
import numpy as np
import requests
import torchvision.transforms as transforms
from pytorch_grad_cam import EigenCAM
from pytorch_grad_cam.utils.image import show_cam_on_image, scale_cam_image
from PIL import Image

COLORS = np.random.uniform(0, 255, size=(80, 3))


def parse_detections(results):
    detections = results.pandas().xyxy[0]
    detections = detections.to_dict()
    boxes, colors, names = [], [], []

    for i in range(len(detections["xmin"])):
        confidence = detections["confidence"][i]
        if confidence < 0.2:
            continue
        xmin = int(detections["xmin"][i])
        ymin = int(detections["ymin"][i])
        xmax = int(detections["xmax"][i])
        ymax = int(detections["ymax"][i])
        name = detections["name"][i]
        category = int(detections["class"][i])
        color = COLORS[category]

        boxes.append((xmin, ymin, xmax, ymax))
        colors.append(color)
        names.append(name)
    return boxes, colors, names


def draw_detections(boxes, colors, names, img):
    for box, color, name in zip(boxes, colors, names):
        xmin, ymin, xmax, ymax = box
        cv2.rectangle(
            img,
            (xmin, ymin),
            (xmax, ymax),
            color,
            2)

        cv2.putText(img, name, (xmin, ymin - 5),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.8, color, 2,
                    lineType=cv2.LINE_AA)
    return img


# image_url = "https://upload.wikimedia.org/wikipedia/commons/f/f1/Puppies_%284984818141%29.jpg"
# 这里文件路径就是自己要检测的图片路径
img = np.array(Image.open(r'C:\Users\gou\Desktop\2.jpg'))
img = cv2.resize(img, (640, 640))
rgb_img = img.copy()
img = np.float32(img) / 255
transform = transforms.ToTensor()
tensor = transform(img).unsqueeze(0)
# 加载github中的模型
#model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True)

# 加载自定义模型
# 根据函数定义：第一个参数定义了hubconf.py位于哪个文件夹下面
# 第二个参数’custom‘ 表示使用的是自己的模型
# 第三个定义了 权重文件路径
# 第四个soure = 'local'表示自己使用的是本地模型
model=torch.hub.load('G:/zongwenjian/code/yolov5_ca_bifpn-20220907T013821Z-001/yolov5_ca_bifpn/yolov5','custom',r'G:\zongwenjian\code\yolov5_ca_bifpn-20220907T013821Z-001\yolov5_ca_bifpn\yolov5\runs\train\exp16\weights\best.pt',source='local')
#model=torch.hub.load('G:\zongwenjian\code\yolov5-20220907T133254Z-001\yolov5','custom',r'G:\zongwenjian\code\yolov5-20220907T133254Z-001\yolov5\runs\train\exp4\weights\best.pt',source='local')

# print 信息用来调试
print(model)

model.eval()
model.cpu()
target_layers = [model.model.model.model[-2]]

print(type(target_layers))
print(target_layers)
results = model([rgb_img])
boxes, colors, names = parse_detections(results)
detections = draw_detections(boxes, colors, names, rgb_img.copy())
im = Image.fromarray(detections)

# 保存图片 此处为yolov5s的识别结果
im.save('./3.jpg')

cam = EigenCAM(model, target_layers, use_cuda=False)
grayscale_cam = cam(tensor)[0, :, :]
cam_image = show_cam_on_image(img, grayscale_cam, use_rgb=True)

Image.fromarray(cam_image).save('./4.jpg')

# 目标框内检测效果查看
def renormalize_cam_in_bounding_boxes(boxes, colors, names, image_float_np, grayscale_cam):
    """Normalize the CAM to be in the range [0, 1]
    inside every bounding boxes, and zero outside of the bounding boxes. """
    renormalized_cam = np.zeros(grayscale_cam.shape, dtype=np.float32)
    for x1, y1, x2, y2 in boxes:
        renormalized_cam[y1:y2, x1:x2] = scale_cam_image(grayscale_cam[y1:y2, x1:x2].copy())
    renormalized_cam = scale_cam_image(renormalized_cam)
    eigencam_image_renormalized = show_cam_on_image(image_float_np, renormalized_cam, use_rgb=True)
    image_with_bounding_boxes = draw_detections(boxes, colors, names, eigencam_image_renormalized)
    return image_with_bounding_boxes


renormalized_cam_image = renormalize_cam_in_bounding_boxes(boxes, colors, names, img, grayscale_cam)
Image.fromarray(renormalized_cam_image).save('./34.jpg')