keras学习之：20分钟，教你通过 feature map 生成 attention 图（heatmap 图）

最新推荐文章于 2024-02-23 15:08:23 发布

暖仔会飞

最新推荐文章于 2024-02-23 15:08:23 发布

阅读量2.9k

点赞数 10

分类专栏： Keras学习文章标签： heatmap keras opencv 计算机视觉 attention map

本文链接：https://blog.csdn.net/qq_42902997/article/details/121599719

版权

Keras学习专栏收录该内容

21 篇文章 5 订阅

订阅专栏

导入库

import numpy as np
import cv2

import matplotlib.pyplot as plt
%matplotlib inline
import keras.applications

通过 opencv 打开图片

因为我们的路径中存在中文路径，所以我们先用 numpy 读取图片的矩阵，然后再用 opencv 进行转码
如果你的路径中不存在中文，则可以直接采用 cv2.imread(“filepath”) 即可

def cv_imread(filePath):
    cv_img=cv2.imdecode(np.fromfile(filePath,dtype=np.uint8),-1)
    ## imdecode读取的是rgb，如果后续需要opencv处理的话，需要转换成bgr，转换后图片颜色会变化
    ##cv_img=cv2.cvtColor(cv_img,cv2.COLOR_RGB2BGR)
    return cv_img

img = cv_imread("../数据/cat.jpeg") # 图片原本很大
img.shape

(2500, 2392, 3)

对图片进行缩放处理

切记要用 cv2.resize(img,newshape) 的形式缩放
不要用 img.resize(newshape) ，这样会出错

img = cv2.resize(img, (224, 224))

ax = plt.imshow(img)

在这里插入图片描述

img.shape

(224, 224, 3)

对图片进行通道调整

opencv 读图片是 BGR, 先转成 RGB 的显示方式
有两种方法进行转换，除了下面这种，还可以通过 $i m g = i m g [:, :, : : - 1]$

img_rgb = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)

plt.imshow(img_rgb)

在这里插入图片描述

通过训练好的网络生成激活图

加载预训练的网络模型

采用 resnet50 的网络模型
采用 imagenet 上预训练的参数

resnet_50 = keras.applications.ResNet50(input_shape=(224, 224, 3),
                                               include_top=False,
                                               weights='imagenet')

Downloading data from https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5
94658560/94653016 [==============================] - 147s 2us/step

resnet_50.summary()  # 查看网络的层数和结构

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            (None, 224, 224, 3)  0                                            
__________________________________________________________________________________________________
conv1_pad (ZeroPadding2D)       (None, 230, 230, 3)  0           input_1[0][0]                    
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, 112, 112, 64) 9472        conv1_pad[0][0]                  
__________________________________________________________________________________________________
bn_conv1 (BatchNormalization)   (None, 112, 112, 64) 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 112, 112, 64) 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, 55, 55, 64)   0           activation_1[0][0]               
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, 55, 55, 64)   4160        max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
bn2a_branch2a (BatchNormalizati (None, 55, 55, 64)   256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 55, 55, 64)   0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, 55, 55, 64)   36928       activation_2[0][0]               
__________________________________________________________________________________________________
bn2a_branch2b (BatchNormalizati (None, 55, 55, 64)   256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 55, 55, 64)   0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, 55, 55, 256)  16640       activation_3[0][0]               
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, 55, 55, 256)  16640       max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
bn2a_branch2c (BatchNormalizati (None, 55, 55, 256)  1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNormalizatio (None, 55, 55, 256)  1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add_1 (Add)                     (None, 55, 55, 256)  0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 55, 55, 256)  0           add_1[0][0]                      
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, 55, 55, 64)   16448       activation_4[0][0]               
__________________________________________________________________________________________________
bn2b_branch2a (BatchNormalizati (None, 55, 55, 64)   256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 55, 55, 64)   0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, 55, 55, 64)   36928       activation_5[0][0]               
__________________________________________________________________________________________________
bn2b_branch2b (BatchNormalizati (None, 55, 55, 64)   256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_6 (Activation)       (None, 55, 55, 64)   0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, 55, 55, 256)  16640       activation_6[0][0]               
__________________________________________________________________________________________________
bn2b_branch2c (BatchNormalizati (None, 55, 55, 256)  1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, 55, 55, 256)  0           bn2b_branch2c[0][0]              
                                                                 activation_4[0][0]               
__________________________________________________________________________________________________
activation_7 (Activation)       (None, 55, 55, 256)  0           add_2[0][0]                      
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, 55, 55, 64)   16448       activation_7[0][0]               
__________________________________________________________________________________________________
bn2c_branch2a (BatchNormalizati (None, 55, 55, 64)   256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_8 (Activation)       (None, 55, 55, 64)   0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, 55, 55, 64)   36928       activation_8[0][0]               
__________________________________________________________________________________________________
bn2c_branch2b (BatchNormalizati (None, 55, 55, 64)   256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_9 (Activation)       (None, 55, 55, 64)   0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, 55, 55, 256)  16640       activation_9[0][0]               
__________________________________________________________________________________________________
bn2c_branch2c (BatchNormalizati (None, 55, 55, 256)  1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_3 (Add)                     (None, 55, 55, 256)  0           bn2c_branch2c[0][0]              
                                                                 activation_7[0][0]               
__________________________________________________________________________________________________
activation_10 (Activation)      (None, 55, 55, 256)  0           add_3[0][0]                      
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, 28, 28, 128)  32896       activation_10[0][0]              
__________________________________________________________________________________________________
bn3a_branch2a (BatchNormalizati (None, 28, 28, 128)  512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_11 (Activation)      (None, 28, 28, 128)  0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, 28, 28, 128)  147584      activation_11[0][0]              
__________________________________________________________________________________________________
bn3a_branch2b (BatchNormalizati (None, 28, 28, 128)  512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_12 (Activation)      (None, 28, 28, 128)  0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, 28, 28, 512)  66048       activation_12[0][0]              
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, 28, 28, 512)  131584      activation_10[0][0]              
__________________________________________________________________________________________________
bn3a_branch2c (BatchNormalizati (None, 28, 28, 512)  2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNormalizatio (None, 28, 28, 512)  2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_4 (Add)                     (None, 28, 28, 512)  0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
activation_13 (Activation)      (None, 28, 28, 512)  0           add_4[0][0]                      
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, 28, 28, 128)  65664       activation_13[0][0]              
__________________________________________________________________________________________________
bn3b_branch2a (BatchNormalizati (None, 28, 28, 128)  512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_14 (Activation)      (None, 28, 28, 128)  0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, 28, 28, 128)  147584      activation_14[0][0]              
__________________________________________________________________________________________________
bn3b_branch2b (BatchNormalizati (None, 28, 28, 128)  512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_15 (Activation)      (None, 28, 28, 128)  0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, 28, 28, 512)  66048       activation_15[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNormalizati (None, 28, 28, 512)  2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, 28, 28, 512)  0           bn3b_branch2c[0][0]              
                                                                 activation_13[0][0]              
__________________________________________________________________________________________________
activation_16 (Activation)      (None, 28, 28, 512)  0           add_5[0][0]                      
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, 28, 28, 128)  65664       activation_16[0][0]              
__________________________________________________________________________________________________
bn3c_branch2a (BatchNormalizati (None, 28, 28, 128)  512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_17 (Activation)      (None, 28, 28, 128)  0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, 28, 28, 128)  147584      activation_17[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNormalizati (None, 28, 28, 128)  512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_18 (Activation)      (None, 28, 28, 128)  0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, 28, 28, 512)  66048       activation_18[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNormalizati (None, 28, 28, 512)  2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_6 (Add)                     (None, 28, 28, 512)  0           bn3c_branch2c[0][0]              
                                                                 activation_16[0][0]              
__________________________________________________________________________________________________
activation_19 (Activation)      (None, 28, 28, 512)  0           add_6[0][0]                      
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, 28, 28, 128)  65664       activation_19[0][0]              
__________________________________________________________________________________________________
bn3d_branch2a (BatchNormalizati (None, 28, 28, 128)  512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_20 (Activation)      (None, 28, 28, 128)  0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, 28, 28, 128)  147584      activation_20[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNormalizati (None, 28, 28, 128)  512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_21 (Activation)      (None, 28, 28, 128)  0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, 28, 28, 512)  66048       activation_21[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNormalizati (None, 28, 28, 512)  2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_7 (Add)                     (None, 28, 28, 512)  0           bn3d_branch2c[0][0]              
                                                                 activation_19[0][0]              
__________________________________________________________________________________________________
activation_22 (Activation)      (None, 28, 28, 512)  0           add_7[0][0]                      
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, 14, 14, 256)  131328      activation_22[0][0]              
__________________________________________________________________________________________________
bn4a_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_23 (Activation)      (None, 14, 14, 256)  0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_23[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_24 (Activation)      (None, 14, 14, 256)  0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_24[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, 14, 14, 1024) 525312      activation_22[0][0]              
__________________________________________________________________________________________________
bn4a_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNormalizatio (None, 14, 14, 1024) 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_8 (Add)                     (None, 14, 14, 1024) 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
activation_25 (Activation)      (None, 14, 14, 1024) 0           add_8[0][0]                      
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, 14, 14, 256)  262400      activation_25[0][0]              
__________________________________________________________________________________________________
bn4b_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_26 (Activation)      (None, 14, 14, 256)  0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_26[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_27 (Activation)      (None, 14, 14, 256)  0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_27[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_9 (Add)                     (None, 14, 14, 1024) 0           bn4b_branch2c[0][0]              
                                                                 activation_25[0][0]              
__________________________________________________________________________________________________
activation_28 (Activation)      (None, 14, 14, 1024) 0           add_9[0][0]                      
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, 14, 14, 256)  262400      activation_28[0][0]              
__________________________________________________________________________________________________
bn4c_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_29 (Activation)      (None, 14, 14, 256)  0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_29[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_30 (Activation)      (None, 14, 14, 256)  0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_30[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_10 (Add)                    (None, 14, 14, 1024) 0           bn4c_branch2c[0][0]              
                                                                 activation_28[0][0]              
__________________________________________________________________________________________________
activation_31 (Activation)      (None, 14, 14, 1024) 0           add_10[0][0]                     
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, 14, 14, 256)  262400      activation_31[0][0]              
__________________________________________________________________________________________________
bn4d_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_32 (Activation)      (None, 14, 14, 256)  0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_32[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_33 (Activation)      (None, 14, 14, 256)  0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_33[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_11 (Add)                    (None, 14, 14, 1024) 0           bn4d_branch2c[0][0]              
                                                                 activation_31[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (None, 14, 14, 1024) 0           add_11[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, 14, 14, 256)  262400      activation_34[0][0]              
__________________________________________________________________________________________________
bn4e_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_35 (Activation)      (None, 14, 14, 256)  0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_35[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_36 (Activation)      (None, 14, 14, 256)  0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_36[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_12 (Add)                    (None, 14, 14, 1024) 0           bn4e_branch2c[0][0]              
                                                                 activation_34[0][0]              
__________________________________________________________________________________________________
activation_37 (Activation)      (None, 14, 14, 1024) 0           add_12[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, 14, 14, 256)  262400      activation_37[0][0]              
__________________________________________________________________________________________________
bn4f_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_38 (Activation)      (None, 14, 14, 256)  0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_38[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_39 (Activation)      (None, 14, 14, 256)  0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_39[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_13 (Add)                    (None, 14, 14, 1024) 0           bn4f_branch2c[0][0]              
                                                                 activation_37[0][0]              
__________________________________________________________________________________________________
activation_40 (Activation)      (None, 14, 14, 1024) 0           add_13[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, 7, 7, 512)    524800      activation_40[0][0]              
__________________________________________________________________________________________________
bn5a_branch2a (BatchNormalizati (None, 7, 7, 512)    2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_41 (Activation)      (None, 7, 7, 512)    0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, 7, 7, 512)    2359808     activation_41[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNormalizati (None, 7, 7, 512)    2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_42 (Activation)      (None, 7, 7, 512)    0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, 7, 7, 2048)   1050624     activation_42[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, 7, 7, 2048)   2099200     activation_40[0][0]              
__________________________________________________________________________________________________
bn5a_branch2c (BatchNormalizati (None, 7, 7, 2048)   8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNormalizatio (None, 7, 7, 2048)   8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_14 (Add)                    (None, 7, 7, 2048)   0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
activation_43 (Activation)      (None, 7, 7, 2048)   0           add_14[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, 7, 7, 512)    1049088     activation_43[0][0]              
__________________________________________________________________________________________________
bn5b_branch2a (BatchNormalizati (None, 7, 7, 512)    2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_44 (Activation)      (None, 7, 7, 512)    0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, 7, 7, 512)    2359808     activation_44[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNormalizati (None, 7, 7, 512)    2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_45 (Activation)      (None, 7, 7, 512)    0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, 7, 7, 2048)   1050624     activation_45[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNormalizati (None, 7, 7, 2048)   8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_15 (Add)                    (None, 7, 7, 2048)   0           bn5b_branch2c[0][0]              
                                                                 activation_43[0][0]              
__________________________________________________________________________________________________
activation_46 (Activation)      (None, 7, 7, 2048)   0           add_15[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, 7, 7, 512)    1049088     activation_46[0][0]              
__________________________________________________________________________________________________
bn5c_branch2a (BatchNormalizati (None, 7, 7, 512)    2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_47 (Activation)      (None, 7, 7, 512)    0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, 7, 7, 512)    2359808     activation_47[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNormalizati (None, 7, 7, 512)    2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_48 (Activation)      (None, 7, 7, 512)    0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, 7, 7, 2048)   1050624     activation_48[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNormalizati (None, 7, 7, 2048)   8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_16 (Add)                    (None, 7, 7, 2048)   0           bn5c_branch2c[0][0]              
                                                                 activation_46[0][0]              
__________________________________________________________________________________________________
activation_49 (Activation)      (None, 7, 7, 2048)   0           add_16[0][0]                     
==================================================================================================
Total params: 23,587,712
Trainable params: 23,534,592
Non-trainable params: 53,120
__________________________________________________________________________________________________

导入 keras 相关的包

from keras.models import *
from keras.layers import *

选取模型特定层

选取第 i 层的特征图作为可视化的材料

out_layer = resnet_50.layers[173]

predict_model = Model(resnet_50.inputs, out_layer.output )

result = predict_model.predict(img_rgb.reshape(-1,*img_rgb.shape))
#这里扩展图片的成 4 维，是因为 predict 操作默认是按照批次进行 predict，即 (batchsize, 224,224,3)
#而我们要 predict 一张图的话就要把原图 reshape 成一个 (1,224,224,3)，因此这里也可以写成 img.reshape(1,224,224,3)

result.shape

(1, 7, 7, 2048)

result_ = result.squeeze()
# result_ 就是特定的层输出的特征图，其维度是 (batchsize, length, width, channels)，这个长宽取决于你选取的层的输出的 size
# sueeze 可以把单独的这张图无用的 维度给删掉，比如 (1,7,7,2048) 第一个 1 就是没用的，因此 squeeze 之后就变成了下述维度

result_.shape

(7, 7, 2048)

将 2048 个 channel 提取的特征进行加和

得到的是这一个 layer 最终的特征图

activations =  result_.sum(axis=-1)

plt.imshow(activations)

在这里插入图片描述

将特征图恢复到和图片尺寸一样

因为每个 layer 的感受野不一样，恢复到原图大小，才能看出原图中哪些部分被激活了
resize 依然用的也是 cv2.resize()

activations = cv2.resize(activations,(224,224))

plt.imshow(activations)

在这里插入图片描述

对激活图进行处理

图中每个像素点都除以像素点中的最大值，这样所有点的值都维持在(0,1)之间
将所有的值都诚 * 255，这样每个点的值都会规范在（0，255）之间
用 255 - 处理过后的值，因为在 heatmap 生成的时候，原本是蓝色表示较大的值，红色表示较小的值，这样处理之后，可以让红色部分表示较大的值；这个过程中，很重要的一点就是：通过 astype(“uint”) 把值变成无符号的 int 类型，因为最后画图的时候我们需要的值是 (0-255) 之间的离散值，而不是浮点值

activations = activations / activations.max()

plt.imshow(activations)

在这里插入图片描述

activations *= 255
# 没有通过 255 取反的特征图
plt.imshow(activations)

在这里插入图片描述

## 通过 255 进行取反之后的特征图
activations_ = 255 - activations.astype("uint8")

plt.imshow(activations_)

在这里插入图片描述

# 没有通过 255 取反的 heatmap
heatmap_activations = cv2.applyColorMap(activations.astype("uint8"),cv2.COLORMAP_JET)

plt.imshow(heatmap_activations)

在这里插入图片描述

# 经过 255 取反后的 heatmap
heatmap_activations_ = cv2.applyColorMap(activations_,cv2.COLORMAP_JET)

plt.imshow(heatmap_activations_)

在这里插入图片描述

叠加原图和heatmap图

将特征的激活 heatmap 和原图按照 7：3 的比例进行叠加

img_heatmap_activations = cv2.addWeighted(heatmap_activations, 0.7, img_rgb, 0.3, 0)
plt.imshow(img_heatmap_activations_)

在这里插入图片描述

总结

opencv 调用函数的时候，要注意数据是否是 0-255 之间的离散值，否则可能出错
产生热图使用的函数是 cv2.applyColorMap()
按照比例叠加图片使用的是 cv2.addWeighted()

定义成函数

def cv_imread(filePath):
    img=cv2.imdecode(np.fromfile(filePath,dtype=np.uint8),-1)
    plt.imshow(img)
    return img

def process_img(img,new_shape=(224,224),isRGB=True):
    img = cv2.resize(img,new_shape)
    img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
    plt.imshow(img)
    return img

x = cv_imread("../数据/cat.jpeg")

在这里插入图片描述

img = process_img(img)

在这里插入图片描述

model = keras.applications.ResNet50(input_shape=(224, 224, 3),
                                               include_top=False,
                                               weights='imagenet')

def generate_feature_map(model,origin_img,layer_index):
    input_ = model.inputs
    output_ = model.layers[layer_index].output
    generate_model = Model(inputs=input_,outputs=output_)
    if len(origin_img.shape) < 4:
        origin_img_size = origin_img.shape
        print("img's shape is %s, it should be expanded if you wants to predict"%str(origin_img_size))
        origin_img = origin_img.reshape(-1,*origin_img.shape)
        
    feature_map = generate_model.predict(origin_img)
    feature_map = np.abs(feature_map)
    feature_map = np.sum(feature_map,axis=-1).squeeze()
    feature_map = cv2.resize(feature_map,(origin_img_size[0],origin_img_size[1]))
    feature_map /=  feature_map.max()
    feature_map *= 255
    feature_map = 255 - feature_map.astype("uint8")
    
    plt.imshow(feature_map)
    return feature_map

feature_map = generate_feature_map(model,img,90)

在这里插入图片描述

def concate_img_and_featuremap(img,feature_map,img_percent, feature_map_percent):
    heatmap = cv2.applyColorMap(feature_map,cv2.COLORMAP_JET)
    plt.imshow(heatmap)
    heatmap = cv2.addWeighted(heatmap,feature_map_percent,img,img_percent,0)
    plt.imshow(heatmap)
    return heatmap

x = concate_img_and_featuremap(img,feature_map,0.3,0.7)

在这里插入图片描述

heatmap.shape

(224, 224, 3)

img.shape

(224, 224, 3)

def plot_heatmaps_from_layers(lst):
    show_img_lst = None
    for i in lst:
        feature_map_ = generate_feature_map(model,img,i)
        heatmap_ = concate_img_and_featuremap(img,feature_map_,0.3,0.7) 
        if show_img_lst is None:
            show_img_lst = heatmap_
        else:
            show_img_lst = np.concatenate([show_img_lst,heatmap_],axis=1)
#    

    plt.figure(figsize=(15,15))
    plt.axis("off")
    ax = plt.imshow(show_img_lst)

lst =

lst1 = [i for i in range(1,5)]
lst2 = [i for i in range(31,35)]
lst3 = [i for i in range(61,65)]
lst4 = [i for i in range(91,95)]
lst5 = [i for i in range(121,125)]
lst6 = [i for i in range(151,155)]
lst7 = [i for i in range(165,172)]

plot_heatmaps_from_layers(lst1)

img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict

在这里插入图片描述

plot_heatmaps_from_layers(lst2)

img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict

在这里插入图片描述

plot_heatmaps_from_layers(lst3)

img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict

在这里插入图片描述

plot_heatmaps_from_layers(lst4)

img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict

在这里插入图片描述

plot_heatmaps_from_layers(lst5)

img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict

在这里插入图片描述

plot_heatmaps_from_layers(lst6)

img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict

在这里插入图片描述

plot_heatmaps_from_layers(lst7)

img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict
img's shape is (224, 224, 3), it should be expanded if you wants to predict

在这里插入图片描述

如果每个 featuremap 只取其中一层

即，例如我们选取第 172 个 layer，那么他的特征图维度是 $7 * 7 * 2048$ 那么这 2048 个 channel 如果单独作为一个特征图是什么样子的，我们可以通过下面的代码得到

def generate_feature_map_channels(model,origin_img,layer_index,channel_index):
    input_ = model.inputs
    output_ = model.layers[layer_index].output
    generate_model = Model(inputs=input_,outputs=output_)
    if len(origin_img.shape) < 4:
        origin_img_size = origin_img.shape
        print("img's shape is %s, it should be expanded if you wants to predict"%str(origin_img_size))
        origin_img = origin_img.reshape(-1,*origin_img.shape)
        
    feature_map = generate_model.predict(origin_img)
    feature_map = np.abs(feature_map)
    print(feature_map.shape)
    feature_map = feature_map[:,:,:,channel_index].squeeze()  ## 这里拿出 feature map 中指定 channel 的图
    print(feature_map.shape)
    feature_map = cv2.resize(feature_map,(origin_img_size[0],origin_img_size[1]))
    feature_map /=  feature_map.max()
    feature_map *= 255
    feature_map = 255 - feature_map.astype("uint8")
    
    plt.imshow(feature_map)
    return feature_map

def plot_heatmaps_from_channels(lst):
    show_img_lst = None
    for i in lst:
        feature_map_ = generate_feature_map_channels(model,img,172,i)
        heatmap_ = concate_img_and_featuremap(img,feature_map_,0.3,0.7) 
        if show_img_lst is None:
            show_img_lst = heatmap_
        else:
            show_img_lst = np.concatenate([show_img_lst,heatmap_],axis=1)
#    

    plt.figure(figsize=(25,25))
    plt.axis("off")
    ax = plt.imshow(show_img_lst)

plot_heatmaps_from_channels(lst1)

img's shape is (224, 224, 3), it should be expanded if you wants to predict
(1, 7, 7, 2048)
(7, 7)
img's shape is (224, 224, 3), it should be expanded if you wants to predict
(1, 7, 7, 2048)
(7, 7)
img's shape is (224, 224, 3), it should be expanded if you wants to predict
(1, 7, 7, 2048)
(7, 7)
img's shape is (224, 224, 3), it should be expanded if you wants to predict
(1, 7, 7, 2048)
(7, 7)

在这里插入图片描述

plot_heatmaps_from_channels(lst7)

img's shape is (224, 224, 3), it should be expanded if you wants to predict
(1, 7, 7, 2048)
(7, 7)
img's shape is (224, 224, 3), it should be expanded if you wants to predict
(1, 7, 7, 2048)
(7, 7)
img's shape is (224, 224, 3), it should be expanded if you wants to predict
(1, 7, 7, 2048)
(7, 7)
img's shape is (224, 224, 3), it should be expanded if you wants to predict
(1, 7, 7, 2048)
(7, 7)
img's shape is (224, 224, 3), it should be expanded if you wants to predict
(1, 7, 7, 2048)
(7, 7)
img's shape is (224, 224, 3), it should be expanded if you wants to predict
(1, 7, 7, 2048)
(7, 7)
img's shape is (224, 224, 3), it should be expanded if you wants to predict
(1, 7, 7, 2048)
(7, 7)

在这里插入图片描述

暖仔会飞

关注

10
点赞
踩
43

收藏

觉得还不错? 一键收藏
打赏
0
评论
keras学习之：20分钟，教你通过 feature map 生成 attention 图（heatmap 图）

导入库import numpy as npimport cv2import matplotlib.pyplot as plt%matplotlib inlineimport keras.applications通过 opencv 打开图片因为我们的路径中存在中文路径，所以我们先用 numpy 读取图片的矩阵，然后再用 opencv 进行转码如果你的路径中不存在中文，则可以直接采用 cv2.imread(“filepath”) 即可def cv_imread(filePath):
复制链接

扫一扫