OpenPose(一)：根据关键点生成置信图(Confidence Map)

最近读了Openpose论文，关于OpenPose的相关资料，博主整理后，留下了一个传送门，将网上写的比较好的博文和github地址记录在传送门里。

---

---

一、置信图(Confidence Map)生成原理

摘自OpenPose论文中的段落：
我们首先为每一个个体 $k$ 产生个体的置信图 $S_{j,k}^{\ast }$, 用 $x_{j,k}\in R^2$ 表示个体 $k$ 的关键点部位 $j$ 的真是位置，则 $S_{j,k}^{\ast }$ 中位置 $p\in R^2$ 的值定义为：

其中 $\sigma$ 控制峰值的传播。 网络预测的置信图是通过一个最大操作将各个置信图聚合在一起：

是的，你没有看错，这就是一个 高斯分布！标准的高斯分布公式为

$x$, $y$ 代表像素的模板坐标，模板中心位置为原点）
$x$, $y$ 直接看成图像中任意点 $(x,y)$到目标点（关键点）的在$x$或者$y$方向上的距离。

二、两种生成置信图的方法

摘自一个COCO数据中，某一图像的关键点标注。
关于COCO数据集的理解，见我之前整理的 理解COCO数据集中的关键点标注

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""                  
*  * *** *  * *  *      
*  *  *   **  *  *             
****  *   **  *  *                 
*  *  *   **  *  *         
*  * **  *  * ****  

@File     :Openpose/showGussMapForPoint.py  
@Date     :2020/12/7 下午2:07  
@Require  :opencv、matplotlib、numpy   
@Author   :hjxu2016， https://blog.csdn.net/hjxu2016/
@Funtion  :二维图像中的 Guss热值图
"""

import cv2
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm as c

bgr_image = cv2.imread("./000000000785.jpg")
rgb_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2RGB)

points = [[367,  81,  2],
          [378, 118,   2],
          [358, 129,   2],
          [341, 159,   2],
          [309, 178,   2],
          [399, 108,   2],
          [433, 142,   2],
          [449, 165,   2],
          [393, 214,   2],
          [367, 273,   2],
          [396, 341,   2],
          [424, 203,   2],
          [429, 294,  2],
          [466, 362,   2],
          [360,  75,   2],
          [374,  73,   2],
          [356,  81,   2],
          [386,  78,   2]]

2.1 循环遍历

循环已成惯性思维，第一个想到的就是循环遍历

import math
def put_heatmap(shape, center, sigma):
    center_x, center_y, _ = center
    heatmap = np.zeros((shape[0], shape[1]))
    height, width, _ = shape
    th = 4.6052
    delta = math.sqrt(th * 2)

    x0 = int(max(0, center_x - delta * sigma))
    y0 = int(max(0, center_y - delta * sigma))

    x1 = int(min(width, center_x + delta * sigma))
    y1 = int(min(height, center_y + delta * sigma))

    for y in range(y0, y1):
        for x in range(x0, x1):
            d = (x - center_x) ** 2 + (y - center_y) ** 2  # 高斯！
            exp = d / 2.0 / sigma / sigma  # 高斯！
            if exp > th:
                continue
            heatmap[y][x] = max(heatmap[y][x], math.exp(-exp))
            heatmap[y][x] = min(heatmap[y][x], 1.0)
    return heatmap
    
for point in points:
    guss_heatmap = put_heatmap(rgb_image.shape, point, sigma = 7)
    fg, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(40, 8))

    ax1.imshow(rgb_image)
    ax2.imshow(guss_heatmap, cmap=c.jet)
    guss_heatmap = np.array(guss_heatmap)
    
    guss_heatmap[guss_heatmap < 0] = 0
    guss_heatmap = cv2.normalize(guss_heatmap, guss_heatmap, 0, 255, cv2.NORM_MINMAX)
    guss_heatmap = np.uint8(guss_heatmap)
    jetmap = cv2.applyColorMap(255-guss_heatmap, cv2.COLORMAP_JET)
    alpha = 0.5
    out = cv2.addWeighted(rgb_image, alpha, jetmap, 1 - alpha, 0, jetmap)
    ax3.imshow(out, cmap=c.jet)

2.2 用矩阵替代循环，加快速度

循环比较太慢了，矩阵则是一个更好的思路。
设置两个$x$和$y$矩阵，分别$0、1、...-w$
那么图像所有点$(x,y)$到目标点 $p$的在$x$或者$y$方向上的距离和就可以用矩阵运算。

def generate_gaussian_heatmap(shape, joint, sigma):
    x, y,_ = joint 
    grid_x = np.tile(np.arange(shape[1]), (shape[0], 1))
    grid_y = np.tile(np.arange(shape[0]), (shape[1], 1)).transpose()
    grid_distance = (grid_x - x) ** 2 + (grid_y - y) ** 2
    gaussian_heatmap = np.exp(-0.5 * grid_distance / sigma ** 2)
    # 产生的就是一整张图的gaussian分布，只不过里中心点远的点非常非常小
    return gaussian_heatmap
for point in points:
    guss_heatmap = generate_gaussian_heatmap(rgb_image.shape, point, sigma = 7)
    fg, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(40, 8))

    ax1.imshow(rgb_image)
    ax2.imshow(guss_heatmap, cmap=c.jet)

    guss_heatmap[guss_heatmap < 0] = 0
    guss_heatmap = cv2.normalize(guss_heatmap, guss_heatmap, 0, 255, cv2.NORM_MINMAX)
    guss_heatmap = np.uint8(guss_heatmap)
    jetmap = cv2.applyColorMap(255-guss_heatmap, cv2.COLORMAP_JET)
    alpha = 0.5
    out = cv2.addWeighted(rgb_image, alpha, jetmap, 1 - alpha, 0, jetmap)
    ax3.imshow(out, cmap=c.jet)

显示一下结果