最近读了Openpose论文,关于OpenPose的相关资料,博主整理后,留下了一个传送门,将网上写的比较好的博文和github地址记录在传送门里。
一、置信图(Confidence Map)生成原理
摘自OpenPose论文中的段落:
我们首先为每一个个体
k
k
k 产生个体的置信图
S
j
,
k
∗
S^{*}_{j,k}
Sj,k∗, 用
x
j
,
k
∈
R
2
x_{j,k} \in R^2
xj,k∈R2 表示个体
k
k
k 的关键点部位
j
j
j 的真是位置,则
S
j
,
k
∗
S^{*}_{j,k}
Sj,k∗ 中位置
p
∈
R
2
p\in R^2
p∈R2 的值定义为:
其中
σ
\sigma
σ 控制峰值的传播。 网络预测的置信图是通过一个最大操作将各个置信图聚合在一起:
是的,你没有看错,这就是一个 高斯分布!标准的高斯分布公式为
x
x
x,
y
y
y 代表像素的模板坐标,模板中心位置为原点)
x
x
x,
y
y
y 直接看成图像中任意点
(
x
,
y
)
(x, y)
(x,y)到目标点(关键点)的在
x
x
x或者
y
y
y方向上的距离。
二、两种生成置信图的方法
摘自一个COCO数据中,某一图像的关键点标注。
关于COCO数据集的理解,见我之前整理的 理解COCO数据集中的关键点标注
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
* * *** * * * *
* * * ** * *
**** * ** * *
* * * ** * *
* * ** * * ****
@File :Openpose/showGussMapForPoint.py
@Date :2020/12/7 下午2:07
@Require :opencv、matplotlib、numpy
@Author :hjxu2016, https://blog.csdn.net/hjxu2016/
@Funtion :二维图像中的 Guss热值图
"""
import cv2
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm as c
bgr_image = cv2.imread("./000000000785.jpg")
rgb_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2RGB)
points = [[367, 81, 2],
[378, 118, 2],
[358, 129, 2],
[341, 159, 2],
[309, 178, 2],
[399, 108, 2],
[433, 142, 2],
[449, 165, 2],
[393, 214, 2],
[367, 273, 2],
[396, 341, 2],
[424, 203, 2],
[429, 294, 2],
[466, 362, 2],
[360, 75, 2],
[374, 73, 2],
[356, 81, 2],
[386, 78, 2]]
2.1 循环遍历
循环已成惯性思维,第一个想到的就是循环遍历
import math
def put_heatmap(shape, center, sigma):
center_x, center_y, _ = center
heatmap = np.zeros((shape[0], shape[1]))
height, width, _ = shape
th = 4.6052
delta = math.sqrt(th * 2)
x0 = int(max(0, center_x - delta * sigma))
y0 = int(max(0, center_y - delta * sigma))
x1 = int(min(width, center_x + delta * sigma))
y1 = int(min(height, center_y + delta * sigma))
for y in range(y0, y1):
for x in range(x0, x1):
d = (x - center_x) ** 2 + (y - center_y) ** 2 # 高斯!
exp = d / 2.0 / sigma / sigma # 高斯!
if exp > th:
continue
heatmap[y][x] = max(heatmap[y][x], math.exp(-exp))
heatmap[y][x] = min(heatmap[y][x], 1.0)
return heatmap
for point in points:
guss_heatmap = put_heatmap(rgb_image.shape, point, sigma = 7)
fg, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(40, 8))
ax1.imshow(rgb_image)
ax2.imshow(guss_heatmap, cmap=c.jet)
guss_heatmap = np.array(guss_heatmap)
guss_heatmap[guss_heatmap < 0] = 0
guss_heatmap = cv2.normalize(guss_heatmap, guss_heatmap, 0, 255, cv2.NORM_MINMAX)
guss_heatmap = np.uint8(guss_heatmap)
jetmap = cv2.applyColorMap(255-guss_heatmap, cv2.COLORMAP_JET)
alpha = 0.5
out = cv2.addWeighted(rgb_image, alpha, jetmap, 1 - alpha, 0, jetmap)
ax3.imshow(out, cmap=c.jet)
2.2 用矩阵替代循环,加快速度
循环比较太慢了,矩阵则是一个更好的思路。
设置两个
x
x
x和
y
y
y矩阵,分别
0
、
1
、
.
.
.
−
w
0、1、...-w
0、1、...−w
那么图像所有点
(
x
,
y
)
(x, y)
(x,y)到目标点
p
p
p的在
x
x
x或者
y
y
y方向上的距离和就可以用矩阵运算。
def generate_gaussian_heatmap(shape, joint, sigma):
x, y,_ = joint
grid_x = np.tile(np.arange(shape[1]), (shape[0], 1))
grid_y = np.tile(np.arange(shape[0]), (shape[1], 1)).transpose()
grid_distance = (grid_x - x) ** 2 + (grid_y - y) ** 2
gaussian_heatmap = np.exp(-0.5 * grid_distance / sigma ** 2)
# 产生的就是一整张图的gaussian分布,只不过里中心点远的点非常非常小
return gaussian_heatmap
for point in points:
guss_heatmap = generate_gaussian_heatmap(rgb_image.shape, point, sigma = 7)
fg, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(40, 8))
ax1.imshow(rgb_image)
ax2.imshow(guss_heatmap, cmap=c.jet)
guss_heatmap[guss_heatmap < 0] = 0
guss_heatmap = cv2.normalize(guss_heatmap, guss_heatmap, 0, 255, cv2.NORM_MINMAX)
guss_heatmap = np.uint8(guss_heatmap)
jetmap = cv2.applyColorMap(255-guss_heatmap, cv2.COLORMAP_JET)
alpha = 0.5
out = cv2.addWeighted(rgb_image, alpha, jetmap, 1 - alpha, 0, jetmap)
ax3.imshow(out, cmap=c.jet)
显示一下结果