这里只写了Removing the bias of integral pose regression方法和自己实现的代码
H
(
p
)
\mathbf{H}\left(\mathbf{p}\right)
H(p)表示热力图
H
~
(
p
)
=
exp
(
β
⋅
H
(
p
)
)
∑
p
′
∈
Ω
exp
(
β
⋅
H
(
p
′
)
)
,
β
>
0
\tilde{\mathbf{H}}(\mathbf{p})=\frac{\exp (\beta \cdot \mathbf{H}(\mathbf{p}))}{\sum_{\mathbf{p}^{\prime} \in \Omega} \exp \left(\beta \cdot \mathbf{H}\left(\mathbf{p}^{\prime}\right)\right)}, \quad \beta>0
H~(p)=∑p′∈Ωexp(β⋅H(p′))exp(β⋅H(p)),β>0
其中
Ω
\Omega
Ω表示整张图的坐标
C
=
∑
p
′
∈
Ω
exp
(
β
⋅
H
(
p
′
)
)
C=\sum_{\mathbf{p}^{\prime} \in \Omega} \exp \left(\beta \cdot \mathbf{H}\left(\mathbf{p}^{\prime}\right)\right)
C=p′∈Ω∑exp(β⋅H(p′))
其中
Ω
1
,
Ω
2
,
Ω
3
,
Ω
4
\Omega_1, \Omega_2, \Omega_3, \Omega_4
Ω1,Ω2,Ω3,Ω4 分别表示左上,右上,左下,右下
(
x
0
,
y
0
)
\left(x_0, y_0\right)
(x0,y0)是没有偏差的坐标
J
r
=
(
x
r
y
r
)
=
∑
p
∈
Ω
H
~
(
p
)
⋅
p
\mathbf{J}^r = \begin{pmatrix} x_r\\ y_r \end{pmatrix} = \sum_{\mathbf{p} \in \Omega} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}
Jr=(xryr)=p∈Ω∑H~(p)⋅p
J
r
o
=
(
x
0
y
0
)
=
C
C
−
h
w
J
r
−
(
h
2
w
2
(
C
−
h
w
)
h
w
2
2
(
C
−
h
w
)
)
\mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\frac{C}{C-hw}\mathbf{J}^r -\begin{pmatrix} \frac{h^2 w}{2(C-h w)} \\ \frac{h w^2}{2(C-h w)} \end{pmatrix}
Jro=(x0y0)=C−hwCJr−(2(C−hw)h2w2(C−hw)hw2)
(不过论文里是
J
r
o
=
(
x
0
y
0
)
=
C
C
−
h
w
J
r
−
(
h
w
2
2
(
C
−
h
w
)
h
2
w
2
(
C
−
h
w
)
)
\mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\frac{C}{C-hw}\mathbf{J}^r -\begin{pmatrix} \frac{h w^2}{2(C-h w)} \\ \frac{h^2 w}{2(C-h w)} \end{pmatrix}
Jro=(x0y0)=C−hwCJr−(2(C−hw)hw22(C−hw)h2w),不知道为什么)
推导
左上
以
2
x
0
,
2
y
0
2x_0,2y_0
2x0,2y0将整张图划分为4个区域
由于其他地方的值比较接近于0,所以热力图可以近似表示为
H
~
(
p
)
≈
{
exp
(
β
⋅
H
(
p
)
)
C
,
p
∈
Ω
1
1
C
,
p
∉
Ω
1
\tilde{\mathbf{H}} \left(p\right) \approx \begin{cases} \frac{\exp (\beta \cdot \mathbf{H}(\mathbf{p}))}{C},& p\in \Omega_1\\ \frac{1}{C},& p\not\in \Omega_1 \end{cases}
H~(p)≈{Cexp(β⋅H(p)),C1,p∈Ω1p∈Ω1
因此
J
r
=
∑
p
∈
Ω
H
~
(
p
)
⋅
p
=
∑
p
∈
Ω
1
H
~
(
p
)
⋅
p
+
∑
p
∈
Ω
2
,
Ω
3
,
Ω
4
H
~
(
p
)
⋅
p
=
w
1
J
1
+
w
2
J
2
+
w
3
J
3
+
w
4
J
4
\begin{aligned} \mathbf{J}^r & =\sum_{\mathbf{p} \in \Omega} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p} \\ & =\sum_{\mathbf{p} \in \Omega_1} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}+\sum_{\mathbf{p} \in \Omega_2, \Omega_3, \Omega_4} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}\\ &= w_1 \mathbf{J}_1 + w_2 \mathbf{J}_2 + w_3 \mathbf{J}_3 + w_4 \mathbf{J}_4 \end{aligned}
Jr=p∈Ω∑H~(p)⋅p=p∈Ω1∑H~(p)⋅p+p∈Ω2,Ω3,Ω4∑H~(p)⋅p=w1J1+w2J2+w3J3+w4J4
其中每个区域的中心点坐标为
J
1
=
(
x
0
y
0
)
J
2
=
(
x
0
2
y
0
+
w
−
2
y
0
2
)
=
(
x
0
y
0
+
w
2
)
J
3
=
(
2
x
0
+
h
−
2
x
0
2
y
0
)
=
(
x
0
+
h
2
y
0
)
J
4
=
(
x
0
+
h
2
y
0
+
w
2
)
\mathbf{J}_1 = \begin{pmatrix} x_0\\ y_0\\ \end{pmatrix}\\ \mathbf{J}_2 = \begin{pmatrix} x_0\\ 2y_0 + \frac{w - 2y_0}{2}\\ \end{pmatrix} = \begin{pmatrix} x_0\\ y_0 + \frac{w}{2}\\ \end{pmatrix}\\ \mathbf{J}_3 = \begin{pmatrix} 2x_0 + \frac{h - 2 x_0}{2}\\ y_0\\ \end{pmatrix} = \begin{pmatrix} x_0 + \frac{h}{2} \\ y_0\\ \end{pmatrix}\\ \mathbf{J}_4 = \begin{pmatrix} x_0 + \frac{h}{2}\\ y_0 + \frac{w}{2}\\ \end{pmatrix}
J1=(x0y0)J2=(x02y0+2w−2y0)=(x0y0+2w)J3=(2x0+2h−2x0y0)=(x0+2hy0)J4=(x0+2hy0+2w)
权重
w
2
=
2
x
0
(
w
−
2
y
0
)
C
w
3
=
2
(
h
−
2
x
0
)
y
0
C
w
4
=
(
h
−
2
x
0
)
(
w
−
2
y
0
)
C
w
1
=
1
−
w
2
−
w
3
−
w
4
w_2 = \frac{2x_0\left(w - 2y_0\right)}{C}\\ w_3 = \frac{2\left(h - 2x_0\right)y_0}{C}\\ w_4 = \frac{\left(h - 2x_0\right)\left(w - 2y_0\right)}{C}\\ w_1 = 1 - w_2 - w_3 - w_4
w2=C2x0(w−2y0)w3=C2(h−2x0)y0w4=C(h−2x0)(w−2y0)w1=1−w2−w3−w4
(计算方法就是区域的像素个数乘
1
C
\frac{1}{C}
C1
因此
J
r
=
(
w
1
x
0
+
w
2
x
0
+
w
3
(
x
0
+
h
2
)
+
w
4
(
x
0
+
h
2
)
w
1
y
0
+
w
2
(
y
0
+
w
2
)
+
w
3
y
0
+
w
4
(
y
0
+
w
2
)
)
=
(
x
0
+
(
w
3
+
w
4
)
h
2
y
0
+
(
w
2
+
w
4
)
w
2
)
=
(
(
1
−
h
w
C
)
x
0
+
h
w
C
h
2
(
1
−
h
w
C
)
y
0
+
h
w
C
w
2
)
\begin{aligned} \mathbf{J}^r &= \begin{pmatrix} w_1 x_0 + w_2 x_0 + w_3 \left(x_0 + \frac{h}{2}\right) + w_4\left(x_0 + \frac{h}{2}\right)\\ w_1 y_0 + w_2 \left(y_0 + \frac{w}{2}\right) + w_3 y_0 + w_4\left(y_0 + \frac{w}{2}\right)\\ \end{pmatrix}\\ &= \begin{pmatrix} x_0 + \left(w_3 + w_4\right)\frac{h}{2}\\ y_0 + \left(w_2 + w_4\right)\frac{w}{2}\\ \end{pmatrix}\\ &=\begin{pmatrix} \left(1-\frac{h w}{C}\right) x_0+\frac{h w}{C} \frac{h}{2}\\ \left(1-\frac{h w}{C}\right) y_0+\frac{h w}{C} \frac{w}{2} \end{pmatrix} \end{aligned}
Jr=(w1x0+w2x0+w3(x0+2h)+w4(x0+2h)w1y0+w2(y0+2w)+w3y0+w4(y0+2w))=(x0+(w3+w4)2hy0+(w2+w4)2w)=((1−Chw)x0+Chw2h(1−Chw)y0+Chw2w)
可以解出来
J
r
o
=
(
x
0
y
0
)
=
(
C
C
−
h
w
x
r
−
h
2
w
2
(
C
−
h
w
)
C
C
−
h
w
y
r
−
h
w
2
2
(
C
−
h
w
)
)
\mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\begin{pmatrix} \frac{C}{C-h w} x_r-\frac{h^2 w}{2(C-h w)} \\ \frac{C}{C-h w} y_r-\frac{h w^2}{2(C-h w)} \end{pmatrix}
Jro=(x0y0)=(C−hwCxr−2(C−hw)h2wC−hwCyr−2(C−hw)hw2)
右上
以
2
x
0
,
2
(
w
−
y
0
)
2x_0,2(w-y_0)
2x0,2(w−y0)将整张图划分为4个区域
由于其他地方的值比较接近于0,所以热力图可以近似表示为
H
~
(
p
)
≈
{
exp
(
β
⋅
H
(
p
)
)
C
,
p
∈
Ω
2
1
C
,
p
∉
Ω
2
\tilde{\mathbf{H}} \left(p\right) \approx \begin{cases} \frac{\exp (\beta \cdot \mathbf{H}(\mathbf{p}))}{C},& p\in \Omega_2\\ \frac{1}{C},& p\not\in \Omega_2 \end{cases}
H~(p)≈{Cexp(β⋅H(p)),C1,p∈Ω2p∈Ω2
因此
J
r
=
∑
p
∈
Ω
H
~
(
p
)
⋅
p
=
∑
p
∈
Ω
2
H
~
(
p
)
⋅
p
+
∑
p
∈
Ω
1
,
Ω
3
,
Ω
4
H
~
(
p
)
⋅
p
=
w
2
J
2
+
w
1
J
1
+
w
3
J
3
+
w
4
J
4
\begin{aligned} \mathbf{J}^r & =\sum_{\mathbf{p} \in \Omega} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p} \\ & =\sum_{\mathbf{p} \in \Omega_2} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}+\sum_{\mathbf{p} \in \Omega_1, \Omega_3, \Omega_4} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}\\ &= w_2 \mathbf{J}_2 + w_1 \mathbf{J}_1 + w_3 \mathbf{J}_3 + w_4 \mathbf{J}_4 \end{aligned}
Jr=p∈Ω∑H~(p)⋅p=p∈Ω2∑H~(p)⋅p+p∈Ω1,Ω3,Ω4∑H~(p)⋅p=w2J2+w1J1+w3J3+w4J4
其中每个区域的中心点坐标为
J
1
=
(
x
0
w
−
2
(
w
−
y
0
)
2
)
=
(
x
0
y
0
−
w
2
)
J
2
=
(
x
0
y
0
)
J
3
=
(
2
x
0
+
h
−
2
x
0
2
w
−
2
(
w
−
y
0
)
2
)
=
(
x
0
+
h
2
y
0
−
w
2
)
J
4
=
(
x
0
+
h
2
y
0
)
\mathbf{J}_1 = \begin{pmatrix} x_0\\ \frac{w - 2\left(w - y_0\right)}{2}\\ \end{pmatrix} = \begin{pmatrix} x_0\\ y_0-\frac{w}{2}\\ \end{pmatrix}\\ \mathbf{J}_2 = \begin{pmatrix} x_0\\ y_0\\ \end{pmatrix}\\ \mathbf{J}_3 = \begin{pmatrix} 2x_0 + \frac{h - 2x_0}{2}\\ \frac{w - 2\left(w - y_0\right)}{2}\\ \end{pmatrix} = \begin{pmatrix} x_0 + \frac{h}{2}\\ y_0-\frac{w}{2}\\ \end{pmatrix}\\ \mathbf{J}_4 = \begin{pmatrix} x_0 + \frac{h}{2}\\ y_0\\ \end{pmatrix}
J1=(x02w−2(w−y0))=(x0y0−2w)J2=(x0y0)J3=(2x0+2h−2x02w−2(w−y0))=(x0+2hy0−2w)J4=(x0+2hy0)
权重
w
1
=
2
x
0
(
2
y
0
−
w
)
C
w
3
=
(
h
−
2
x
0
)
(
2
y
0
−
w
)
C
w
4
=
2
(
h
−
2
x
0
)
(
w
−
y
0
)
C
w
2
=
1
−
w
1
−
w
3
−
w
4
w_1 = \frac{2x_0\left(2y_0-w\right)}{C}\\ w_3 = \frac{\left(h - 2x_0\right)\left(2y_0 - w\right)}{C}\\ w_4 = \frac{2\left(h - 2x_0\right)\left(w - y_0\right)}{C}\\ w_2 = 1 - w_1 - w_3 - w_4
w1=C2x0(2y0−w)w3=C(h−2x0)(2y0−w)w4=C2(h−2x0)(w−y0)w2=1−w1−w3−w4
(计算方法就是区域的像素个数乘
1
C
\frac{1}{C}
C1
因此
J
r
=
(
w
1
x
0
+
w
2
x
0
+
w
3
(
x
0
+
h
2
)
+
w
4
(
x
0
+
h
2
)
w
1
(
y
0
−
w
2
)
+
w
2
y
0
+
w
3
(
y
0
−
w
2
)
+
w
4
y
0
)
=
(
x
0
+
(
w
3
+
w
4
)
h
2
y
0
−
(
w
1
+
w
3
)
w
2
)
=
(
(
1
−
h
w
C
)
x
0
+
h
w
C
h
2
(
1
−
h
w
C
)
y
0
+
h
w
C
w
2
)
\begin{aligned} \mathbf{J}^r &= \begin{pmatrix} w_1 x_0 + w_2 x_0 + w_3 \left(x_0 + \frac{h}{2}\right) + w_4\left(x_0 + \frac{h}{2}\right)\\ w_1 \left(y_0 - \frac{w}{2}\right) + w_2 y_0 + w_3 \left(y_0 - \frac{w}{2}\right) + w_4y_0\\ \end{pmatrix}\\ &= \begin{pmatrix} x_0 + \left(w_3 + w_4\right)\frac{h}{2}\\ y_0 - \left(w_1 + w_3\right)\frac{w}{2}\\ \end{pmatrix}\\ &=\begin{pmatrix} \left(1-\frac{h w}{C}\right) x_0+\frac{h w}{C} \frac{h}{2}\\ \left(1-\frac{h w}{C}\right) y_0+\frac{h w}{C} \frac{w}{2} \end{pmatrix} \end{aligned}
Jr=(w1x0+w2x0+w3(x0+2h)+w4(x0+2h)w1(y0−2w)+w2y0+w3(y0−2w)+w4y0)=(x0+(w3+w4)2hy0−(w1+w3)2w)=((1−Chw)x0+Chw2h(1−Chw)y0+Chw2w)
可以解出来
J
r
o
=
(
x
0
y
0
)
=
(
C
C
−
h
w
x
r
−
h
2
w
2
(
C
−
h
w
)
C
C
−
h
w
y
r
−
h
w
2
2
(
C
−
h
w
)
)
\mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\begin{pmatrix} \frac{C}{C-h w} x_r-\frac{h^2 w}{2(C-h w)} \\ \frac{C}{C-h w} y_r-\frac{h w^2}{2(C-h w)} \end{pmatrix}
Jro=(x0y0)=(C−hwCxr−2(C−hw)h2wC−hwCyr−2(C−hw)hw2)
左下
以
2
(
h
−
x
0
)
,
2
y
0
2(h - x_0),2y_0
2(h−x0),2y0将整张图划分为4个区域
由于其他地方的值比较接近于0,所以热力图可以近似表示为
H
~
(
p
)
≈
{
exp
(
β
⋅
H
(
p
)
)
C
,
p
∈
Ω
3
1
C
,
p
∉
Ω
3
\tilde{\mathbf{H}} \left(p\right) \approx \begin{cases} \frac{\exp (\beta \cdot \mathbf{H}(\mathbf{p}))}{C},& p\in \Omega_3\\ \frac{1}{C},& p\not\in \Omega_3 \end{cases}
H~(p)≈{Cexp(β⋅H(p)),C1,p∈Ω3p∈Ω3
因此
J
r
=
∑
p
∈
Ω
H
~
(
p
)
⋅
p
=
∑
p
∈
Ω
3
H
~
(
p
)
⋅
p
+
∑
p
∈
Ω
1
,
Ω
2
,
Ω
4
H
~
(
p
)
⋅
p
=
w
3
J
3
+
w
1
J
1
+
w
2
J
2
+
w
4
J
4
\begin{aligned} \mathbf{J}^r & =\sum_{\mathbf{p} \in \Omega} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p} \\ & =\sum_{\mathbf{p} \in \Omega_3} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}+\sum_{\mathbf{p} \in \Omega_1, \Omega_2, \Omega_4} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}\\ &= w_3 \mathbf{J}_3 + w_1 \mathbf{J}_1 + w_2 \mathbf{J}_2 + w_4 \mathbf{J}_4 \end{aligned}
Jr=p∈Ω∑H~(p)⋅p=p∈Ω3∑H~(p)⋅p+p∈Ω1,Ω2,Ω4∑H~(p)⋅p=w3J3+w1J1+w2J2+w4J4
其中每个区域的中心点坐标为
J
1
=
(
h
−
2
(
h
−
x
0
)
2
y
0
)
=
(
x
0
−
h
2
y
0
)
J
2
=
(
x
0
−
h
2
2
y
0
+
w
−
2
y
0
2
)
=
(
x
0
−
h
2
y
0
+
w
2
)
J
3
=
(
x
0
y
0
)
J
4
=
(
x
0
y
0
+
w
2
)
\mathbf{J}_1 = \begin{pmatrix} \frac{h - 2(h-x_0)}{2}\\ y_0\\ \end{pmatrix}= \begin{pmatrix} x_0 - \frac{h}{2}\\ y_0\\ \end{pmatrix} \\ \mathbf{J}_2 = \begin{pmatrix} x_0 - \frac{h}{2}\\ 2y_0 + \frac{w - 2y_0}{2}\\ \end{pmatrix}= \begin{pmatrix} x_0 - \frac{h}{2}\\ y_0 + \frac{w}{2}\\ \end{pmatrix}\\ \mathbf{J}_3 = \begin{pmatrix} x_0\\ y_0\\ \end{pmatrix}\\ \mathbf{J}_4 = \begin{pmatrix} x_0\\ y_0 + \frac{w}{2}\\ \end{pmatrix}
J1=(2h−2(h−x0)y0)=(x0−2hy0)J2=(x0−2h2y0+2w−2y0)=(x0−2hy0+2w)J3=(x0y0)J4=(x0y0+2w)
权重
w
1
=
2
(
2
x
0
−
h
)
y
0
C
w
2
=
(
2
x
0
−
h
)
(
w
−
2
y
0
)
C
w
4
=
2
(
h
−
x
0
)
(
w
−
2
y
0
)
C
w
3
=
1
−
w
1
−
w
2
−
w
4
w_1 = \frac{2\left(2x_0 - h\right)y_0}{C}\\ w_2 = \frac{\left(2x_0 - h\right)\left(w - 2y_0\right)}{C}\\ w_4 = \frac{2\left(h - x_0\right)\left(w - 2y_0\right)}{C}\\ w_3 = 1 - w_1 - w_2 - w_4
w1=C2(2x0−h)y0w2=C(2x0−h)(w−2y0)w4=C2(h−x0)(w−2y0)w3=1−w1−w2−w4
(计算方法就是区域的像素个数乘
1
C
\frac{1}{C}
C1
因此
J
r
=
(
w
1
(
x
0
−
h
2
)
+
w
2
(
x
0
−
h
2
)
+
w
3
x
0
+
w
4
x
0
w
1
y
0
+
w
2
(
y
0
+
w
2
)
+
w
3
y
0
+
w
4
(
y
0
+
w
2
)
)
=
(
x
0
−
(
w
1
+
w
2
)
h
2
y
0
+
(
w
2
+
w
4
)
w
2
)
=
(
(
1
−
h
w
C
)
x
0
+
h
w
C
h
2
(
1
−
h
w
C
)
y
0
+
h
w
C
w
2
)
\begin{aligned} \mathbf{J}^r &= \begin{pmatrix} w_1 \left(x_0 - \frac{h}{2}\right) + w_2 \left(x_0 - \frac{h}{2}\right) + w_3 x_0 + w_4 x_0\\ w_1 y_0 + w_2 \left(y_0+\frac{w}{2}\right) + w_3 y_0 + w_4 \left(y_0+\frac{w}{2}\right)\\ \end{pmatrix}\\ &= \begin{pmatrix} x_0 - \left(w_1 + w_2\right)\frac{h}{2}\\ y_0 + \left(w_2 + w_4\right)\frac{w}{2}\\ \end{pmatrix}\\ &=\begin{pmatrix} \left(1-\frac{h w}{C}\right) x_0+\frac{h w}{C} \frac{h}{2}\\ \left(1-\frac{h w}{C}\right) y_0+\frac{h w}{C} \frac{w}{2} \end{pmatrix} \end{aligned}
Jr=(w1(x0−2h)+w2(x0−2h)+w3x0+w4x0w1y0+w2(y0+2w)+w3y0+w4(y0+2w))=(x0−(w1+w2)2hy0+(w2+w4)2w)=((1−Chw)x0+Chw2h(1−Chw)y0+Chw2w)
可以解出来
J
r
o
=
(
x
0
y
0
)
=
(
C
C
−
h
w
x
r
−
h
2
w
2
(
C
−
h
w
)
C
C
−
h
w
y
r
−
h
w
2
2
(
C
−
h
w
)
)
\mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\begin{pmatrix} \frac{C}{C-h w} x_r-\frac{h^2 w}{2(C-h w)} \\ \frac{C}{C-h w} y_r-\frac{h w^2}{2(C-h w)} \end{pmatrix}
Jro=(x0y0)=(C−hwCxr−2(C−hw)h2wC−hwCyr−2(C−hw)hw2)
右下
以
2
(
h
−
x
0
)
,
2
(
w
−
y
0
)
2(h - x_0),2(w-y_0)
2(h−x0),2(w−y0)将整张图划分为4个区域
由于其他地方的值比较接近于0,所以热力图可以近似表示为
H
~
(
p
)
≈
{
exp
(
β
⋅
H
(
p
)
)
C
,
p
∈
Ω
4
1
C
,
p
∉
Ω
4
\tilde{\mathbf{H}} \left(p\right) \approx \begin{cases} \frac{\exp (\beta \cdot \mathbf{H}(\mathbf{p}))}{C},& p\in \Omega_4\\ \frac{1}{C},& p\not\in \Omega_4 \end{cases}
H~(p)≈{Cexp(β⋅H(p)),C1,p∈Ω4p∈Ω4
因此
J
r
=
∑
p
∈
Ω
H
~
(
p
)
⋅
p
=
∑
p
∈
Ω
4
H
~
(
p
)
⋅
p
+
∑
p
∈
Ω
1
,
Ω
2
,
Ω
3
H
~
(
p
)
⋅
p
=
w
4
J
4
+
w
1
J
1
+
w
2
J
2
+
w
3
J
3
\begin{aligned} \mathbf{J}^r & =\sum_{\mathbf{p} \in \Omega} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p} \\ & =\sum_{\mathbf{p} \in \Omega_4} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}+\sum_{\mathbf{p} \in \Omega_1, \Omega_2, \Omega_3} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}\\ &= w_4 \mathbf{J}_4 + w_1 \mathbf{J}_1 + w_2 \mathbf{J}_2 + w_3 \mathbf{J}_3 \end{aligned}
Jr=p∈Ω∑H~(p)⋅p=p∈Ω4∑H~(p)⋅p+p∈Ω1,Ω2,Ω3∑H~(p)⋅p=w4J4+w1J1+w2J2+w3J3
其中每个区域的中心点坐标为
J
1
=
(
h
−
2
(
h
−
x
0
)
2
w
−
2
(
w
−
y
0
)
2
)
=
(
x
0
−
h
2
y
0
−
w
2
)
J
2
=
(
x
0
−
h
2
y
0
)
J
3
=
(
x
0
y
0
−
w
2
)
J
4
=
(
x
0
y
0
)
\mathbf{J}_1 = \begin{pmatrix} \frac{h-2(h-x_0)}{2}\\ \frac{w-2(w-y_0)}{2}\\ \end{pmatrix}= \begin{pmatrix} x_0 - \frac{h}{2}\\ y_0 - \frac{w}{2}\\ \end{pmatrix} \\ \mathbf{J}_2 = \begin{pmatrix} x_0 - \frac{h}{2}\\ y_0\\ \end{pmatrix}\\ \mathbf{J}_3 = \begin{pmatrix} x_0\\ y_0-\frac{w}{2}\\ \end{pmatrix}\\ \mathbf{J}_4 = \begin{pmatrix} x_0\\ y_0\\ \end{pmatrix}
J1=(2h−2(h−x0)2w−2(w−y0))=(x0−2hy0−2w)J2=(x0−2hy0)J3=(x0y0−2w)J4=(x0y0)
权重
w
1
=
(
2
x
0
−
h
)
(
2
y
0
−
w
)
C
w
2
=
2
(
2
x
0
−
h
)
(
w
−
y
0
)
C
w
3
=
2
(
h
−
x
0
)
(
2
y
0
−
w
)
C
w
4
=
1
−
w
1
−
w
2
−
w
3
w_1 = \frac{\left(2x_0 - h\right)\left(2y_0 - w\right)}{C}\\ w_2 = \frac{2\left(2x_0 - h\right)\left(w - y_0\right)}{C}\\ w_3 = \frac{2\left(h - x_0\right)\left(2y_0 - w\right)}{C}\\ w_4 = 1 - w_1 - w_2 - w_3
w1=C(2x0−h)(2y0−w)w2=C2(2x0−h)(w−y0)w3=C2(h−x0)(2y0−w)w4=1−w1−w2−w3
(计算方法就是区域的像素个数乘
1
C
\frac{1}{C}
C1
因此
J
r
=
(
w
1
(
x
0
−
h
2
)
+
w
2
(
x
0
−
h
2
)
+
w
3
x
0
+
w
4
x
0
w
1
(
y
0
−
w
2
)
+
w
2
y
0
+
w
3
(
y
0
−
w
2
)
+
w
4
y
0
)
=
(
x
0
−
(
w
1
+
w
2
)
h
2
y
0
−
(
w
1
+
w
3
)
w
2
)
=
(
(
1
−
h
w
C
)
x
0
+
h
w
C
h
2
(
1
−
h
w
C
)
y
0
+
h
w
C
w
2
)
\begin{aligned} \mathbf{J}^r &= \begin{pmatrix} w_1 \left(x_0 - \frac{h}{2}\right) + w_2 \left(x_0 - \frac{h}{2}\right) + w_3 x_0 + w_4 x_0\\ w_1 \left(y_0 - \frac{w}{2}\right) + w_2 y_0 + w_3 \left(y_0 - \frac{w}{2}\right) + w_4 y_0\\ \end{pmatrix}\\ &= \begin{pmatrix} x_0 - \left(w_1 + w_2\right)\frac{h}{2}\\ y_0 - \left(w_1 + w_3\right)\frac{w}{2}\\ \end{pmatrix}\\ &=\begin{pmatrix} \left(1-\frac{h w}{C}\right) x_0+\frac{h w}{C} \frac{h}{2}\\ \left(1-\frac{h w}{C}\right) y_0+\frac{h w}{C} \frac{w}{2} \end{pmatrix} \end{aligned}
Jr=(w1(x0−2h)+w2(x0−2h)+w3x0+w4x0w1(y0−2w)+w2y0+w3(y0−2w)+w4y0)=(x0−(w1+w2)2hy0−(w1+w3)2w)=((1−Chw)x0+Chw2h(1−Chw)y0+Chw2w)
可以解出来
J
r
o
=
(
x
0
y
0
)
=
(
C
C
−
h
w
x
r
−
h
2
w
2
(
C
−
h
w
)
C
C
−
h
w
y
r
−
h
w
2
2
(
C
−
h
w
)
)
\mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\begin{pmatrix} \frac{C}{C-h w} x_r-\frac{h^2 w}{2(C-h w)} \\ \frac{C}{C-h w} y_r-\frac{h w^2}{2(C-h w)} \end{pmatrix}
Jro=(x0y0)=(C−hwCxr−2(C−hw)h2wC−hwCyr−2(C−hw)hw2)
#!/usr/bin/env python
# _*_ coding:utf-8 _*_
import torch
from heatmap import get_heatmap_coordination_batch
from soft_argmax import spatial_expectation2d, spatial_soft_argmax2d
from torch import tensor
from torch.nn.functional import mse_loss
if __name__ == '__main__':
x = torch.zeros(1, 4, 512, 512, dtype=torch.float).cuda()
batch, channel, height, width = x.shape
x[:, 0, 0, 0] = 1
x[:, 1, 0, width-1] = 1
x[:, 2, height-1, 0] = 1
x[:, 3, height-1, width-1] = 1
beta = tensor(15.0)
bias_coordination = spatial_soft_argmax2d(x, beta, normalized_coordinates=False, remove_bias=False)
remove_bias_coordination = spatial_soft_argmax2d(x, beta, normalized_coordinates=False, remove_bias=True)
remove_bias_coordination = torch.clamp(remove_bias_coordination, min=torch.tensor(0, device=x.device), max=torch.tensor([height, width], device=x.device))
ans = get_heatmap_coordination_batch(x)
print(bias_coordination)
print()
print(remove_bias_coordination)
print()
print(ans)
print()
print(mse_loss(bias_coordination, ans, reduction='sum'))
print()
print(mse_loss(remove_bias_coordination, ans, reduction='sum'))
虽然说,最终确实使得landmark距离真实的landmark比较近,但是在训练的时候loss贼大
然后这个remove bias有可能超出图片范围(比如<0或者>Height)
至于怎么训练,我是没调出来