1 INTRODUCTION
本文是CVPR18的扩展
3 METHOD
3.1 Overview
loss包括the traditional squared Euclidean loss、the GAN loss、the identity loss
结构上,判别器是pyramid-structured discriminator
3.2 Generator
生成器是Encoder-Decoder结构,接收input young face x x x和target age label(or age range) c c c作为输入,首先使用卷积层转换到latent space,然后连接4个residual blocks,最后连接3个反卷积层,生成age progression result y y y,即 G ( x , c ) → y G(x,c)\rightarrow y G(x,c)→y
卷积层之后是Instance Normalization和ReLU,最后一层是total variation regularization layer
3.3 Adversarial Learning
3.3.1 Aging Modeling
定义young faces的分布为 x ∼ P y o u n g x\sim P_{young} x∼Pyoung,generated faces的分布为 G ( x , c ) ∼ P G G(x, c)\sim P_G G(x,c)∼PG,target age faces的分布为 P o l d P_{old} Pold,我们希望 P G = P o l d P_G=P_{old} PG=Pold
在原版GAN的优化目标中,判别器的损失函数如下
L
G
A
N
D
=
−
E
x
∼
P
y
o
u
n
g
,
c
log
[
1
−
D
(
G
(
x
,
c
)
)
]
(
2
)
−
E
x
∼
P
o
l
d
log
[
D
(
x
)
]
\begin{aligned} \mathcal{L}_{GAN_D}=&-\mathbb{E}_{x\sim P_{young},c}\log\left [ 1-D\left ( G\left ( x,c \right ) \right ) \right ] \qquad(2)\\ &-\mathbb{E}_{x\sim P_{old}}\log\left [ D(x) \right ] \end{aligned}
LGAND=−Ex∼Pyoung,clog[1−D(G(x,c))](2)−Ex∼Poldlog[D(x)]
因为JS divergence is locally saturated,所以当 D D D训练得太好时,造成优化 G G G时梯度消失,本文使用Least Squares GAN的版本
对于
D
D
D,将actual young faces和generated age-progressed faces作为阜样本,将true elderly face of age range
c
c
c作为正样本,于是
G
G
G和
D
D
D的优化目标分别为
L
G
A
N
_
G
=
E
x
∼
P
y
o
u
n
g
,
c
H
(
1
,
D
(
ϕ
a
g
e
(
G
(
x
,
c
)
)
)
)
(
3
)
\mathcal{L}_{GAN\_G}=\mathbb{E}_{x\sim P_{young},c}\ H\left ( 1,D\left ( \phi_{age}\left ( G\left ( x,c \right ) \right ) \right ) \right ) \qquad(3)
LGAN_G=Ex∼Pyoung,c H(1,D(ϕage(G(x,c))))(3)
L
G
A
N
_
D
=
E
x
∼
P
y
o
u
n
g
,
y
∼
P
o
l
d
,
c
H
(
[
1
,
0
,
0
]
,
[
D
(
ϕ
a
g
e
(
y
)
)
,
D
(
ϕ
a
g
e
(
G
(
x
,
c
)
)
)
,
D
(
ϕ
a
g
e
(
x
)
)
]
)
(
4
)
\begin{aligned} \mathcal{L}_{GAN\_D}=&\mathbb{E}_{x\sim P_{young},y\sim P_{old},c}\ H([1,0,0],\\ &\left [ D\left ( \phi_{age}(y) \right ), D\left ( \phi_{age}\left ( G\left ( x,c \right ) \right ) \right ), D\left ( \phi_{age}(x) \right ) \right ]) \qquad(4) \end{aligned}
LGAN_D=Ex∼Pyoung,y∼Pold,c H([1,0,0],[D(ϕage(y)),D(ϕage(G(x,c))),D(ϕage(x))])(4)
其中,
H
H
H表示least squares distance,
ϕ
a
g
e
\phi_{age}
ϕage是提取extract age-related features的网络,预先训练一个VGG16进行age classification,然后去掉FC layer
ϕ a g e \phi_{age} ϕage提取的特征包括第2、4、7、10层的feature map,与之对应, D D D有4个分支,每一个分支的输出都是3x3,最终拼接成12x3
D D D的结构是Conv-BN-LeakyReLU
3.3.2 Progressive Aging Modeling
原始的框架如Figure 4(a)所示
比较常见的做法是对D加一个auxiliary classifier,增加the age classification loss L a g e L_{age} Lage,如Figure 4(b)所示
3.4 Identity Preservation
采用the network of deep face descriptor,记为 ϕ i d \phi_{id} ϕid
the identity loss定义如下
L
i
d
e
n
t
i
t
y
=
E
x
∼
P
y
o
u
n
g
,
c
d
(
ϕ
i
d
(
x
)
,
ϕ
i
d
(
G
(
x
,
c
)
)
)
(
9
)
\mathcal{L}_{identity}=\mathbb{E}_{x\sim P_{young}, c}\ d\left ( \phi_{id}(x), \phi_{id}\left ( G(x,c) \right ) \right ) \qquad(9)
Lidentity=Ex∼Pyoung,c d(ϕid(x),ϕid(G(x,c)))(9)
其中
d
d
d表示squared Euclidean distance
3.5 Objective
为了缩小生成图像与原图之间的gap(保证颜色相近),添加一项pixel-wise L2 loss
L
p
i
x
e
l
=
E
x
∼
P
y
o
u
n
g
,
c
1
W
×
H
×
C
∥
G
(
x
,
c
)
−
x
∥
2
2
(
10
)
\mathcal{L}_{pixel}=\mathbb{E}_{x\sim P_{young}, c}\ \frac{1}{W\times H\times C}\left \| G(x,c)-x\right \|_2^2 \qquad(10)
Lpixel=Ex∼Pyoung,c W×H×C1∥G(x,c)−x∥22(10)
借鉴文献[38]中的total variation regularizer loss,增加一项 L t v \mathcal{L}_{tv} Ltv,保证spatial smoothness
最终整个framework的训练目标如下
L
G
=
λ
a
∑
i
L
G
A
N
_
G
i
+
λ
p
L
p
i
x
e
l
+
λ
i
L
i
d
e
n
t
i
t
y
+
λ
t
L
t
v
(
11
)
\mathcal{L}_G=\lambda_a\sum_{i}\mathcal{L}_{GAN\_G_i}+\lambda_p\mathcal{L}_{pixel}+\lambda_i\mathcal{L}_{identity}+\lambda_{t}\mathcal{L}_{tv} \qquad(11)
LG=λai∑LGAN_Gi+λpLpixel+λiLidentity+λtLtv(11)
L
D
i
=
L
G
A
N
_
D
i
(
12
)
\mathcal{L}_{D_i}=\mathcal{L}_{GAN\_D_i} \qquad(12)
LDi=LGAN_Di(12)