![image-20210623170057542](https://i-blog.csdnimg.cn/blog_migrate/9ff093da496c9ce2ea11948adcbe7802.png)
![image-20210623105436151](https://i-blog.csdnimg.cn/blog_migrate/6530b01281fe0f32b9d53103367f5146.png)
KL divergence,如果KL的值越大,代表2个分布之间的差异越大,KL的值越小,代表2个分布之间的差异最小。
KL divergence:
D
K
L
(
P
∣
∣
Q
)
=
∑
i
=
1
N
P
(
x
i
)
l
o
g
P
(
x
i
)
Q
(
x
i
)
D_{KL}(P||Q) = \sum^N_{i=1} P(x_i)log\frac{P(x_i)}{Q(x_i)}
DKL(P∣∣Q)=i=1∑NP(xi)logQ(xi)P(xi)
JS divergence:
J
S
D
(
P
∣
∣
Q
)
=
1
2
D
(
P
∣
∣
M
)
+
1
2
D
(
Q
∣
∣
M
)
M
=
1
2
(
P
+
Q
)
JSD(P||Q) = \frac{1}{2}D(P||M)+ \frac{1}{2}D(Q||M) \\ M = \frac{1}{2}(P+Q)
JSD(P∣∣Q)=21D(P∣∣M)+21D(Q∣∣M)M=21(P+Q)
G ∗ = a r g min G D i v ( P G , P d a t a ) x = G ( z ) P G ( x ) G^* = arg\min_{G}Div(P_G, P_{data}) \\ x = G(z) \\ P_G(x) G∗=argGminDiv(PG,Pdata)x=G(z)PG(x)
![image-20210622224853677](https://i-blog.csdnimg.cn/blog_migrate/ec0e9406ef42b255ddbedfd620208600.png)
D
∗
=
a
r
g
max
D
V
(
D
,
G
)
=
P
d
a
t
a
(
x
)
P
d
a
t
a
(
x
)
+
P
G
(
x
)
\begin{aligned} D^* &=arg \max_D V(D,G) \\&= \frac{P_{data(x)}}{P_{data}(x)+ P_G(x)} \end{aligned}
D∗=argDmaxV(D,G)=Pdata(x)+PG(x)Pdata(x)
when G is fixed:
max
D
V
(
G
,
D
)
=
V
(
G
,
D
∗
)
=
E
x
∼
P
d
a
t
a
log
(
D
∗
(
x
)
)
+
E
x
∼
P
G
log
(
1
−
D
∗
(
x
)
)
=
−
2
l
o
g
2
+
2
J
S
D
(
P
d
a
t
a
∣
∣
P
G
)
\begin{aligned} \max_D V(G, D)&=V(G,D^*) \\&=E_{x \sim P_{data}} \log(D^*(x)) + E_{x \sim P_G}\log(1-D^*(x)) \\ &= -2log2 +2JSD(P_{data}||P_G) \end{aligned}
DmaxV(G,D)=V(G,D∗)=Ex∼Pdatalog(D∗(x))+Ex∼PGlog(1−D∗(x))=−2log2+2JSD(Pdata∣∣PG)
- Since the Jensen–Shannon divergence between two distributions is always non-negative and zero only when they are equal, we have shown that C ∗ = − l o g ( 4 ) C^∗ = −log(4) C∗=−log(4) is the global minimum of C(G) and that the only solution is p g = p d a t a p_g = p_{data} pg=pdata, i.e., the generative model perfectly replicating the data generating process
- 证明出 V ( D , G ) V(D,G) V(D,G)和 d i v ( P d a t a , P G ) div(P_{data}, P_G) div(Pdata,PG)是有关系的。
G ∗ = a r g min G D i v ( P G , P d a t a ) = a r g min G max D V ( G , D ) \begin{aligned} G^* &= arg \min_G Div(P_G, P_{data}) \\&= arg \min_G \max_D V(G,D) \end{aligned} G∗=argGminDiv(PG,Pdata)=argGminDmaxV(G,D)
![image-20210622223201719](https://i-blog.csdnimg.cn/blog_migrate/fcf6617365b68a03724432364b0f8d67.png)
Theoretical Results
![image-20210623164205863](https://i-blog.csdnimg.cn/blog_migrate/eb5de557c4d47193efebec6f9955afde.png)
![image-20210623170116919](https://i-blog.csdnimg.cn/blog_migrate/de23b96eae4add9193ddabf3f5964539.png)
![image-20210623170129612](https://i-blog.csdnimg.cn/blog_migrate/84c4bbb7d0534e9f03e66879ed958ddb.png)
![image-20210623170144309](https://i-blog.csdnimg.cn/blog_migrate/c86d2dcc16376d8d90132fce6423c319.png)
![image-20210623170154044](https://i-blog.csdnimg.cn/blog_migrate/5ad58009b7238dc6dfb893b7d0a041b4.png)
![image-20210623170208532](https://i-blog.csdnimg.cn/blog_migrate/fab2f8cd74ada800c2a6f8ff3ce47e30.png)
![image-20210623170311425](https://i-blog.csdnimg.cn/blog_migrate/7292ae6ee04872c14fa30d18ed3e39f7.png)
![image-20210623170315938](https://i-blog.csdnimg.cn/blog_migrate/2fe44ebb63a5c50627e9e51702dc9b6a.png)