GAN[NIPS. 2014]

KL divergence，如果KL的值越大，代表2个分布之间的差异越大，KL的值越小，代表2个分布之间的差异最小。

KL divergence:
$$D_{KL}(P||Q)=\sum _{i=1}^{N}P(x_i)log\frac{P(x_i)}{Q(x_i)}$$
JS divergence:
$$\begin{gathered} JSD(P||Q)=\frac{1}{2}D(P||M)+\frac{1}{2}D(Q||M) \\ M=\frac{1}{2}(P+Q) \end{gathered}$$

$$\begin{gathered} G^{\ast }=arg\underset{G}{\min }Div(P_G,P_{data}) \\ x=G(z) \\ {P}_G(x) \end{gathered}$$

$$\begin{array}{rl}{D}^{\ast }& =arg\underset{D}{\max }V(D,G)\\ & =\frac{P_{data(x)}}{P_{data}(x)+P_G(x)}\end{array}$$
when G is fixed:
$$\begin{array}{rl}\underset{D}{\max }V(G,D)& =V(G,D^{\ast })\\ & =E_{x\sim P_{data}}\log (D^{\ast }(x))+E_{x\sim P_G}\log (1-D^{\ast }(x))\\ & =-2log2+2JSD(P_{data}||P_G)\end{array}$$

• Since the Jensen–Shannon divergence between two distributions is always non-negative and zero only when they are equal, we have shown that $C^{\ast }=-log(4)$ is the global minimum of C(G) and that the only solution is $p_g=p_{data}$, i.e., the generative model perfectly replicating the data generating process
• 证明出$$V(D,G)$$和$$div(P_{data},P_G)$$是有关系的。

$$\begin{array}{rl}{G}^{\ast }& =arg\underset{G}{\min }Div(P_G,P_{data})\\ & =arg\underset{G}{\min }\underset{D}{\max }V(G,D)\end{array}$$

Theoretical Results