1. What is f-divergence ?
《f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization》, NIPS, 2016
P and
f is convex,
①if
f(x)=xlogx
——(KL divergence)
Df(P||Q)=∫xq(x)p(x)q(x)log(p(x)q(x))dx=∫xp(x)log(p(x)q(x))dx
②if f(x)=−logx ——( Reverse KL divergence)
Df(P||Q)=∫xq(x)(−log(p(x)q(x)))dx=∫xq(x)log(q(x)p(x))dx
②if f(x)=(x−1)2 ——( Chi Square divergence)
Df(P||Q)=∫xq(x)(p(x)q(x)−1)2dx=∫x(p(x)−q(x))2q(x)dx
2.Fenchel Conjugate (Fenchel共轭)
3. Connection with GAN