Flow-based Generative Model

Generative models

• Auto-regressive Model(Component-by-Component)
  • What is the best order for the component?
  • Slow generation
• Variational Auto-encoder
  • Optimizing the lower bound.
• Generative Adversasial Nerwork
  • Unstable training
• Flow-based Generative Model

Flow-based Generative Model

A generator G is a network.The network defines a probability distribution $p_G$

$$z\sim \pi (z)$$

$$x=G(z)\sim p_G(x)$$

$$p_G(x)=\pi (z)|det(J_{G^{-1}})|$$

G ∗ = a r g m a x G ∑ i = 1 m l o g p G ( x i ) ,   { x 1 , x 2 , . . . , x m } f r o m   p d a t a ( x ) G^* = arg max_G\sum_{i=1}^mlogp_G(x^i),\space \{x^1,x^2,...,x^m\}from\space p_{data}(x) G∗=argmaxG​i=1∑m​logpG​(xi), {x1,x2,...,xm}from pdata​(x)

$$=argma{x}_G\sum _{i=1}^m(log\pi (z)+log|det(J_{f^{-1}})|)$$

$$=argma{x}_G\sum _{i=1}^m(log\pi (G^{-1}(x_i))+log|det(J_{G^{-1}})|)$$

$$\approx argmi{n}_{G}KL(p_{data}||p_G)$$

Chain of generator $G_h,h=1,...,K$:

$$logp(x_i)=log\pi (z_i)+\sum _{h=1}^{K}log(det(J_{G_h^{-1}}))$$

to be continued…