(NIPS 2014) Generative Adversarial Nets
We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G.
The training procedure for G is to maximize the probability of D making a mistake.
This framework corresponds to a minimax two-player game.
Deep generative models have had less of an impact, due to the difficulty of approximating many intractable probabilistic computations that arise in maximum likelihood estimation and related strategies, and due to difficulty of leveraging the benefits of piecewise linear units in the generative context.
We propose a new generative model estimation procedure that sidesteps these difficulties.
The generative model can be thought of as analogous to a team of counterfeiters, trying to produce fake currency and use it without detection, while the discriminative model is analogous to the police, trying to detect the counterfeit currency.
we can train both models using only the highly successful backpropagation and dropout algorithms  and sample from the generative model using only forward propagation. No approximate inference or Markov chains are necessary.
D and G play the following two-player minimax game with value function
Optimizing D to completion in the inner loop of training is computationally prohibitive, and on finite datasets would result in overfitting. Instead, we alternate between k steps of optimizing D and one step of optimizing G.
Figure 1: Generative adversarial nets are trained by simultaneously updating the discriminative distribution (D, blue, dashed line) so that it discriminates between samples from the data generating distribution (black, dotted line)
The lower horizontal line is the domain from which
The horizontal line above is part of the domain of
The upward arrows show how the mapping
G contracts in regions of high density and expands in regions of low density of
(a) Poorly fit model
(b) After updating D
(c) After updating G
(d) Mixed strategy equilibrium
Global Optimality of
Convergence of Algorithm 1
We trained adversarial nets an a range of datasets including MNIST, the Toronto Face Database (TFD) , and CIFAR-10 .