【论文阅读笔记】论文总结 AdaGAN、GAIL、SeqGAN等四篇

论文总结

AdaGAN: Boosting Generative Models

AdaGAN论文阅读笔记

A Connection Between GAN, IRL and EBM

###Motivation

Then maximizing likelihood will lead to a distribution which “covers” all of the modes, but puts most of its mass in parts of the space that have negligible density under the data distribution.

A generator trained adversarially will instead try to “fill in” as many of modes as it can.

Complex multimodal distribution很难通过maximize likelihood评估密度分布。它会将尽可能地覆盖$P_{data}$而使其$P_G$主要分布在各个mode之间的空白区域；而GAN则会尽量填满$P_{data}$而使$P_G$更接近$P_{data}$。

###Summary

IRL methods are in fact mathematically equivalent to GANs.

In particular, a sample-based algorithm for maximum entropy IRL and a GAN.

Definition

Boltzmann distribution

$p_{\theta }(\tau )=\frac{1}{Z}{e}^{-E_{\theta }(\tau )}$.

Partition function

$Z=\int e^{-E_{\theta }(x)}dx$.

Discriminator loss

${\mathcal{L}}_{discriminator}(D)={\mathbb{E}}_{x\sim p}[-\log D(x)]+{\mathbb{E}}_{x\sim G}[-\log (1-D(x))]$.

Generator loss

${\mathcal{L}}_{generator}(G)={\mathbb{E}}_{x\sim G}[-\log D(x)]+{\mathbb{E}}_{x\sim G}[\log (1-D(x))]$.

Calculation

使用cost function $c_{\theta }$表示energy $E_{\theta }$。

$Z=\int e^{-c_{\theta }(\tau )}d\tau$

使用sampling distribution $q(\tau )$估算$Z$。

${\mathcal{L}}_{cost}(\theta )={\mathbb{E}}_{\tau \sim p}[c_{\theta }(\tau )]+\log ({\mathbb{E}}_{\tau \sim q}[\frac{e^{-c_{\theta }(\tau )}}{q(\tau )}])$

通常最小化$q(\tau )$与$\frac{1}{Z}{e}^{-c_{\theta (\tau )}}$的KL散度来更新$q(\tau )$，其等价于最小化learned cost同时最大化熵值（最小化交叉熵等价于最小化KL散度，花书P49）。

${\mathcal{L}}_{sampler}(q)={\mathbb{E}}_{\tau \sim q}[c_{\theta }(\tau )]+{\mathbb{E}}_{\tau \sim q}[\log q(\tau )]$

为了防止