NICE: NON-LINEAR INDEPENDENT COMPONENTS ESTIMATION

NICE: NON-LINEAR INDEPENDENT COMPONENTS ESTIMATION

Laurent Dinh, David Krueger, Yoshua Bengio

Abstract

modeling complex high-dimensional densities
basic assumption: a good representation is one in which the data has a distribution that is easy to model
maps it to a latent space so as to make the transformed data conform to a factorized distribution
computing the determinant of the Jacobian and inverse Jacobian is trivial, yet maintain the ability to learn complex non-linear transformations

(a) $\max \log p_X(x)=\log p_H(f(x)) + \log \lvert \det(\frac{\partial f}{\partial x}) \rvert$
(b) $h\sim p_H(h), x = f^{-1}(h)$
(c) $\max_{x_H} \log p_X((x_O,x_H))$

Introduction

ask the learner to find a transformation $h = f(x)$ of the data into a new space such that the resulting distribution factorizes, i.e., the components $h_d$ are independent

$$p_H(h)=\prod _d p_{H_d}(h_d)$$

$h=f(x)$, $f$ is invertible, $h$ and $x$ have the same dimension