tensorflow玻尔兹曼机_玻尔兹曼机在TensorFlow中的例子

Boltzmann Machines

This repository implements generic and flexible RBM and DBM models with lots of features and reproduces some experiments from "Deep boltzmann machines" [1], "Learning with hierarchical-deep models" [2], "Learning multiple layers of features from tiny images" [3], and some others.

Table of contents

What's Implemented

Restricted Boltzmann Machines (RBM)

k-step Contrastive Divergence;

whether to sample or use probabilities for visible and hidden units;

variable learning rate, momentum and number of Gibbs steps per weight update;

regularization: L2 weight decay, dropout, sparsity targets;

different types of stochastic layers and RBMs: implement new type of stochastic units or create new RBM from existing types of units;

predefined stochastic layers: Bernoulli, Multinomial, Gaussian;

predefined RBMs: Bernoulli-Bernoulli, Bernoulli-Multinomial, Gaussian-Bernoulli;

initialize weights randomly, from np.ndarray-s or from another RBM;

can be modified for greedy layer-wise pretraining of DBM (see notes or [1] for details);

visualizations in Tensorboard (hover images for details) and more:

Deep Boltzmann Machines (DBM)

EM-like learning algorithm based on PCD and mean-field variational inference [1];

arbitrary number of layers of any types;

initialize from greedy layer-wise pretrained RBMs (no random initialization for now);

whether to sample or use probabilities for visible and hidden units;

variable learning rate, momentum and number of Gibbs steps per weight update;

regularization: L2 weight decay, maxnorm, sparsity targets;

estimate partition function using Annealed Importance Sampling [1];

estimate variational lower-bound (ELBO) using logẐ (currently only for 2-layer binary BM);

generate samples after training;

initialize negative particles (visible and hidden in all layers) from data;

DBM class can be used also for training RBM and its features: more powerful learning algorithm, estimating logẐ and ELBO, generating samples after training;

visualizations in Tensorboard (hover images for details) and more:

Common features

easy to use with sklearn-like interface;

easy to load and save models;

easy to reproduce (random_seed make reproducible both TensorFlow and numpy operations inside the model);

all models suppo