RNN-RBM网络介绍及代码分析

官方学习教程的地址：http://deeplearning.net/tutorial/rnnrbm.html#rnnrbm

关于RBM的介绍及代码：http://deeplearning.net/tutorial/rbm.html#rbm

但只从介绍很难明白RNN与RBM究竟怎么在训练过程中互相结合的。

————————————————————————————————————

首先RBM网络：

其参数主要有隐层与显层的关系矩阵W，隐层的偏置bh，显层的偏置bv。

对于输入数据V矩阵，V的每一行代表一个sample，这里针对每个sample，训练的都是同样的一组bh与bv，即从始至终bh与bv都只是一个一维向量。

训练过程中的loss函数如何得到：有样本V，根据Gibbs采样生成的样本V‘，计算V对应的能量函数值，计算V’对应的能量函数值，二者相减即为loss函数。

————————————————————————————————————

LSTM网络学习：http://deeplearning.net/tutorial/lstm.html

————————————————————————————————————

而如何将RNN与RBM相结合呢？

在原有的RBM中，一组bh与bv针对是所有时序的采样，而这里为了体现出时序的特点，采样每个时序对应一组bh与bv。

而具体bh与bv则由RNN的循环递归生成（V也参与其中）。

一个更容易理解的图：

代码：

# Author: Nicolas Boulanger-Lewandowski
# University of Montreal (2012)
# RNN-RBM deep learning tutorial
# More information at http://deeplearning.net/tutorial/rnnrbm.html


from __future__ import print_function


import glob
import os
import sys


import numpy
try:
    import pylab
except ImportError:
    print ("pylab isn't available. If you use its functionality, it will crash.")
    print("It can be installed with 'pip install -q Pillow'")


from midi.utils import midiread, midiwrite
import theano
import theano.tensor as T
from theano.sandbox.rng_mrg import MRG_RandomStreams as RandomStreams


#Don't use a python long as this don't work on 32 bits computers.
numpy.random.seed(0xbeef)
rng = RandomStreams(seed=numpy.random.randint(1 << 30))
theano.config.warn.subtensor_merge_bug = False




def build_rbm(v, W, bv, bh, k):
    '''Construct a k-step Gibbs chain starting at v for an RBM.


    v : Theano vector or matrix
        If a matrix, multiple chains will be run in parallel (batch).
    W : Theano matrix
        Weight matrix of the RBM.
    bv : Theano vector
        Visible bias vector of the RBM.
    bh : Theano vector
        Hidden bias vector of the RBM.
    k : scalar or Theano scalar
        Length of the Gibbs chain.


    Return a (v_sample, cost, monitor, updates) tuple:


    v_sample : Theano vector or matrix with the same shape as `v`
        Corresponds to the generated sample(s).
    cost : Theano scalar
        Expression whose gradient with respect to W, bv, bh is the CD-k
        approximation to the log-likelihood of `v` (training