RBM深层理解

原创 2016年08月30日 10:32:08
要理解RBM,不得不首先理解什么是热平衡状态(thermal equilibrium),

 The thing that settles down is the probability distribution(P(v,h)) over configurations(给定可见层和隐层单元的一个指派). That's a difficult concept the first time you meet it, and so I'm going to giveyou an example. The probability distribution settles to a particular distribution called the Stationary Distribution. The stationary distribution is determined by the energy function of the system.(当然包括权重W了,且W是固定的) And, in fact,in the stationary distribution, the probability of any configuration is proportional to each of the minus its energy. A nice intuitive way to think about thermal equilibrium is to imagine a huge ensemble ofidentical systems that all have exactly the same energy function. So, imagine a very large number of stochastic Hopfield nets all with the same weights. Now, in that huge ensemble(全体), we can define the probability of configuration as the fraction of the systems that are in that configuration.

So, now we canunderstand what's happening as we approach thermal equilibrium.

We can start with any distribution we like

over all these identical systems. We could

make them all,be in the same

configuration.So, that's the distribution

with a property of one on one

configuration,and zero on everything

else. Or we could start them off, with an

equal number ofsystems in each possible

configuration.

So that's auniform distribution.

And then, we're going to keep applying our

stochastic update rule.

Which, in the case of a stochastic

Hopfield net would mean,

You pick a unit, and you look at its

energy gap.

And you make a random decision based on

that energy gap about whether to turn it

on orturn it off.

Then, you go and pick another unit, and so

on.

We keep applying that stochastic rule.

And after we've run systems stochastically

in this way,

We may eventually reach a situation where

the fraction of the systems in each

configuration remains constant.

We apply the update rule,

And the states of its units will keep

flipping between zero and one.

But, the fraction of systems in any

particular configuration doesn't change.

And that's because we have many, many more

systems than we have configurations.

In general, we're interested in reaching

equilibrium for systems where some

configurations have lower energy than

others.


如果理解了上面的概念,那试着看看下面的说法能否理解:

We want to maximize the product of the probabilities that the Boltzmann machine assigns to the binary vectors in the training set.
– This is equivalent to maximizing the sum of the
log probabilities that the Boltzmann machine
assigns to the training vectors.

It is also equivalent to maximizing the probability that we would obtain exactly the N training cases if we did the following
– Let the network settle to its stationary distribution N
different times with no external input.
– Sample the visible vector  once each time.

第一个概念很容易理解,第二个是什么意思呢,通过从stationary distribution抽样得到的visible vector和我们的 training data visible vector用于学习BM网络效果是一样的,也就是最后得到的网络是一致的


下面让我们看一个有趣的事


权重更新有两项,第一项固定V可见单元达到热平衡状态,第二项不固定v达到热平衡状态,为什么要减去第二项呢,因为第二项与V无关,即使没有V,si和sj同时为1的期望,不减掉的话就多学习了


An inefficient way to collect the statistics required for learning

Positive phase: Clamp a data
vector on the visible units and set
the hidden units to random
binary states.
– Update the hidden units one
at a time until the network
reaches thermal equilibrium
at a temperature of 1.
– Sample for every
connected pair of units.
– Repeat for all data vectors in
the training set and average

Negative phase: Set all the
units to random binary states.
– Update all the units one at
a time until the network
reaches thermal
equilibrium at a
temperature of 1.
– Sample for every
connected pair of units.
– Repeat many times (how
many?) and average to get
good estimates.


Deep learning:十九(RBM简单理解)

这篇博客主要用来简单介绍下RBM网络,因为deep learning中的一个重要网络结构DBN就可以由RBM网络叠加而成,所以对RBM的理解有利于我们对DBN算法以及deep learning算法的进...
  • xyz568760310
  • xyz568760310
  • 2015年01月07日 15:22
  • 1519

Autoencoder与RBM结合使用

AutoEncoder是多层神经网络,其中输入层和输出层表示相同的含义,具有相同的节点数。AutoEncode学习的是一个输入输出相同的“恒等函数”。不过输入和输出相同,使得这个网络的输出没有任何意义...
  • wtq1993
  • wtq1993
  • 2016年05月26日 20:39
  • 3625

RBM应用于推荐系统

不错的一篇文章,将RBM应用于推荐系统 摘要:与传统的浅层机器学习相比,深度学习具有优秀的自动提取抽象特征的能力,并且随着分布式计算平台的快速发展,大数据的处理能力得到极大的提升,使得近年来DL...
  • u012108367
  • u012108367
  • 2017年04月13日 11:29
  • 1022

无监督学习之RBM和AutoEncoder

有标签的数据固然好,可是一方面打标签的代价太高,另一方面大部分数据是无标签的。这样就涉及到无监督、半监督、Self-taught学习的问题。本文将介绍两种适用于无标签数据的学习方法,可以找到数据中的隐...
  • Young_Gy
  • Young_Gy
  • 2017年04月19日 16:13
  • 3390

使用CD-K算法实现RBM

#encoding:utf-8 import matplotlib.pylab as plt import numpy as np import random from scipy.linalg im...
  • liugan5371375
  • liugan5371375
  • 2016年04月29日 09:48
  • 2321

数据库原理,有深度好文

转自:http://blog.jobbole.com/100349/ 一提到关系型数据库,我禁不住想:有些东西被忽视了。关系型数据库无处不在,而且种类繁多,从小巧实用的 SQLite 到强大...
  • sinat_33363493
  • sinat_33363493
  • 2016年07月10日 09:08
  • 1128

简单易学的机器学习算法——受限玻尔兹曼机RBM

yi
  • google19890102
  • google19890102
  • 2016年07月26日 18:52
  • 5810

限制玻尔兹曼机(RBM)学习笔记

一、预备知识 1. 结构 限制玻尔兹曼机,主要将玻尔兹曼机限制了2点:1.变成二分图 2. 各边内不准连接。 v层:可见层,输入特征。(好比黑白图片,v层就是某处是...
  • sinat_23137713
  • sinat_23137713
  • 2016年10月19日 16:27
  • 1418

C++浅层与深层复制构造函数

//浅层复制构造函数代码 #include using namespace std; class Test { public: Test() { x=new int; ...
  • liupengcheng201
  • liupengcheng201
  • 2014年03月14日 22:16
  • 1135

RBM(受限玻尔兹曼机)解析

1.RBM结构RBM包括隐层、可见层和偏置层。与前馈神经网络不一样,RBM在可见层和隐层间的链接方向不定的(值可以双向传播,隐层—>可见层和可见层—>隐层)和完全链接的。 Boltzmann分布:...
  • zhoutaochun
  • zhoutaochun
  • 2016年12月14日 22:11
  • 1070
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:RBM深层理解
举报原因:
原因补充:

(最多只允许输入30个字)