# RBM深层理解

The thing that settles down is the probability distribution（P(v,h)） over configurations（给定可见层和隐层单元的一个指派）. That's a difficult concept the first time you meet it, and so I'm going to giveyou an example. The probability distribution settles to a particular distribution called the Stationary Distribution. The stationary distribution is determined by the energy function of the system.(当然包括权重W了,且W是固定的) And, in fact,in the stationary distribution, the probability of any configuration is proportional to each of the minus its energy. A nice intuitive way to think about thermal equilibrium is to imagine a huge ensemble ofidentical systems that all have exactly the same energy function. So, imagine a very large number of stochastic Hopfield nets all with the same weights. Now, in that huge ensemble(全体), we can define the probability of configuration as the fraction of the systems that are in that configuration.

So, now we canunderstand what's happening as we approach thermal equilibrium.

over all these identical systems. We could

make them all,be in the same

configuration.So, that's the distribution

with a property of one on one

configuration,and zero on everything

else. Or we could start them off, with an

equal number ofsystems in each possible

configuration.

So that's auniform distribution.

And then, we're going to keep applying our

stochastic update rule.

Which, in the case of a stochastic

Hopfield net would mean,

You pick a unit, and you look at its

energy gap.

And you make a random decision based on

that energy gap about whether to turn it

on orturn it off.

Then, you go and pick another unit, and so

on.

We keep applying that stochastic rule.

And after we've run systems stochastically

in this way,

We may eventually reach a situation where

the fraction of the systems in each

configuration remains constant.

We apply the update rule,

And the states of its units will keep

flipping between zero and one.

But, the fraction of systems in any

particular configuration doesn't change.

And that's because we have many, many more

systems than we have configurations.

In general, we're interested in reaching

equilibrium for systems where some

configurations have lower energy than

others.

We want to maximize the product of the probabilities that the Boltzmann machine assigns to the binary vectors in the training set.
– This is equivalent to maximizing the sum of the
log probabilities that the Boltzmann machine
assigns to the training vectors.

It is also equivalent to maximizing the probability that we would obtain exactly the N training cases if we did the following
– Let the network settle to its stationary distribution N
different times with no external input.
– Sample the visible vector  once each time.

An inefficient way to collect the statistics required for learning

Positive phase: Clamp a data
vector on the visible units and set
the hidden units to random
binary states.
– Update the hidden units one
at a time until the network
reaches thermal equilibrium
at a temperature of 1.
– Sample for every
connected pair of units.
– Repeat for all data vectors in
the training set and average

Negative phase: Set all the
units to random binary states.
– Update all the units one at
a time until the network
reaches thermal
equilibrium at a
temperature of 1.
– Sample for every
connected pair of units.
– Repeat many times (how
many?) and average to get
good estimates.

• 本文已收录于以下专栏：

举报原因： 您举报文章：RBM深层理解 色情 政治 抄袭 广告 招聘 骂人 其他 (最多只允许输入30个字)