Hopfield网络是一种循环神经网络，用于内容寻址存储和联想记忆。每个神经元与其他所有神经元都有连接，当信号相反时，神经元会翻转其状态，导致能量函数下降。该网络在每次翻转后都朝着能量最小值演化，从而稳定在存储的模式上。然而，Hopfield网络存在存储容量限制和非正交模式的问题，可能导致错误的记忆召回。
Hopfield Net
- So far, neural networks for computation are all feedforward structures
Loopy network
- Each neuron is a perceptron with +1/-1 output
- Every neuron receives input from every other neuron
- Every neuron outputs signals to every other neuron
- At each time each neuron receives a “field” ∑ j ≠ i w j i y j + b i \sum_{j \neq i} w_{j i} y_{j}+b_{i} ∑j=iwjiyj+bi
- If the sign of the field matches its own sign, it does not respond
- If the sign of the field opposes its own sign, it “flips” to match the sign of the field
- If the sign of the field at any neuron opposes its own sign, it “flips” to match the field
- Which will change the field at other nodes
- Which may then flip… and so on…
Filp behavior
-
Let y i − y^{-}_{i} yi− be the output of the i i i-th neuron just before it responds to the current field
-
Let y i + y_{i}^{+} yi+ be the output of the i i i-th neuron just after it responds to the current field
-
if y i − = sign ( ∑ j ≠ i w j i y j + b i ) y_{i}^{-}=\operatorname{sign}\left(\sum_{j \neq i} w_{j i} y_{j}+b_{i}\right) yi−=sign(∑j=iwjiyj+bi), then y i + = − y i − y_{i}^{+} = -y_{i}^{-} yi+=−yi−
-
If the sign of the field matches its own sign, it does not flip
-
y i + ( ∑ j ≠ i w j i y j + b i ) − y i − ( ∑ j ≠ i w j i y j + b i ) = 0 y_{i}^{+}\left(\sum_{j \neq i} w_{j i} y_{j}+b_{i}\right)-y_{i}^{-}\left(\sum_{j \neq i} w_{j i} y_{j}+b_{i}\right)=0 yi+⎝⎛j=i∑wjiyj+bi⎠⎞−yi−⎝⎛j=i∑wjiyj+bi⎠⎞=0
-
-
if y i − ≠ sign ( ∑ j ≠ i w j i y j + b i ) y_{i}^{-}\neq\operatorname{sign}\left(\sum_{j \neq i} w_{j i} y_{j}+b_{i}\right) yi−=sign(∑j=iwjiyj+bi), then y i + = − y i − y_{i}^{+} = -y_{i}^{-} yi+
最低0.47元/天 解锁文章
260
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
