此文章主要是结合哔站shuhuai008大佬的白板推导视频:深度信念网络_72min
全部笔记的汇总贴:机器学习-白板推导系列笔记
对应花书20.3
一、介绍
DBN(深度信念网络 )的引入开始了当前深度学习的复兴。实质上是一个混合模型
P ( v , h ( 1 ) , h ( 2 ) , h ( 3 ) ) = P ( v ∣ h ( 1 ) , h ( 2 ) , h ( 3 ) ) ⋅ P ( h ( 1 ) , h ( 2 ) , h ( 3 ) ) = P ( v ∣ h ( 1 ) ) ⋅ P ( h ( 1 ) , h ( 2 ) , h ( 3 ) ) = P ( v ∣ h ( 1 ) ) ⋅ P ( h ( 1 ) ∣ h ( 2 ) ) ⋅ P ( h ( 2 ) , h ( 3 ) ) = ∏ i P ( v i ∣ h ( 1 ) ) ⋅ ∏ j P ( h j ( 1 ) ∣ h ( 2 ) ) ⋅ P ( h ( 2 ) , h ( 3 ) ) P(v,h^{(1)},h^{(2)},h^{(3)})=P(v|h^{(1)},h^{(2)},h^{(3)})\cdot P(h^{(1)},h^{(2)},h^{(3)})\\=P(v|h^{(1)})\cdot P(h^{(1)},h^{(2)},h^{(3)})\\=P(v|h^{(1)})\cdot P(h^{(1)}|h^{(2)})\cdot P(h^{(2)},h^{(3)})\\=\prod_iP(v_i|h^{(1)})\cdot\prod_jP(h_j^{(1)}|h^{(2)})\cdot P(h^{(2)},h^{(3)}) P(v,h(1),h(2),h(3))=P(v∣h(1),h(2),h(3))⋅P(h(1),h(2),h(3))=P(v∣h(1))⋅P(h(1),h(2),h(3))=P(v∣h(1))⋅P(h(1)∣h(2))⋅P(h(2),h(3))=i∏P(vi∣h(1))⋅j∏P(hj(1)∣h(2))⋅P(h(2),h(3))
θ = { w ( 1 ) , w ( 2 ) , w