1
第l层第j个神经元上的误差
δ
j
l
\delta_j^l
δjl
δ
j
l
≡
∂
C
∂
z
j
l
\delta_j^l\equiv\frac{\partial{C}}{\partial{z_j^l}}
δjl≡∂zjl∂C
δ j L = ∂ C ∂ a j L σ ′ ( z j L ) a j L = σ ( z j L ) z j L = ∑ k w j k L a k L − 1 + b j L \delta_j^L = \frac{\partial{C}}{\partial{a_j^L}}\sigma'(z_j^L)\\ a_j^L = \sigma(z_j^L)\\ z_j^L = \sum_k{w_{jk}^La_k^{L-1}}+b_j^L δjL=∂ajL∂Cσ′(zjL)ajL=σ(zjL)zjL=k∑wjkLakL−1+bjL
δ L = ∇ a C ⊙ σ ′ ( z L ) \delta^L=\nabla_aC\odot\sigma'(z^L) δL=∇aC⊙σ′(zL)
此处 ⊙ \odot ⊙表示矩阵对应位置相乘
2
误差反向传播
δ
l
、
δ
l
+
1
\delta^l、\delta^{l+1}
δl、δl+1之间的关系
δ
l
=
(
(
w
l
+
1
)
T
δ
l
+
1
)
⊙
σ
′
(
z
l
)
z
k
l
+
1
=
∑
j
w
k
j
l
+
1
a
j
l
+
b
k
l
+
1
=
∑
j
w
k
j
l
+
1
σ
(
z
j
l
)
+
b
k
l
+
1
∂
z
k
l
+
1
∂
z
j
l
=
w
k
j
l
+
1
σ
′
(
z
j
l
)
δ
j
l
=
∂
C
∂
z
j
l
=
∑
k
∂
C
∂
z
k
l
+
1
∂
z
k
l
+
1
∂
z
j
l
=
∑
k
∂
z
k
l
+
1
∂
z
j
l
δ
k
l
+
1
=
∑
k
w
k
j
l
+
1
δ
k
l
+
1
σ
′
(
z
j
l
)
\delta^l=((w^{l+1})^T\delta^{l+1})\odot\sigma'(z^l)\\ z_k^{l+1}=\sum_jw_{kj}^{l+1}a_j^l+b_k^{l+1}=\sum_jw_{kj}^{l+1}\sigma(z_j^l)+b_k^{l+1}\\ \frac{\partial z_k^{l+1}}{\partial z_j^l}=w_{kj}^{l+1}\sigma'(z_j^l) \\ \delta_j^l=\frac{\partial{C}}{\partial z_j^l}=\sum_k{\frac{\partial C}{\partial z_k^{l+1}}\frac{\partial z_k^{l+1}}{\partial z_j^l}}=\sum_k{\frac{\partial z_k^{l+1}}{\partial z_j^l}\delta_k^{l+1}}\\=\sum_kw_{kj}^{l+1}\delta_k^{l+1}\sigma'(z_j^l)
δl=((wl+1)Tδl+1)⊙σ′(zl)zkl+1=j∑wkjl+1ajl+bkl+1=j∑wkjl+1σ(zjl)+bkl+1∂zjl∂zkl+1=wkjl+1σ′(zjl)δjl=∂zjl∂C=k∑∂zkl+1∂C∂zjl∂zkl+1=k∑∂zjl∂zkl+1δkl+1=k∑wkjl+1δkl+1σ′(zjl)
3
∂ C ∂ b j l = δ j l ∂ C ∂ z j l = δ j l z j l = ∑ k w j k l a k l − 1 + b j l ∂ z j l ∂ b j l = 1 \frac{\partial C}{\partial b_j^l}=\delta_j^l\\ \frac{\partial C}{\partial z_j^l}=\delta_j^l\\ z_j^l=\sum_kw_{jk}^la_k^{l-1}+b_j^l\\ \frac{\partial z_j^l}{\partial b_j^l} = 1 ∂bjl∂C=δjl∂zjl∂C=δjlzjl=k∑wjklakl−1+bjl∂bjl∂zjl=1
4
∂ C ∂ w j k l = a k l − 1 δ j l ∂ C ∂ w l = δ l ( a l − 1 ) T ∂ z j l ∂ w j k l = a k l − 1 \frac{\partial C}{\partial w_{jk}^l}=a_k^{l-1}\delta_j^l\\ \frac{\partial C}{\partial w^l}=\delta^l(a^{l-1})^T\\ \frac{\partial z_j^l}{\partial w_{jk}^l}=a_k^{l-1} ∂wjkl∂C=akl−1δjl∂wl∂C=δl(al−1)T∂wjkl∂zjl=akl−1
5、反向传播过程
扫一扫
19万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
