已知:
Z
b
×
m
[
l
]
=
W
b
×
a
[
l
]
⋅
A
a
×
m
[
l
−
1
]
+
v
⃗
b
×
1
A
b
×
m
[
l
]
=
g
(
Z
[
l
]
)
Z_{b\times m}^{[l]} = W_{b\times a}^{[l]} \cdot A_{a\times m}^{[l-1]} + \vec{v}_{b \times 1}\\ A_{b\times m}^{[l]} = g(Z^{[l]})
Zb×m[l]=Wb×a[l]⋅Aa×m[l−1]+vb×1Ab×m[l]=g(Z[l])
g ( . ) g(.) g(.) 是 activation function, d W dW dW 表示 d J d W \frac{dJ}{dW} dWdJ，其他的类似。
已知 d A [ l ] dA^{[l]} dA[l]，则:
d Z b × m [ l ] = d A [ l ] ∗ g ′ ( Z [ l ] ) d W b × a [ l ] = ∂ L ∂ W [ l ] = 1 m d Z [ l ] A [ l − 1 ] T d b b × 1 [ l ] = ∂ L ∂ b [ l ] = 1 m ∑ i = 1 m d Z [ l ] ( i ) ( 横向求和 ) d A a × m [ l − 1 ] = ∂ L ∂ A [ l − 1 ] = W [ l ] T d Z [ l ] dZ^{[l]}_{b\times m} = dA^{[l]} * g'(Z^{[l]})\\ dW^{[l]}_{b\times a} = \frac{\partial \mathcal{L} }{\partial W^{[l]}} = \frac{1}{m} dZ^{[l]} A^{[l-1] T}\\ db^{[l]}_{b\times 1} = \frac{\partial \mathcal{L} }{\partial b^{[l]}} = \frac{1}{m} \sum_{i = 1}^{m} dZ^{[l](i)} (\text{横向求和})\\ dA^{[l-1]}_{a\times m} = \frac{\partial \mathcal{L} }{\partial A^{[l-1]}} = W^{[l] T} dZ^{[l]} dZb×m[l]=dA[l]∗g′(Z[l])dWb×a[l]=∂W[l]∂L=m1dZ[l]A[l−1]Tdbb×1[l]=∂b[l]∂L=m1i=1∑mdZ[l](i)(横向求和)dAa×m[l−1]=∂A[l−1]∂L=W[l]TdZ[l]
1041
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
