【c1w4】- 吴恩达深度学习作业总结
文章目录
0.总结
1.L-layer网络
输入为 X (12888,209)
m = 209 例子
2.前向传播
1.Linear Forward
Z [ l ] = W [ l ] A [ l − 1 ] + b [ l ] Z^{[l]} = W^{[l]}A^{[l-1]} +b^{[l]} Z[l]=W[l]A[l−1]+b[l]
2.Linear-Activation Forward
Sigmoid
ReLu
A [ l ] = g ( Z [ l ] ) = g ( W [ l ] A [ l − 1 ] + b [ l ] ) A^{[l]} = g(Z^{[l]}) = g(W^{[l]}A^{[l-1]} +b^{[l]}) A[l]=g(Z[l])=g(W[l]A[l−1]+b[l])
激活函数"g" 可以是 sigmoid() 或者 relu()
3.L-Layer Model
- [LINEAR -> RELU] × (L-1) -> LINEAR -> SIGMOID model
4.代价函数
− 1 m ∑ i = 1 m ( y ( i ) log ( a [ L ] ( i ) ) + ( 1 − y ( i ) ) log ( 1 − a [ L ] ( i ) ) ) -\frac{1}{m} \sum\limits_{i = 1}^{m} (y^{(i)}\log\left(a^{[L] (i)}\right) + (1-y^{(i)})\log\left(1- a^{[L](i)}\right)) −m1i=1∑m(y(i)log(a[L](i))+(1−y(i))log(1−a[L](i)))
'''交叉熵'''
cost = - 1/m* np.sum((Y*np.log(AL))+(1-Y)*np.log(1-AL))
5.反向传播
1.Linear backward
dz 和 cache 求解 dW, db, da
d W [ l ] = ∂ L ∂ W [ l ] = 1 m d Z [ l ] A [ l − 1 ] T d b [ l ] = ∂ L ∂ b [ l ] = 1 m ∑ i = 1 m d Z [ l ] ( i ) d A [ l − 1 ] = ∂ L ∂ A [ l − 1 ] = W [ l ] T d Z [ l ] \begin{aligned} &dW^{[l]} = \frac{\partial \mathcal{L} }{\partial W^{[l]}} = \frac{1}{m} dZ^{[l]} A^{[l-1] T} \\ &db^{[l]} = \frac{\partial \mathcal{L} }{\partial b^{[l]}} = \frac{1}{m} \sum_{i = 1}^{m} dZ^{[l](i)} \\ &dA^{[l-1]} = \frac{\partial \mathcal{L} }{\partial A^{[l-1]}} = W^{[l] T} dZ^{[l]} \end{aligned} dW[l]=∂W[l]∂L=m1dZ[l]A[l−1]Tdb[l]=∂b[l]∂L=m1i=1∑mdZ[l](i)dA[l−1]=∂A[l−1]∂L=W[l]TdZ[l]
2.Linear-Activation backward
d Z [ l ] = d A [ l ] ∗ g ′ ( Z [ l ] ) dZ^{[l]} = dA^{[l]} * g'(Z^{[l]}) dZ[l]=dA[l]∗g′(Z[l])
6.参数更新
W [ l ] = W [ l ] − α d W [ l ] b [ l ] = b [ l ] − α d b [ l ] \begin{aligned} &W^{[l]} = W^{[l]} - \alpha \text{ } dW^{[l]} \\ &b^{[l]} = b^{[l]} - \alpha \text{ } db^{[l]} \end{aligned} W[l]=W[l]−α dW[l]b[l]=b[l]−α db[l]
1.难点说明
1.np.sum()
d b [ l ] = ∂ L ∂ b [ l ] = 1 m ∑ i = 1 m d Z [ l ] ( i ) db^{[l]} = \frac{\partial \mathcal{L} }{\partial b^{[l]}} = \frac{1}{m} \sum_{i = 1}^{m} dZ^{[l](i)} db[l]=∂b[l]∂L=m1i=1∑mdZ[l](i)
# 压缩成一列
np.sum(dZ, axis=1, keepdim=True)
2.反向传播
对代价函数求导
'''对交叉熵的求导''' dAL = - (np.divide(Y, AL) - np.divide(1 - Y, 1 - AL))