假设:
我们的神经网络:
- nx 个输入,即n ^ [0]个输入
- n1 个hidden Layer
- n2 个output
那么对应有参数:
- W ^ [1] , b ^ [1] ,W ^ [2] , b ^ [2]
- 对应参数的维度:
- W ^ [1] :n1 * n0
- W ^ [2] :n2 * n1
- b ^ [1] : n1 * 1
- b ^ [2] : n2 * 1
costFunction : J ( W ^ [1] , b ^ [1] ,W ^ [2] , b ^ [2] ) = 1 / m ∑ L ( yhat,y ) 其中yhat = a ^ [2]
梯度下降
repeat {
- 计算 yhat ( shape 为 (1,m) )
- 计算 dw[1] = dJ / dw[1]
- 计算 db[1] = dJ / db[1]
- 计算 dw[2] , db[2]
- 更新 : w[1] , w[2] , b[1] , b[2]
- w[1] := w[1] - α * dw[1] …w[2] …b[1]…b[2]…
}
导数计算公
前向传播
Z[1] = w[1] X + b[1]
A[1] = g[1] ( Z[1] )
Z[2] = w[2] A[1] + b[2]
A[2] = g[2] ( Z[2] ) = sigmoid( Z[2] )——如果是二元分类的话,否则不是sigmoid
反向传播
Y = [y(1) y(2) … y(m) ]
dZ[2] = A[2] - Y
dw[2] = 1/m dZ[2] A[1].T
db[2] = 1/m np.sum(dZ[2] , axis =1,keepdims = true)
dZ[1] = w[2].T dZ[2] * g[1] ’ ( Z[1] )
dw[1] = 1/m dZ[1] X.T ——dZ[1].shape = (n[1],1)
db[1] = 1/m np.sum(dZ[1] , axis =1,keepdims = true)