神经网络笔记 - 反向传播(BackPropagation) 续

继续来证明BP3和BP4.

$$\frac{\partial C}{\partial b_j^l} = \frac{\partial C}{\partial a_j^l}\frac{\partial a_j^l}{\partial z_j^l}\frac{\partial z_j^l}{\partial b_j^l}$$
因为:
$$z_j^l = w_j^la^{l-1} + b_j^l$$
所以
$$\frac{\partial z_j^l}{\partial b_j^l} = 1$$
代入上式, 可得:
$$\frac{\partial C}{\partial b_j^l} = \frac{\partial C}{\partial a_j^l}\frac{\partial a_j^l}{\partial z_j^l} = \delta_j^l$$
这就是公式 BP3.

来证明公式BP4

$$\frac{\partial C}{\partial w_{jk}^l} = \frac{\partial C}{\partial a_j^l}\frac{\partial a_j^l}{\partial z_j^l}\frac{\partial z_j^l}{\partial w_{jk}^l}$$
因为:
$$z_j^l = \sum_kw_{jk}^la_k^{l-1} + b_j^l$$
所以
$$\frac{\partial z_j^l}{\partial w_{jk}^l} = a_k^{l-1}$$
代入上式, 可得:
$$\frac{\partial C}{\partial w_{jk}^l} = \frac{\partial C}{\partial a_j^l}\frac{\partial a_j^l}{\partial z_j^l}a_k^{l-1}=\delta_j^la_k^{l-1}$$
这就是公式BP4

Reference

• http://neuralnetworksanddeeplearning.com/ 强烈推荐
• 机器学习 - 周志华 清华大学出版社
• https://en.wikipedia.org/wiki/Hadamard_product_(matrices)