Coursera deeplearning.ai 深度学习笔记1-3-Shallow Neural Networks-浅层神经网络原理推导与代码实现

在掌握了逻辑回归算法后，先来学习浅层神经网络，之后再对深度神经网络进行学习。

1. 原理推导

1.1 神经网络表示

神经网络由输入层、隐含层和输出层构成。L层神经网络，隐含层为第1 ~ (L - 1)层，输出层为第L层。为了方便，将输入层写成第0层。
定义：上标[l]表示第l层，下标j表示第j个节点。
例如，下图为2层神经网络，包含1个隐藏层：

输入层和隐含层可以写成：
${a^{\left[ 0 \right]}} = x = \left[ \begin{array}{l}{x_1}\\{x_2}\\{x_3}\end{array} \right],\;\;{a^{\left[ 1 \right]}} = \left[ \begin{array}{l}a_1^{\left[ 1 \right]}\\a_2^{\left[ 1 \right]}\\a_3^{\left[ 1 \right]}\\a_4^{\left[ 1 \right]}\end{array} \right]\tag{1}$

1.2 正向传播(Forward Propagation)

计算过程与逻辑回归类似，只是多了隐含层，如下：
$\begin{array}{l}z_1^{\left[ 1 \right]} = w_1^{\left[ 1 \right]T}x + b_1^{\left[ 1 \right]},\;\;a_1^{\left[ 1 \right]} = g\left( {z_1^{\left[ 1 \right]}} \right)\\z_2^{\left[ 1 \right]} = w_2^{\left[ 1 \right]T}x + b_2^{\left[ 1 \right]},\;\;a_2^{\left[ 1 \right]} = g\left( {z_2^{\left[ 1 \right]}} \right)\\z_3^{\left[ 1 \right]} = w_3^{\left[ 1 \right]T}x + b_3^{\left[ 1 \right]},\;\;a_3^{\left[ 1 \right]} = g\left( {z_3^{\left[ 1 \right]}} \right)\\z_4^{\left[ 1 \right]} = w_4^{\left[ 1 \right]T}x + b_4^{\left[ 1 \right]},\;\;a_4^{\left[ 1 \right]} = g\left( {z_4^{\left[ 1 \right]}} \right)\end{array}\tag{2}$
式中，g(z)为激活函数。可将上式向量化为：
$\left[ \begin{array}{l}z_1^{\left[ 1 \right]}\\z_2^{\left[ 1 \right]}\\z_3^{\left[ 1 \right]}\\z_4^{\left[ 1 \right]}\end{array} \right] = \left[ {\begin{array}{*{20}{c}} - &{w_1^{\left[ 1 \right]T}}& - \\ - &{w_2^{\left[ 1 \right]T}}& - \\ - &{w_3^{\left[ 1 \right]T}}& - \\ - &{w_4^{\left[ 1 \right]T}}& - \end{array}} \right]\left[ \begin{array}{l}{x_1}\\{x_2}\\{x_3}\end{array} \right] + \left[ \begin{array}{l}b_1^{\left[ 1 \right]}\\b_2^{\left[ 1 \right]}\\b_3^{\left[ 1 \right]}\\b_4^{\left[ 1 \right]}\end{array} \right],\;\;\left[ \begin{array}{l}a_1^{\left[ 1 \right]}\\a_2^{\left[ 1 \right]}\\a_3^{\left[ 1 \right]}\\a_4^{\left[ 1 \right]}\end{array} \right] = g\left( {\left[ \begin{array}{l}z_1^{\left[ 1 \right]}\\z_2^{\left[ 1 \right]}\\z_3^{\left[ 1 \right]}\\z_4^{\left[ 1 \right]}\end{array} \right]} \right)\tag{3}$
即：
${z^{\left[ 1 \right]}} = {W^{\left[ 1 \right]}}x + {b^{\left[ 1 \right]}},\;\;{a^{\left[ 1 \right]}} = g\left( {{z^{\left[ 1 \right]}}} \right)\tag{4}$
式中，
z[1]=⎡⎣⎢⎢