# Coursera吴恩达《神经网络与深度学习》课程笔记（5）-- 深层神经网络

### 2. Forward Propagation in a Deep Network

${z}^{\left[1\right]}={W}^{\left[1\right]}x+{b}^{\left[1\right]}={W}^{\left[1\right]}{a}^{\left[0\right]}+{b}^{\left[1\right]}$

${a}^{\left[1\right]}={g}^{\left[1\right]}\left({z}^{\left[1\right]}\right)$

${z}^{\left[2\right]}={W}^{\left[2\right]}{a}^{\left[1\right]}+{b}^{\left[2\right]}$

${a}^{\left[2\right]}={g}^{\left[2\right]}\left({z}^{\left[2\right]}\right)$

${z}^{\left[3\right]}={W}^{\left[3\right]}{a}^{\left[2\right]}+{b}^{\left[3\right]}$

${a}^{\left[3\right]}={g}^{\left[3\right]}\left({z}^{\left[3\right]}\right)$

${z}^{\left[4\right]}={W}^{\left[4\right]}{a}^{\left[3\right]}+{b}^{\left[4\right]}$

${a}^{\left[4\right]}={g}^{\left[4\right]}\left({z}^{\left[4\right]}\right)$

${Z}^{\left[1\right]}={W}^{\left[1\right]}X+{b}^{\left[1\right]}={W}^{\left[1\right]}{A}^{\left[0\right]}+{b}^{\left[1\right]}$

${A}^{\left[1\right]}={g}^{\left[1\right]}\left({Z}^{\left[1\right]}\right)$

${Z}^{\left[2\right]}={W}^{\left[2\right]}{A}^{\left[1\right]}+{b}^{\left[2\right]}$

${A}^{\left[2\right]}={g}^{\left[2\right]}\left({Z}^{\left[2\right]}\right)$

${Z}^{\left[3\right]}={W}^{\left[3\right]}{A}^{\left[2\right]}+{b}^{\left[3\right]}$

${A}^{\left[3\right]}={g}^{\left[3\right]}\left({Z}^{\left[3\right]}\right)$

${Z}^{\left[4\right]}={W}^{\left[4\right]}{A}^{\left[3\right]}+{b}^{\left[4\right]}$

${A}^{\left[4\right]}={g}^{\left[4\right]}\left({Z}^{\left[4\right]}\right)$

${Z}^{\left[l\right]}={W}^{\left[l\right]}{A}^{\left[l-1\right]}+{b}^{\left[l\right]}$

${A}^{\left[l\right]}={g}^{\left[l\right]}\left({Z}^{\left[l\right]}\right)$

### 3. Getting your matrix dimensions right

${z}^{\left[l\right]}$$z^{[l]}$${a}^{\left[l\right]}$$a^{[l]}$的维度是一样的，且$d{z}^{\left[l\right]}$$dz^{[l]}$$d{a}^{\left[l\right]}$$da^{[l]}$的维度均与${z}^{\left[l\right]}$$z^{[l]}$${a}^{\left[l\right]}$$a^{[l]}$的维度一致。

$d{Z}^{\left[l\right]}$$dZ^{[l]}$$d{A}^{\left[l\right]}$$dA^{[l]}$的维度分别与${Z}^{\left[l\right]}$$Z^{[l]}$${A}^{\left[l\right]}$$A^{[l]}$的相同。

### 4. Why deep representations?

$y={x}_{1}\oplus {x}_{2}\oplus {x}_{3}\oplus \cdots \oplus {x}_{n}$

$1+2+\cdots +{2}^{lo{g}_{2}\left(n\right)-1}=1\cdot \frac{1-{2}^{lo{g}_{2}\left(n\right)}}{1-2}={2}^{lo{g}_{2}\left(n\right)}-1=n-1$

### 6. Forward and Backward Propagation

${z}^{\left[l\right]}={W}^{\left[l\right]}{a}^{\left[l-1\right]}+{b}^{\left[l\right]}$

${a}^{\left[l\right]}={g}^{\left[l\right]}\left({z}^{\left[l\right]}\right)$

m个训练样本，向量化形式为：

${Z}^{\left[l\right]}={W}^{\left[l\right]}{A}^{\left[l-1\right]}+{b}^{\left[l\right]}$

${A}^{\left[l\right]}={g}^{\left[l\right]}\left({Z}^{\left[l\right]}\right)$

$d{z}^{\left[l\right]}=d{a}^{\left[l\right]}\ast {g}^{\left[l{\right]}^{\prime }}\left({z}^{\left[l\right]}\right)$

$d{W}^{\left[l\right]}=d{z}^{\left[l\right]}\cdot {a}^{\left[l-1\right]}$

$d{b}^{\left[l\right]}=d{z}^{\left[l\right]}$

$d{a}^{\left[l-1\right]}={W}^{\left[l\right]T}\cdot d{z}^{\left[l\right]}$

$d{z}^{\left[l\right]}={W}^{\left[l+1\right]T}\cdot d{z}^{\left[l+1\right]}\ast {g}^{\left[l{\right]}^{\prime }}\left({z}^{\left[l\right]}\right)$

m个训练样本，向量化形式为：

$d{Z}^{\left[l\right]}=d{A}^{\left[l\right]}\ast {g}^{\left[l{\right]}^{\prime }}\left({Z}^{\left[l\right]}\right)$

$d{W}^{\left[l\right]}=\frac{1}{m}d{Z}^{\left[l\right]}\cdot {A}^{\left[l-1\right]T}$

$d{b}^{\left[l\right]}=\frac{1}{m}np.sum\left(d{Z}^{\left[l\right]},axis=1,keepdim=True\right)$

$d{A}^{\left[l-1\right]}={W}^{\left[l\right]T}\cdot d{Z}^{\left[l\right]}$

$d{Z}^{\left[l\right]}={W}^{\left[l+1\right]T}\cdot d{Z}^{\left[l+1\right]}\ast {g}^{\left[l{\right]}^{\prime }}\left({Z}^{\left[l\right]}\right)$

• 广告
• 抄袭
• 版权
• 政治
• 色情
• 无意义
• 其他

120