deeplearning course-01-03 Shallow Neural Network

deeplearning course-01-03 Shallow Neural Network

@(学习笔记)

---

本笔记为Andrew NG的DeepLearning课程，其链接为：Coursera链接 网易云课堂链接
个人建议用coursera的比较好，虽然付费了，但是有题目可以做

PS:如果有错误，欢迎指正！

---

一. 网络表示方式：

$$z^{[1]}_{1} = w^{[1]}_{1}a^{[1]}_1 + b^{[1]}_1$$
其中，右上角的 1 代表网络层数，下标1代表该层第1个神经元

 一个神经元，其中的圆形代表了两次计算 向量化： 横向为训练数据序号，纵向为神经元序号


二. 激活函数

sigmoid function：二分类的时候用，输出是(0~,1）

$$sigmoid(x) = \dfrac{1}{1+e^{-x}}$$  tanh function: $$tanh(x) = \dfrac{e^z-e^{-z}}{e^z+e^{-z}}$$

Relu function: $$relu(x) = \begin{cases} 0 & x<0\\ x & x >0 \end{cases}$$

单层神经网络公式：

Logistic regression did not work well on the “flower dataset”. You are going to train a Neural Network with a single hidden layer.
Mathematically:

For one example $x^{(i)}$:

$$z^{[1] (i)} = W^{[1]} x^{(i)} + b^{[1]}\tag{1}$$
$$a^{[1] (i)} = \tanh(z^{[1] (i)})\tag{2}$$
$$z^{[2] (i)} = W^{[2]} a^{[1] (i)} + b^{[2]}\tag{3}$$
$$\hat{y}^{(i)} = a^{[2] (i)} = \sigma(z^{ [2] (i)})\tag{4}$$
$$y^{(i)}_{prediction} = \begin{cases} 1 & \mbox{if } a^{[2](i)} > 0.5 \\ 0 & \mbox{otherwise } \end{cases}\tag{5}$$

Given the predictions on all the examples, you can also compute the cost $J$ as follows:

$$J = - \frac{1}{m} \sum\limits_{i = 0}^{m} \large\left(\small y^{(i)}\log\left(a^{[2] (i)}\right) + (1-y^{(i)})\log\left(1- a^{[2] (i)}\right) \large \right) \small \tag{6}$$

注意事项：

1. 保持维度

db1 =  np.sum(b1, axis = 0, keepdims = 1)

之前少了，keepdims项，计算导致了错误

2. 维度计算公式

$$W: (n^{[l]},n^{l-1})$$
$$b: (n^{[l]},1)$$