吴恩达.深度学习系列-C1神经网络与深度学习-w3-（作业：一个隐藏层进行二维数据分类）

前言

**注意：coursera要求不要在互联网公布自己的作业。如果你在学习这个课程，建议你进入课程系统自行完成作业。使用逻辑回归作为一个最简单的类似神经网络来进行图像判别。我觉得代码有参考和保留的意义。v
使用一个 2×4×1的网络来对数据进行二分类。**
比较麻烦的是什么时候用点乘，什么时候用矩阵乘法。见【4.3】小节的代码。虽然代码都提交通过审核，但自己还需要梳理一下。
I.Cost的计算
$J = - \frac{1}{m} \sum\limits_{i = 0}^{m} \large{(} \small y^{(i)}\log\left(a^{[2] (i)}\right) + (1-y^{(i)})\log\left(1- a^{[2] (i)}\right) \large{)} \small\tag{13}$
用点乘：

    logprobs = np.multiply(np.log(A2),Y)+np.multiply(np.log(1-A2),1-Y)
    cost = - np.sum(logprobs)/m 

II.gradient的计算

    # Backward propagation: calculate dW1, db1, dW2, db2. 
    ### START CODE HERE ### (≈ 6 lines of code, corresponding to 6 equations on slide above)
    #按单样本进行简化考量，参数的梯度的形应该与参数的形一致，如：dZ2.shape==Z2.shape=（1,1）;dW1.shape==W1=(4,2)
    dZ2 = A2-Y  # A2(1,1)-Y(1,1)=dZ2(1,1)
    dW2 = np.dot(dZ2,A1.T)/m   #dZ2(1,1)*A1.T(1,4)=dW2(1,4),==>使用矩阵乘!
    db2 = np.sum(dZ2,axis=1,keepdims=True)/m   #np.sum(dW2(1,4))/m=db2(1,1)
    dZ1 = np.multiply(np.multiply(W2.T,dZ2),1 - np.power(A1, 2))  #输出dZ1=(4,1)。W2.T(4,1)×dZ2(1,1)  [1 - np.power(A1, 2)],所以内外两层的乘号都是点乘。
    dW1 = np.dot(dZ1,X.T)/m   #dZ1(4,1)*X.T(1,2)=dW1(4,2)===>使用矩阵乘
    db1 = np.sum(dZ1,axis=1,keepdims=True)/m    
    ### END CODE HERE ###

是否使用点乘还是矩阵乘，有一个判别方法，把输入的shape和应该输出的shape提前标出来，那么比较容易判别用点乘还是矩阵乘。输入的形与输出一致，那么肯定是点乘，其他情况是用矩阵乘。如果标出的形还不好判别，可以进一步画出是行还是列用来表达单个样本中的n个特征。或者只按一条样本的shape来进行判断。如上。

Planar data classification with one hidden layer

Welcome to your week 3 programming assignment. It’s time to build your first neural network, which will have a hidden layer. You will see a big difference between this model and the one you implemented using logistic regression.

You will learn how to:
- Implement a 2-class classification neural network with a single hidden layer
- Use units with a non-linear activation function, such as tanh
- Compute the cross entropy loss
- Implement forward and backward propagation

1 - Packages

Let’s first import all the packages that you will need during this assignment.
- numpy is the fundamental package for scientific computing with Python.
- sklearn provides simple and efficient tools for data mining and data analysis.
- matplotlib is a library for plotting graphs in Python.
- testCases provides some test examples to assess the correctness of your functions
- planar_utils provide various useful functions used in this assignment

# Package imports
import numpy as np
import matplotlib.pyplot as plt
from testCases_v2 import *
import sklearn
import sklearn.datasets
import sklearn.linear_model
from planar_utils import plot_decision_boundary, sigmoid, load_planar_dataset, load_extra_datasets

%matplotlib inline

np.random.seed(1) # set a seed so that the results are consistent

/opt/conda/lib/python3.5/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
/opt/conda/lib/python3.5/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')

2 - Dataset

First, let’s get the dataset you will work on. The following code will load a “flower” 2-class dataset into variables X and Y.

X, Y = load_planar_dataset()

Visualize the dataset using matplotlib. The data looks like a “flower” with some red (label y=0) and some blue (y=1) points. Your goal is to build a model to fit this data.

# Visualize the data:
plt.scatter(X[0, :], X[1, :], c=Y, s=40, cmap=plt.cm.Spectral);

You have:
- a numpy-array (matrix) X that contains your features (x1, x2)
- a numpy-array (vector) Y that contains your labels (red:0, blue:1).

Lets first get a better sense of what our data is like.

Exercise: How many training examples do you have? In addition, what is the shape of the variables X and Y?

Hint: How do you get the shape of a numpy array? (help)

### START CODE HERE ### (≈ 3 lines of code)
shape_X = X.shape
shape_Y = Y.shape
m = shape_X[1]  # training set size
### END CODE HERE ###

print ('The shape of X is: ' + str(shape_X))
print ('The shape of Y is: ' + str(shape_Y))
print ('I have m = %d training examples!' % (m))

The shape of X is: (2, 400)
The shape of Y is: (1, 400)
I have m = 400 training examples!

Expected Output:

**shape of X**	(2, 400)
**shape of Y**	(1, 400)
**m**	400

3 - Simple Logistic Regression

Before building a full neural network, lets first see how logistic regression performs on this problem. You can use sklearn’s built-in functions to do that. Run the code below to train a logistic regression classifier on the dataset.

# Train the logistic regression classifier
clf = sklearn.linear_model.LogisticRegressionCV();
clf.fit(X.T, Y.T);

/opt/conda/lib/python3.5/site-packages/sklearn/utils/validation.py:515: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)

You can now plot the decision boundary of these models. Run the code below.

# Plot the decision boundary for logistic regression
plot_decision_boundary(lambda x: clf.predict(x), X, Y)
plt.title("Logistic Regression")

# Print accuracy
LR_predictions = clf.predict(X.T)
print ('Accuracy of logistic regression: %d ' % float((np.dot(Y,LR_predictions) + np.dot(1-Y,1-LR_predictions))/float(Y.size)*100) +
       '% ' + "(percentage of correctly labelled datapoints)")

Accuracy of logistic regression: 47 % (percentage of correctly labelled datapoints)

Expected Output:

**Accuracy**

47%

Interpretation: The dataset is not linearly separable, so logistic regression doesn’t perform well. Hopefully a neural network will do better. Let’s try this now!

4 - Neural Network model

Logistic regression did not work well on the “flower dataset”. You are going to train a Neural Network with a single hidden layer.

Here is our model:

Mathematically:

For one example $x^{(i)}$:

$$z^{[1] (i)} = W^{[1]} x^{(i)} + b^{[1]}\tag{1}$$