深度学习 吴恩达 01. 神经网络和深度学习 神经网络基础 作业

1.3 Reshaping arrays

• X.shape: 用来获取matrix/vector 的shape(dimention)
• X.reshape(): 用来reshape X 到其他维度

def image2vector(image):
    """
    Argument:
    image -- a numpy array of shape (length, height, depth)

    Returns:
    v -- a vector of shape (length*height*depth, 1)
    """

    ### START CODE HERE ### (≈ 1 line of code)
    v = image.reshape((image.shape[0] * image.shape[1] * image.shape[2], 1))
    ### END CODE HERE ###

    return v

1.4 Normalizing rows

• normalize 后一般效果较好，因为在normalization后梯度下降收敛的更快了

# -*- encoding: utf-8 -*-
import numpy as np
def normalize_rows(x):
    x_norm = np.linalg.norm(x, axis=1, keepdims=True)   # 求二范数
    return x / x_norm

if __name__ == '__main__':
    x = np.array(
        [
            [0, 3, 4],
            [1, 6, 4]
        ]
    )
    print(normalize_rows(x))

1.5 Broadcasting and the softmax function

Exercise: Implement a softmax function using numpy. You can think of softmax as a normalizing function used when your algorithm needs to classify two or more classes. You will learn more about softmax in the second course of this specialization.

# -*- encoding: utf-8 -*-
import numpy as np
def my_softmax(x):
    x_exp = np.exp(x)
    x_sum = np.sum(x_exp, axis=1, keepdims=True)
    return x_exp / x_sum

if __name__ == '__main__':
    x = np.array(
        [
            [9,2,5,0,0],
            [7,5,0,0,0]
        ]
    )
    print(my_softmax(x))

2.1 Implement the L1 and L2 loss functions

Exercise: Implement the numpy vectorized version of the L1 loss. You may find the function abs(x) (absolute value of x) useful.

Exercise: Implement the numpy vectorized version of the L2 loss. There are several way of implementing the L2 loss but you may find the function np.dot() useful. As a reminder, if x=[x1,x2,…,xn]x=[x1,x2,…,xn], then np.dot(x,x) = ∑nj=0x2j∑j=0nxj2.

# -*- encoding: utf-8 -*-
import numpy as np
def L1(y_hat, y):
    return np.sum(np.abs(y_hat- y))

def L2(y_hat, y):
    return np.sum(np.power(y_hat-y, 2))
if __name__ == '__main__':
    y_hat = np.array(
        [0.9, 0.2, 0.1, 0.4, 0.9]
    )
    y = np.array(
        [1, 0, 0, 1, 1]
    )
    print(L1(y_hat, y))
    print(L2(y_hat, y))

Part 2： Logistic Regression with a Neural Network mindset

Exercise: Find the values for:

• m_train (number of training examples)
• m_test (number of test examples)
• num_px (= height = width of a training image)

Exercise: Reshape the training and test data sets so that images of size (num_px, num_px, 3) are flattened into single vectors of shape (num_px ∗∗ num_px ∗∗ 3, 1).

Key steps:
In this exercise, you will carry out the following steps:

• Initialize the parameters of the model
• Learn the parameters for the model by minimizing the cost
• Use the learned parameters to make predictions (on the test set)
• Analyse the results and conclude

The main steps for building a Neural Network are:

1. Define the model structure (such as number of input features)
2. Initialize the model’s parameters
3. Loop:

• Calculate current loss (forward propagation)
• Calculate current gradient (backward propagation)
• Update parameters (gradient descent)

# -*- encoding: utf-8 -*-
import numpy as np
import matplotlib.pyplot as plt
import h5py          # a common package to interact with a dataset that is stored on an H5 file
import scipy
from PIL import Image
from scipy import ndimage
from lr_utils import load_dataset

# Loading the data (cat/non-cat)
train_set_x_orig, train_set_y, test_set_x_orig, test_set_y, classes = load_dataset()
index = 25
plt.imshow(train_set_x_orig[index])
print ("y = " + str(train_set_y[:, index]) + ", it's a '" + classes[np.squeeze(train_set_y[:, index])].decode("utf-8") +  "' picture.")
print(np.squeeze(train_set_y[:, index]).shape)  # np.squeeze可以删去指定维度的条目，上面原来为[1],处理后变为 1
plt.show()

### START CODE HERE ### (≈ 3 lines of code)
m_train = train_set_x_orig.shape[0]
m_test = test_set_x_orig.shape[0]
num_px = train_set_x_orig.shape[1]
### END CODE HERE ###

print ("Number of training examples: m_train = " + str(m_train))                # 209
print ("Number of testing examples: m_test = " + str(m_test))                   # 50
print ("Height/Width of each image: num_px = " + str(num_px))                   # 64
print ("Each image is of size: (" + str(num_px) + ", " + str(num_px) + ", 3)")  # (64, 64, 3)
print ("train_set_x shape: " + str(train_set_x_orig.shape))                     # (209, 64, 64, 3)
print ("train_set_y shape: " + str(train_set_y.shape))                          # (1, 209)
print ("test_set_x shape: " + str(test_set_x_orig.shape))                       # (50, 64, 64, 3)
print ("test_set_y shape: " + str(test_set_y.shape))                            # (1, 50)


# Reshape the training and test examples

### START CODE HERE ### (≈ 2 lines of code)
train_set_x_flatten = train_set_x_orig.reshape(m_train, -1).T
test_set_x_flatten = test_set_x_orig.reshape(m_test, -1).T
### END CODE HERE ###

print ("train_set_x_flatten shape: " + str(train_set_x_flatten.shape))       # (12288, 209)
print ("train_set_y shape: " + str(train_set_y.shape))                       # (1, 209)
print ("test_set_x_flatten shape: " + str(test_set_x_flatten.shape))         # (12288, 50)
print ("test_set_y shape: " + str(test_set_y.shape))                         # (1, 50)
print ("sanity check after reshaping: " + str(train_set_x_flatten[0:5,0]))   # [17 31 56 22 33]

train_set_x = train_set_x_flatten/255.
test_set_x = test_set_x_flatten/255.


def sigmoid(z):
    return 1.0 / (1 + np.exp(-z))

def initialize_with_zeros(dim):
    w = np.zeros((dim, 1))
    b = 0
    return w, b

def propagate(w, b, X, Y):
    m = X.shape[1]
    A = sigmoid(np.dot(w.T, X)+b)
    cost = - (1.0 / m) * np.sum(Y * np.log(A) + (1 - Y) * np.log(1-A))
    dw = (1.0 / m) * np.dot(X, (A-Y).T)
    db = (1.0 / m) * np.sum(A - Y)
    assert(dw.shape == w.shape)
    assert(db.dtype == float)
    cost = np.squeeze(cost)
    assert(cost.shape == ())
    grads = {"dw": dw, "db":db}
    return grads, cost

def optimize(w, b, X, Y, num_iterations, learning_rate, print_cost=False):
    costs = []
    for i in range(num_iterations):
        grads, cost = propagate(w, b, X, Y)
        dw = grads["dw"]
        db = grads["db"]
        w -= learning_rate * dw
        b -= learning_rate * db
        if i % 100 == 0:
            costs.append(cost)
        if print_cost and i % 100 == 0:
            print("Cost after iteration %i: %f"%(i, cost))
    params = {"w": w, "b": b}
    grads = {"dw": dw, "db":db}
    return params, grads, costs


def predict(w, b, X):
    m = X.shape[1]
    Y_prediction = np.zeros((1, m))
    w = w.reshape(X.shape[0], 1)
    A = sigmoid(np.dot(w.T, X) + b)
    for i in range(A.shape[1]):
        if A[0, i] > 0.5:
            Y_prediction[0, i] = 1
        else:
            Y_prediction[0, i] = 0
    return Y_prediction


def model(X_train, Y_train, X_test, Y_test, num_iterations = 2000, learning_rate = 0.5, print_cost=False):
    w, b = initialize_with_zeros(X_train.shape[0])
    parameters, grads, costs = optimize(w, b, X_train, Y_train, num_iterations, learning_rate, print_cost)
    w = parameters["w"]
    b = parameters["b"]
    Y_prediction_test = predict(w, b, X_test)
    Y_prediction_train = predict(w, b, X_train)
    print("train accuracy: {} %".format(100 - np.mean(np.abs(Y_prediction_train - Y_train)) * 100))
    print("test accuracy: {} %".format(100 - np.mean(np.abs(Y_prediction_test - Y_test)) * 100))

    d = {"costs": costs,
         "Y_prediction_test": Y_prediction_test,
         "Y_prediction_train": Y_prediction_train,
         "w": w,
         "b": b,
         "learning_rate": learning_rate,
         "num_iterations": num_iterations}

    return d
if __name__ == '__main__':
    pass
    # print("sigmoid([0, 2]) = " + str(sigmoid(np.array([0, 2]))))   # [0.5        0.88079708]
    # dim = 2
    # w, b = initialize_with_zeros(dim)
    # print("w = " + str(w))
    # print("b = " + str(b))

    w, b, X, Y = np.array([[1.], [2.]]), 2., np.array([[1., 2., -1.], [3., 4., -3.2]]), np.array([[1, 0, 1]])
    # grads, cost = propagate(w, b, X, Y)
    # print("dw = " + str(grads["dw"]))   # [[0.99845601], [2.39507239]]
    # print("db = " + str(grads["db"]))   # 0.001455578136784208
    # print("cost = " + str(cost))        # 5.801545319394553

    # params, grads, costs = optimize(w, b, X, Y, num_iterations=100, learning_rate=0.009, print_cost=False)
    #
    # print("w = " + str(params["w"]))
    # print("b = " + str(params["b"]))
    # print("dw = " + str(grads["dw"]))
    # print("db = " + str(grads["db"]))
    # w = np.array([[0.1124579], [0.23106775]])
    # b = -0.3
    # X = np.array([[1., -1.1, -3.2], [1.2, 2., 0.1]])
    # print("predictions = " + str(predict(w, b, X)))
    d = model(train_set_x, train_set_y, test_set_x, test_set_y, num_iterations=2000, learning_rate=0.005,
              print_cost=True)

    costs = np.squeeze(d['costs'])
    plt.plot(costs)
    plt.ylabel('cost')
    plt.xlabel('iterations (per hundreds)')
    plt.title("Learning rate =" + str(d["learning_rate"]))
    plt.show()