吴恩达深度学习学习笔记——C4W1——卷积神经网络——作业1——卷积神经网络入门

这里主要梳理一下作业的主要内容和思路，完整作业文件可参考:

https://github.com/pandenghuang/Andrew-Ng-Deep-Learning-notes/tree/master/assignments/C4W1/Assignment

作业完整截图，参考本文结尾：作业完整截图。

作业目录：

 

Convolutional Neural Networks: Step by Step（卷积神经网络入门）

Welcome to Course 4's first assignment! In this assignment, you will implement convolutional (CONV) and pooling (POOL) layers in numpy, including both forward propagation and (optionally) backward propagation.

...

1 - Packages（依赖包）

Let's first import all the packages that you will need during this assignment.

• numpy is the fundamental package for scientific computing with Python.
• matplotlib is a library to plot graphs in Python.
• np.random.seed(1) is used to keep all the random function calls consistent. It will help us grade your work.

...

2 - Outline of the Assignment（作业提纲）

You will be implementing the building blocks of a convolutional neural network! Each function you will implement will have detailed instructions that will walk you through the steps needed:

• Convolution functions, including:
  • Zero Padding
  • Convolve window
  • Convolution forward
  • Convolution backward (optional)
• Pooling functions, including:
  • Pooling forward
  • Create mask
  • Distribute value
  • Pooling backward (optional)

This notebook will ask you to implement these functions from scratch in numpy. In the next notebook, you will use the TensorFlow equivalents of these functions to build the following model:

In this part, you will build every step of the convolution layer. You will first implement two helper functions: one for zero padding and the other for computing the convolution function itself.

3 - Convolutional Neural Networks（卷积神经网络）

Although programming frameworks make convolutions easy to use, they remain one of the hardest concepts to understand in Deep Learning. A convolution layer transforms an input volume into an output volume of different size, as shown below.

...

3.1 - Zero-Padding（填充）

Zero-padding adds zeros around the border of an image:

The main benefits of padding are the following:

• It allows you to use a CONV layer without necessarily shrinking the height and width of the volumes. This is important for building deeper networks, since otherwise the height/width would shrink as you go to deeper layers. An important special case is the "same" convolution, in which the height/width is exactly preserved after one layer.

• It helps us keep more of the information at the border of an image. Without padding, very few values at the next layer would be affected by pixels as the edges of an image.

...

3.2 - Single step of convolution（单层卷积）

In this part, implement a single step of convolution, in which you apply the filter to a single position of the input. This will be used to build a convolutional unit, which:

• Takes an input volume
• Applies a filter at every position of the input
• Outputs another volume (usually of different size)

 

In a computer vision application, each value in the matrix on the left corresponds to a single pixel value, and we convolve a 3x3 filter with the image by multiplying its values element-wise with the original matrix, then summing them up and adding a bias. In this first step of the exercise, you will implement a single step of convolution, corresponding to applying a filter to just one of the positions to get a single real-valued output.

Later in this notebook, you'll apply this function to multiple positions of the input to implement the full convolutional operation.

...

3.3 - Convolutional Neural Networks - Forward pass（卷积神经网络——前向传播）

In the forward pass, you will take many filters and convolve them on the input. Each 'convolution' gives you a 2D matrix output. You will then stack these outputs to get a 3D volume:

...

4 - Pooling layer（汇聚层）

The pooling (POOL) layer reduces the height and width of the input. It helps reduce computation, as well as helps make feature detectors more invariant to its position in the input. The two types of pooling layers are:

• Max-pooling layer: slides an (𝑓,𝑓) window over the input and stores the max value of the window in the output.

• Average-pooling layer: slides an (𝑓,𝑓) window over the input and stores the average value of the window in the output.

4.1 - Forward Pooling（前向池化）

Now, you are going to implement MAX-POOL and AVG-POOL, in the same function.

...

5 - Backpropagation in convolutional neural networks (OPTIONAL / UNGRADED)（卷积神经网络中的反向传播）

In modern deep learning frameworks, you only have to implement the forward pass, and the framework takes care of the backward pass, so most deep learning engineers don't need to bother with the details of the backward pass. The backward pass for convolutional networks is complicated. If you wish however, you can work through this optional portion of the notebook to get a sense of what backprop in a convolutional network looks like.

When in an earlier course you implemented a simple (fully connected) neural network, you used backpropagation to compute the derivatives with respect to the cost to update the parameters. Similarly, in convolutional neural networks you can to calculate the derivatives with respect to the cost in order to update the parameters. The backprop equations are not trivial and we did not derive them in lecture, but we briefly presented them below.

5.1 - Convolutional layer backward pass（卷积层后向传播）

Let's start by implementing the backward pass for a CONV layer.

5.1.1 - Computing dA:

5.1.2 - Computing dW:

5.1.3 - Computing db:
...

5.2 Pooling layer - backward pass（汇聚层反向传播）

Next, let's implement the backward pass for the pooling layer, starting with the MAX-POOL layer. Even though a pooling layer has no parameters for backprop to update, you still need to backpropagation the gradient through the pooling layer in order to compute gradients for layers that came before the pooling layer.

5.2.1 Max pooling - backward pass

Before jumping into the backpropagation of the pooling layer, you are going to build a helper function called create_mask_from_window() which does the following:

5.2.2 - Average pooling - backward pass

In max pooling, for each input window, all the "influence" on the output came from a single input value--the max. In average pooling, every element of the input window has equal influence on the output. So to implement backprop, you will now implement a helper function that reflects this.

...

5.2.3 Putting it together: Pooling backward（模型组装）

You now have everything you need to compute backward propagation on a pooling layer.

...

Congratulations !

Congratulation on completing this assignment. You now understand how convolutional neural networks work. You have implemented all the building blocks of a neural network. In the next assignment you will implement a ConvNet using TensorFlow.

 

 

作业完整截图：