深度学习笔记第二周：深度卷积神经网络

Classic Networks：

LeNet（Handwritten digits recognition），AlexNet,  VGGNet

Residual Network：

In order to get better performance of the deep network, we inject residual blocks into the plain network.

As we all know, the deeper the network, the more accurate the predictions. But also it's getting harder to train with the depth increasing and errors can be higher, too. Thus we introduce ResNet to fix the problem.

As for the structure of the ResNet, it is simply add some shortcuts from one output a[l] to a deeper layer. And then add the a[l] to the linear activation output of this layer. Compute the non-linear activation(such as Relu) of the former result.

One by One Convolution: shrink the number of channels

Using 1x1 convolution will not change the height and the width of input data, but will change the channel of the input image. So if you want to reduce the computation cost, you can try 1x1 convolution by changing the channels.

Inception Network:

If you are not sure about the size of the filters or whether to use a pooling later, you can just concat the channels to form a multiple block, which is called an inception module. 

Transfer learning

 

Data Augmentation:

Common Augmentation Method:

A. Mirroring Opration: By flipping an image horizontally.

B. Random Cropping(随机裁剪）

C. Color Shifting: Distort the color channels by changing values of RGB channels.