使用tensorflow 2 0理解并实现resnet 50

重点 (Top highlight)

Our intuition may suggest that deeper neural networks should be able to catch more complex features and thus they can be used for representing more complex functions compared to the shallower ones. The question that should arise is — if learning a better network is equivalent to stacking more and more layers? What are the problems and benefits of this approach? These questions and some very important other concepts were discussed in the Deep Residual Learning for Image Recognition paper by K. He et al. in 2017. This architecture is known as ResNet and many important must-know concepts related to Deep Neural Network (DNN) were introduced in this paper and, these will all be addressed in this post including an implementation of 50 layer ResNet in TensorFlow 2.0. What you can expect to learn from this post —

我们的直觉可能表明，较深的神经网络应能够捕获更复杂的特征，因此与较浅的神经网络相比，它们可用于表示更复杂的功能。 应该出现的问题是–学习更好的网络是否等同于堆叠越来越多的层？ 这种方法有什么问题和好处？ 这些问题以及一些非常重要的其他概念在K. He等人的“ 深度残差图像识别学习”一文中进行了讨论。 该架构在2017年被称为ResNet，并且在本文中介绍了许多与深度神经网络(DNN)相关的重要必不可少的概念，所有这些都将在本文中解决，包括在TensorFlow 2.0中实现50层ResNet的实现。 您可以从这篇文章中学到什么—

1. Problem with Very Deep Neural Network.
   非常深的神经网络的问题。
2. Mathematical Intuition Behind ResNet.
   ResNet背后的数学直觉。
3. Residual Block and Skip Connection.
   残余阻滞和跳过连接。
4. Structuring ResNet and Importance of 1×1 Convolution.
   构建ResNet和1×1卷积的重要性。
5. Implement ResNet with TensorFlow.
   使用TensorFlow实施ResNet。

Let’s begin!

让我们开始！

降级问题： (Degradation Problem :)

The main motivation of the ResNet original work was to address the degradation problem in a deep network. Adding more layers to a sufficiently deep neural network would first see saturation in accuracy and then the accuracy degrades. He et.al. presented the following picture of train and test error with Cifar-10 data-set using vanilla net--

ResNet原始工作的主要动机是解决深度网络中的降级问题。 向足够深的神经网络添加更多层将首先看到精度饱和，然后精度下降。 他等人 展示了以下使用香草网的Cifar-10数据集的火车和测试错误的图片-

Fig. 1: Classification error with Cifar-10 data increases with increasing no. of layers for both training (left) and test data (right) in a plain DNN. Reference: [1]

图1：Cifar-10数据的分类误差随编号的增加而增加。 普通DNN中训练(左)和测试数据(右)的层数。 参考：[1]

As we can see the training (left) and test errors (right) for the deeper network (56 layer) are higher than the 20 layer network. More the depth and with increasing epochs, the error increases. At first, it appears that as the number of layers increase, the number of parameters increase, thus this is a problem of overfitting. But it is not, let’s understand.

如我们所见，较深层网络(56层)的训练(左)和测试错误(右)高于20层网络。 随着深度的增加和历时的增加，误差也会增加。 首先，似乎随着层数的增加，参数的数目也增加，因此这是过度拟合的问题。 但事实并非如此，让我们了解一下。

One way of thinking about the problem is to consider a sufficiently DNN that calculates a sufficiently strong set of features that is necessary for the task in hand (ex: Image classification). If we add one more layer of the network to this already very DNN, what will this additional layer do? If already the