efficientnet_EfficientNet：神经网络扩展的新方法

efficientnet

Recently I came across a paper EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. They propose some amazing approach for scaling (increasing depth, width and resolution) neural network models. Here I will discuss the approach and try to build the intuition behind this approach. For mathematical and deeper insight, I recommend reading the original paper which is linked below.

最近，我遇到了一篇论文《 EfficientNet：对卷积神经网络的模型缩放的重新思考》 。 他们提出了一些用于扩展(增加深度，宽度和分辨率)神经网络模型的惊人方法。 在这里，我将讨论该方法，并尝试建立该方法背后的直觉。 为了获得数学知识和更深入的了解，我建议阅读下面链接的原始论文。

You may have heard of EfficientNet as an image classification algorithm. Then why am I referring this as an approach, not an algorithm, which one is it? Let's find out!

您可能听说过EfficientNet作为图像分类算法。 那我为什么要把它当作一种方法而不是一种算法呢？ 让我们找出答案！

First things first. Why EfficientNet? EfficientNet models (or approach) has gained the new state of the art accuracy for 5 out of the 8 datasets, with 9.6 times fewer parameters on average.

首先是第一件事。 为什么选择EfficientNet？ EfficientNet模型(或方法)在8个数据集中的5个数据集中获得了最新的精度，平均参数减少了9.6倍。

At the paper, the authors firstly find out the relationship between the accuracy and the scaling(size) of a model.It is found that , performance increasing along with increase of width(number of channels used in each layer), depth(how many layers in the model) and resolution(size of the input image).

在本文中，作者首先发现了模型精度与缩放比例(大小)之间的关系，发现随着性能的增加，宽度(每层使用的通道数)，深度(多少模型中的图层)和分辨率(输入图像的大小)。

We can see, at every case, initially the accuracy increases dramatically (of course along with the computational cost), but after sometime the curves almost flatten.

我们可以看到，在每种情况下，精度最初都会急剧提高(当然也伴随着计算成本)，但是一段时间后曲线几乎趋于平坦。

At this case, where depth is increased, everything else(width, resolution) is kept fixed. But the performance doesn’t saturate so easily, if we increase all of them simultaneously and that’s the first key observation in that paper.

在这种情况下，深度增加时，其他所有内容(宽度，分辨率)均保持不变。 但是，如果我们同时提高所有性能，那么性能就不会轻易达到饱和，这是该论文中的第一个关键观察。

Intuitively also, this makes a great sense. Because, for higher resolution image we can extract more features with a deeper neural network. Similarly, with increased width of the model, more fine grained patterns can be captured from a higher number of pixels. From this we can sense that, co-ordination and balance between different scaling dimensions can help us achieve greater performance.

从直觉上讲，这也很有意义。 因为，对于更高分辨率的图像，我们可以使用更深的神经网络提取更多特征。 类似地，随着模型宽度的增加，可以从更多数量的像素中捕获更多细粒度的图案。 由此我们可以感觉到，不同缩放比例尺寸之间的协调和平衡可以帮助我们实现更高的性能。

Generally this balance is tried to achieve by arbitrary change and manual tuning. But here we will take a more methodological approach. And this approach, compound scaling method is the second key observation of this paper.

通常，尝试通过任意更改和手动调整来实现这种平衡。 但是这里我们将采用一种更加方法论的方法。 而这种方法， 复合缩放方法是本文的第二个重点观察。

Without getting into the mathematical details (for which I will recommend reading the original paper linked below), we can say that our authors have found a way , that if we increase the depth, width and height by a power of N, our computation cost will increase by a power of 2 to the power N. And that is the compound scaling method.

无需深入研究数学细节(我建议您阅读下面的原始文章)，可以说我们的作者找到了一种方法，即如果将深度，宽度和高度增加N的幂，我们的计算成本将会以2的幂增加到N。这就是复合缩放方法。

By applying this concept and some nitty-gritty math details, authors first scaled up some existing state of the art ConvNets, like MobileNets and ResNets, which showed great improvement over their authentic performance, as well as their manually scaled versions.

通过应用这个概念和一些实质性的数学细节，作者首先扩大了现有的一些最先进的ConvNets，例如MobileNets和ResNets，它们在真实性能以及手动缩放版本方面显示出了很大的改进。

Then ,they set out to a journey, for developing their own optimized model. By using some recently developed neural architecture search methods, at first a baseline model EfficientNet-B0 was created. Then it was scaled up by using previously discussed compound scaling method, developing the other versions, from EfficientNet-B1 to EfficientNet-B7. And that’s what did the magic!!

然后，他们开始了一段旅程，以开发自己的优化模型。 通过使用一些最近开发的神经体系结构搜索方法，首先创建了一个基线模型EfficientNet-B0。 然后使用先前讨论的复合缩放方法将其放大 ，开发其他版本，从EfficientNet-B1到EfficientNet-B7。 这就是魔术的作用！！

Here is their performance on the ImageNet dataset compared with models of similar Top-1/Top-5 accuracy(If you aren’t sure what the Top-1/Top-5 accuracy, just take the Top-1 accuracy as the regular household accuracy,besides a little Google/Youtube search doesn’t hurt).

这是与类似Top-1 / Top-5精度的模型相比，它们在ImageNet数据集上的性能(如果您不确定Top-1 / Top-5精度是多少，只需将Top-1精度作为普通家庭准确度，除了一点点Google / Youtube搜索不会受到伤害)。

And that’s even not the whole story, the real magic is on the rightmost column.If the term Flops is unknown,take that as number of total addition or multiplication operation needed for the process (again a little googling, you may search FLOPS vs FLOPs), actually it is the measure of computational power needed. And do you find something amazing??

甚至还不是全部，真正的魔力在最右边的栏中。如果术语“触发器”未知，则将其视为该过程所需的总加法或乘法运算的次数(再进行一次谷歌搜索，您可以搜索FLOPS与FLOPs )，实际上是所需计算能力的量度。 而且您发现了什么了不起的东西吗？

Yap! So do I. The dramatic difference between EfficientNet models and their comparable models.Thing how much computationally efficient these EfficientNet models really are. Are they really comparable in those groups??

没错 我也是如此。EfficientNet模型与其可比模型之间的巨大差异。这些EfficientNet模型的实际计算效率是多少。 他们在那些群体中真的具有可比性吗？

So, that’s almost the story of one one the most powerful and scalable image classification models in recent years. Although a lot is left, like use of EfficientNet in transfer learning, as well as inference Latency comparison with other models. But lets leave that for your reading the paper.

因此，这几乎是近年来最强大和可扩展的图像分类模型之一的故事。 尽管还有很多事情，例如在迁移学习中使用EfficientNet以及将推理延迟与其他模型进行比较。 但是，让我们将其留给您阅读。

Hope you get the main idea behind this amazing approach. Do you know , our EfficienNet has a detective brother, EfficientDet!! But that’s a story for another day.

希望您能得到此惊人方法背后的主要思想。 您知道吗，我们的EfficienNet有一个侦探兄弟EfficientDet！ 但这是另一回事了。

Thanks for reading till here.You can read the original paper at https://arxiv.org/pdf/1905.11946. All images are taking from it.

感谢您阅读本文。您可以在https://arxiv.org/pdf/1905.11946上阅读原始文章。 所有图像都从中获取。

This is actually my first blog as well as first paper review. So, if you have any suggestions for me, I’m always open to them. Stay safe!!!

这实际上是我的第一个博客，也是第一篇论文评论。 因此，如果您对我有任何建议，我将始终向他们开放。 注意安全！！！

翻译自: https://medium.com/swlh/efficientnet-a-new-approach-to-neural-network-scaling-eae825f21cad

efficientnet