MobileNet V1

版权声明：本文为博主原创文章，未经博主允许不得转载

论文：MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

链接：https://arxiv.org/abs/1704.04861 

tensorflow官方链接：https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.py

2017CVPR的文章

---

论文详解：

Section 2 reviews prior work in building small models. Section 3 describes the MobileNet architecture and two hyper-parameters width multiplier and resolution multiplier to define smaller and more efficient MobileNets. Section 4 describes experiments on ImageNet as well a variety of different applications and use cases.

嗯嗯，就按照作者的顺序逐一介绍。

 

prior work：

这里就不详细说了，有时间再去读下那些文章。

There has been rising interest in building small and efficient neural networks in the recent literature,Many different approaches can be generally categorized into either compressing pretrained networks or training small networks directly.

A different approach for obtaining small networks is shrinking, factorizing or compressing pretrained networks.

 

Compression based on product quantization [36], hashing[2], and pruning, vector quantization and Huffman coding[5]

[36] J. Wu, C. Leng, Y. Wang, Q. Hu, and J. Cheng. Quantized convolutional neural networks for mobile devices. arXiv
preprint arXiv:1512.06473, 2015.

[2] W. Chen, J. T. Wilson, S. Tyree, K. Q. Weinberger, and Y. Chen. Compressing neural networks with the hashing
trick. CoRR, abs/1504.04788, 2015. 2

[5] S. Han, H. Mao, and W. J. Dally. Deep compression: Compressing deep neural network with pruning, trained quantization
and huffman coding. CoRR, abs/1510.00149, 2, 2015.

 

various factorizations have been proposed to speed up pretrained networks [14, 20].

[14] M. Jaderberg, A. Vedaldi, and A. Zisserman. Speeding up convolutional neural networks with low rank expansions.
arXiv preprint arXiv:1405.3866, 2014.

[20] V. Lebedev, Y. Ganin, M. Rakhuba, I. Oseledets, and V. Lempitsky. Speeding-up convolutional neural networks
using fine-tuned cp-decomposition. arXiv preprint arXiv:1412.6553, 2014.

 

Another method for training small networks is distillation [9] which uses a larger network to teach a smaller network

[9] G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.

 

Another emerging approach is low bit networks[4, 22, 11].

[4] M. Courbariaux, J.-P. David, and Y. Bengio. Training deep neural networks with low precision multiplications. arXiv
preprint arXiv:1412.7024, 2014

[22]M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi. Xnornet: Imagenet classification using binary convolutional neural
networks. arXiv preprint arXiv:1603.05279, 2016.

[11] I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio. Quantized neural networks: Training neural networks
with low precision weights and activations. arXiv preprint arXiv:1609.07061, 2016.

---

MobileNet Architecture：

这里作者先探到了MobileNet模型是基于深度可分卷积，简单来说就是把一个标准卷积因式分解成一个depthwise convolution和一个1*1的 pointwise convolution .depthwise convolution  applies a single filter to each input channel(这里我不知道怎么表示，英文更直白点。。).pointwise convolution将上一层卷积的结果合并。如下图：

标准卷积的计算量：DK*DK*M*N*DF*DF

标准卷积的参数量：DK*DK*M*N

深度可分卷积的计算量：DK*DK*M*DF*DF+M*N*DF*DF

深度可分卷积的参数量：DK*DK*M+M*N

 

如果卷积核大小为3*3，那么计算量大概是原来的1/9.

 

接着作者谈到了MobileNet structure，然后说了标准卷积和因式分解后卷积的差别以及计算量和参量在各类型层的比例。

 

 

再接着作者为了得到更小更快的模型，作者引入两个超参数，Width Multiplier和Resolution Multiplier。

 

最后作者展示了几个结果图：

Tab4深度可分卷积和标准的卷积对比，可以看出Accuracy减少得并不明显，但却大大减少了计算量和参量。

Tab5减少网络的宽度和减少网络的深度对比，可以看出减少宽度更有好些。

Tab6 Tab7表示了两个超差数对实验结果的影响，可以发现MobileNets可以适用多个不同的应用。

---

总结：

MobileNets思路比较简单，就是利用depthwise separable convolution 来减少计算量，空间和通道卷积独立，减少了filter的耦合性，有效利用计算力。（Xception已经证明过了）

引入了Width multiplier α和resolution multiplier ρ，应用于不同的场景。