How to calculate the number of parameters of Convolutional Neural Networks(CNNs) correctly?

原创 2016年08月29日 21:48:29

I can't give the correct number of parameters of AlexNet or VGG Net.

For example, to calculate the number of parameters of a conv3-256 layer of VGG Net, the answer is 0.59M = (3*3)*(256*256), that is (kernel size) * (product of both number of channels in the joint layers), however in that way, I can't get the 138M parameters.

So could you please show me where is wrong with my calculation, or show me the right calculation procedure?

If you refer to VGG Net with 16-layer (table 1, column D) then 138M refers to the total number of parameters of this network, i.e including all convolutional layers, but also the fully connected ones.

Looking at the 3rd convolutional stage composed of 3 x conv3-256 layers:

  • the first one has N=128 input planes and F=256 output planes,
  • the two other ones have N=256 input planes and F=256 output planes.

The convolution kernel is 3x3 for each of these layers. In terms of parameters this gives:

  • 128x3x3x256 (weights) + 256 (biases) = 295,168 parameters for the 1st one,
  • 256x3x3x256 (weights) + 256 (biases) = 590,080 parameters for the two other ones.

As explained above you have to do that for all layers, but also the fully-connected ones, and sum these values to obtain the final 138M number.


UPDATE: the breakdown among layers give:

conv3-64  x 2       : 38,720
conv3-128 x 2       : 221,440
conv3-256 x 3       : 1,475,328
conv3-512 x 3       : 5,899,776
conv3-512 x 3       : 7,079,424
fc1                 : 102,764,544
fc2                 : 16,781,312
fc3                 : 4,097,000
TOTAL               : 138,357,544

In particular for the fully-connected layers (fc):

 fc1 (x): (512x7x7)x4,096 (weights) + 4,096 (biases)
 fc2    : 4,096x4,096     (weights) + 4,096 (biases)
 fc3    : 4,096x1,000     (weights) + 1,000 (biases)

(x) see section 3.2 of the article: the fully-connected layers are first converted to convolutional layers (the first FC layer to a 7 × 7 conv. layer, the last two FC layers to 1 × 1 conv. layers).

Details about fc1

As precised above the spatial resolution right before feeding the fully-connected layers is 7x7 pixels. This is because this VGG Net uses spatial padding before convolutions, as detailed within section 2.1 of the paper:

[...] the spatial padding of conv. layer input is such that the spatial resolution is preserved after convolution, i.e. the padding is 1 pixel for 3×3 conv. layers.

With such a padding, and working with a 224x224 pixels input image, the resolution decreases as follow along the layers: 112x112, 56x56, 28x28, 14x14 and 7x7 after the last convolution/pooling stage which has 512 feature maps.

This gives a feature vector passed to fc1 with dimension: 512x7x7.

Convolutional Neural Networks (CNNs / ConvNets) 翻译第二段

Architecture Overview Recall: Regular Neural Nets. As we saw in the previous chapter, Neura...
  • tz1215354983
  • tz1215354983
  • 2017年01月03日 10:15
  • 176

卷积神经网络, Convolutional Neural Networks , CNN

1,简介 CNN是deep learning在图像处理领域的一个应用 由具有可学习的权重和偏置常量(biases)的神经元组成,是deeplearning在图像处理的一个应用 2,卷积层(C...
  • qq_34562093
  • qq_34562093
  • 2018年01月20日 17:21
  • 83

论文笔记之Learning Convolutional Neural Networks for Graphs

本篇论文是2016ICML上的一篇论文,对于如何将cnn应用在graph上提供了一种新的思路。架构: 总体上讲,就是用w个固定size=(k+1)的子图来表示输入的graph,再将这w个子图正则...
  • BVL10101111
  • BVL10101111
  • 2016年12月06日 11:07
  • 2857

导读ICML2016 - Learning Convolutional Neural Networks for Graphs
  • mmc2015
  • mmc2015
  • 2016年06月20日 15:50
  • 1766


Caffe是深度学习的一种框架,由C++和Python编写,底层是C++源码。一、Caffe-master源代码大框架: 关键文件如下: - data:用于存放caffe-master中程序所需...
  • Lucifer_zzq
  • Lucifer_zzq
  • 2017年06月14日 22:50
  • 798

CNN(Convolutional Neural Networks)没有原理只有实现

零.说明:        本文的所有代码均可在 DML 找到,欢迎点星星。         注.CNN的这份代码非常慢,基本上没有实际使用的可能,所以我只是发出来,代表我还是实践过而已 一.引入...
  • Dark_Scope
  • Dark_Scope
  • 2013年12月03日 23:03
  • 20601

《Architecture of Convolutional Neural Networks (CNNs) demystified》,译名:卷积神经网络(CNNs)架构解密

  • jiangsgyx
  • jiangsgyx
  • 2018年01月06日 21:25
  • 122


原文链接: Papers Reading List. T...
  • dlfxjc2
  • dlfxjc2
  • 2018年01月26日 09:36
  • 376

SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks文章详细解读

SCNN: An Accelerator for Compressed-sparse Convolutional Neural NetworksAs FPGAs are intrinsically p...
  • fsFengQingYangheihei
  • fsFengQingYangheihei
  • 2018年01月18日 15:46
  • 257

Notes on Convolutional Neural Networks(阅读)

这篇文章是大家熟悉的CNN,这是被埋没了很久的一篇,是金子总会发光。 =================================================================...
  • langb2014
  • langb2014
  • 2015年09月15日 18:08
  • 4521
您举报文章:How to calculate the number of parameters of Convolutional Neural Networks(CNNs) correctly?