How to calculate the number of parameters of Convolutional Neural Networks(CNNs) correctly?

原创 2016年08月29日 21:48:29


I can't give the correct number of parameters of AlexNet or VGG Net.

For example, to calculate the number of parameters of a conv3-256 layer of VGG Net, the answer is 0.59M = (3*3)*(256*256), that is (kernel size) * (product of both number of channels in the joint layers), however in that way, I can't get the 138M parameters.

So could you please show me where is wrong with my calculation, or show me the right calculation procedure?


If you refer to VGG Net with 16-layer (table 1, column D) then 138M refers to the total number of parameters of this network, i.e including all convolutional layers, but also the fully connected ones.

Looking at the 3rd convolutional stage composed of 3 x conv3-256 layers:

  • the first one has N=128 input planes and F=256 output planes,
  • the two other ones have N=256 input planes and F=256 output planes.

The convolution kernel is 3x3 for each of these layers. In terms of parameters this gives:

  • 128x3x3x256 (weights) + 256 (biases) = 295,168 parameters for the 1st one,
  • 256x3x3x256 (weights) + 256 (biases) = 590,080 parameters for the two other ones.

As explained above you have to do that for all layers, but also the fully-connected ones, and sum these values to obtain the final 138M number.

-

UPDATE: the breakdown among layers give:

conv3-64  x 2       : 38,720
conv3-128 x 2       : 221,440
conv3-256 x 3       : 1,475,328
conv3-512 x 3       : 5,899,776
conv3-512 x 3       : 7,079,424
fc1                 : 102,764,544
fc2                 : 16,781,312
fc3                 : 4,097,000
TOTAL               : 138,357,544

In particular for the fully-connected layers (fc):

 fc1 (x): (512x7x7)x4,096 (weights) + 4,096 (biases)
 fc2    : 4,096x4,096     (weights) + 4,096 (biases)
 fc3    : 4,096x1,000     (weights) + 1,000 (biases)

(x) see section 3.2 of the article: the fully-connected layers are first converted to convolutional layers (the first FC layer to a 7 × 7 conv. layer, the last two FC layers to 1 × 1 conv. layers).

Details about fc1

As precised above the spatial resolution right before feeding the fully-connected layers is 7x7 pixels. This is because this VGG Net uses spatial padding before convolutions, as detailed within section 2.1 of the paper:

[...] the spatial padding of conv. layer input is such that the spatial resolution is preserved after convolution, i.e. the padding is 1 pixel for 3×3 conv. layers.

With such a padding, and working with a 224x224 pixels input image, the resolution decreases as follow along the layers: 112x112, 56x56, 28x28, 14x14 and 7x7 after the last convolution/pooling stage which has 512 feature maps.

This gives a feature vector passed to fc1 with dimension: 512x7x7.


论文笔记 Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus

最近读了几篇关于DRIVE数据集的文章,来完成斯坦福公开课cs231n最后的Final Project,还有zju两门课的作业,哈哈哈,一举三得(^__^) ~~ 大概目标就是分割血管,如下图...

论文阅读:End-to-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for H

human pose estimation DPM End-to-End Learning of Deformable Mixture of Parts and Deep Convolutional ...

神经网络压缩(6):Exploring the Regularity of Sparse Structure in Convolutional Neural Networks

Exploring the Regularity of Sparse Structure in Convolutional Neural Networks方法介绍 目的: 探索稀疏性和预测精度之间的...

深度学习笔记 (一) 卷积神经网络基础 (Foundation of Convolutional Neural Networks)

原文: http://www.cnblogs.com/rgvb178/p/6017991.html 一、卷积 卷积神经网络(Convolutional Neural Netwo...

VGG Convolutional Neural Networks Practical(5)the theory of back-propagation

第2部分:反向传播和衍生 例如,这是如何查找卷积运算符:y = vl_nnconv(x,w,b) ; % forward mode (get output) p = randn(size(y), '...

能帮你更好理解分析深度卷积神经网络《Better Analysis of Deep Convolutional Neural Networks》 CNNVis

深度卷积神经网络(CNNs)在许多模式识别任务中取得了很大的性能突破, 然而高质量深度模型的发展依赖于大量的尝试,这是由于没有很好的理解深度模型是怎么工作的,在本文中,提出了一个可视化分析系统,CNN...

An Intuitive Explanation of Convolutional Neural Networks

原文地址 An Intuitive Explanation of Convolutional Neural Networks Posted on August 11, 2016...
  • zjdxwwx
  • zjdxwwx
  • 2016年12月28日 11:27
  • 264

深入理解卷积神经网络(卷积篇)(Convolutional Neural Networks, CNNs)

概述卷积神经网络,也称为卷积网络,简称CNN,是神经网络的一种。其可用于一维时间序列的处理,也可以用于二维(如图像)序列的处理。目前,卷积神经网络可以说是应用最为广泛、效果最为出众的方法之一。相比于普...

A Beginner's Guide To Understanding Convolutional Neural Networks - Part 1

Introduction                 Convolutional neural networks. Sounds like a weird combination of bi...

A Beginner's Guide To Understanding Convolutional Neural Networks Part 2

Introduction         本文将会更加详细地介绍卷积网络。声明:现在我意识到有些内容相当复杂,甚至需要用一整篇文章才能解释清楚。为了内容详尽的同时尽量保证文章精简,我会将有关内容的...
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:How to calculate the number of parameters of Convolutional Neural Networks(CNNs) correctly?
举报原因:
原因补充:

(最多只允许输入30个字)