Convolutional Neural Networks

Padding

Output Dimension

$n + 2 p - f + 1$

Padding Types

Valid: $p = 0$
Same: $n + 2 p - f + 1 = n \Rightarrow p = \dfrac {f - 1} {2}$

Stride

Output Dimension

$\left \lfloor \dfrac {n + 2p - f} {s} \right \rfloor + 1$

Convolutions Over Volume

$n \times n \times {n_{c}}_{prev}, f \times f \times {n_{c}}_{prev} \Rightarrow \left ( \left \lfloor \dfrac {n + 2p - f} {s} \right \rfloor + 1 \right ) \times \left ( \left \lfloor \dfrac {n + 2p - f} {s} \right \rfloor + 1 \right ) \times n_{c}$

Pooling

Output Dimension

$\left \lfloor \dfrac {n - f} {s} \right \rfloor + 1$

One Layer of a Convolution Network

Size Symbol	Meaning
$f ^{[l]}$	Filter Size
$p ^{[l]}$	Padding
$s ^{[l]}$	Stride
$n _{h} ^{[l - 1]} \times n _{w} ^{[l - 1]} \times n _{c} ^{[l - 1]}$	Input
$n _{h} ^{[l]} \times n _{w} ^{[l]} \times n _{c} ^{[l]}$	Output
$n _{c} ^{[l]}$	Number of filters
$f ^{[l]} \times f ^{[l]} \times n _{c} ^{[l - 1]}$	Each filter size
$a ^{[l]} \rightarrow n _{h} ^{[l]} \times n _{w} ^{[l]} \times n _{c} ^{[l]}$ $A ^{[l]} \rightarrow m \times n _{h} ^{[l]} \times n _{w} ^{[l]} \times n _{c} ^{[l]}$	Activations
$f ^{[l]} \times f ^{[l]} \times n _{c} ^{[l - 1]} \times n _{c} ^{[l]}$	Weights
$n _{c} ^{[l]}$	Bias

$n _{h} ^{[l]} = \left \lfloor \dfrac {n _{h} ^{[l - 1]} + 2 p ^{[l]} - f ^{[l]} } {s ^{[l]} } \right \rfloor + 1$
$n _{w} ^{[l]} = \left \lfloor \dfrac {n _{w} ^{[l - 1]} + 2 p ^{[l]} - f ^{[l]} } {s ^{[l]} } \right \rfloor + 1$

Types of Layers in a Convolutional Network

Type	Abbr.
Convolution	CONV
Pooling	POOL
Fully Connected	FC

Why Convolutions?

1. Parameter Sharing: A feature detector that’s useful in one part of the image is probably useful in aother part of the image.
2. Sparsity of Connections: In each layer, each output value depends only on a small number of inputs.