Convolutional neural networks

Convolutional neural networks

@(深度学习)

a convolutional neural network can learn a neuron once and use it in many places, making it easier to learn the model and reducing error.
学习一次,使用多次

传统的fully-connected layer:

We care a lot about local properties of the data: What frequency of sounds are there around a given time? Are they increasing or decreasing? And so on.
我们更关系数据局部的性质

Convolutional layers are often interweaved with pooling layers. In particular, there is a kind of layer called a max-pooling layer that is extremely popular.

应用到2D:

针对一张图片而言,没有必要精确的指定每个像素的信息

理解卷积

cifar10

代码组织

• cifar10_input.py 读取数据
• cifar10.py 构建模型
• cifar10_train.py 训练
• cifar10_eval.py 评价和预测

To NLP

理解卷积：应用于矩阵的滑动窗口函数

sliding window也被称为kernel,filter or feature detector

Location Invariance and Compositionality.

把词矩阵当作是一个图，词和对应的词向量

All models are wrong, but some are useful.

CNN Hyperparameters

Narrow vs. Wide convolution

How would you apply the filter to the first element of a matrix that doesn’t have any neighboring elements to the top and left?

zero-padding(0-填充)，所有矩阵以外的元素都用0表示，也称为wide convolution
不使用zero-padding的称为narrow convolution

Stride Size

步幅大小,filter移动的大小

Pooling Layers

pooling layers 对输入进行二次采样

为什需要pooling？
- pooling提供了固定大小的输出矩阵，例如对1000个filters应用max pooling，会得到1000维的输出，与每个filters的大小无关。这样，用不同长度的句子进行输入
- pooling具有降维的作用

Channels

不同的通道，表示了对输入数据不同角度的观察，针对图片识别而言，有红，绿，蓝三个通道。针对自然语言而言，例如可以从不同的词向量角度

max-pooling always beat average pooling
regularization doesn’t seem to make a big different in the NLP tasks that were considered