Alexnet 论文 深度之眼学习笔记

图片引于它出网络参数及连接计算
Reducing overfitting
4.1 Data Augmentation

• image translations(平移）and horizontal reflections (水平翻转）
• extract randomly 224 * 224 patches(截图）from 256 * 256 images
  in this case, data size grow by 2048 (=2*(256 - 224) * (256 - 224) )倍
• perform PCA on the set of RGB, 并对主成分进行标准差为0.1的高斯扰动， increase noise and reduce error rate
  PCA(降维）详解
• at test time, extract five 224 * 224 patches (one from center and other four from corner) as well as horizontal reflection, hence 共提取10次
  4.2 Dropout
• dropout rate is 0.5

Hyperparameters

• 批量大小 mini batchsize = 128
• 权重衰减 weight decay =0.0005
• 学习率 learning rate = 0.01 衰减率为0.1
• 轮数 epoches = 90
• initialization of weights : zero-mean Gaussian distribustion with standard deviation 0.01 ( 均值为0 方差为1 的高斯分布)
• initialization of neuron biases is constant 1 in 2,4,5 convolutional layers , the remaining layers is constant 0

知识补充
steps 迭代次数 - 学习完所有数据所需要的次数
epoch 轮数 - 所有数据学习一遍为一个epoch

Feature of network

• Using ReLU
• Dropout
• max-pool layer
• LRN layer(but not widely use)
• some tircks about data augmentation