Iteration 60000, Testing net (#0) Test net output #0: accuracy = 0.1 Test net output #1: loss = 2.30292 可见加了sigmoid层的方法在此模型中无效。
模型7:(换成ReLU层)
层类别
具体信息
conv1
output: 32, kernel: 5, stride: 1 pad: 2
pool1
pool: MAX, kernel: 3, stride: 2
relu1
Sigmoid
norm1
LRN
conv2
output: 16, kernel: 5, stride: 1 pad: 2
pool2
pool: MAX, kernel: 3, stride: 2
relu1
ReLU
ip1
200
ip2
100
ip3
10
实验结果:
Iteration 60000, loss = 0.620338
Iteration 60000, Testing net (#0) Test net output #0: accuracy = 0.6391 Test net output #1: loss = 1.05354
模型8:(全部换成ReLU层)
层类别
具体信息
conv1
output: 32, kernel: 5, stride: 1 pad: 2
pool1
pool: MAX, kernel: 3, stride: 2
relu1
ReLU
norm1
LRN
conv2
output: 16, kernel: 5, stride: 1 pad: 2
pool2
pool: MAX, kernel: 3, stride: 2
relu1
ReLU
ip1
200
ip2
100
ip3
10
实验结果:
Iteration 60000, loss = 0.416507
Iteration 60000, Testing net (#0) Test net output #0: accuracy = 0.6794 Test net output #1: loss = 1.15119
在两个卷积层后面加上ReLU层后,识别效果提升了较多,识别率为67.94%。
模型9:(加一个Dropout层)
层类别
具体信息
conv1
output: 32, kernel: 5, stride: 1 pad: 2
pool1
pool: MAX, kernel: 3, stride: 2
relu1
ReLU
norm1
LRN
conv2
output: 16, kernel: 5, stride: 1 pad: 2
pool2
pool: MAX, kernel: 3, stride: 2
relu1
ReLU
ip1
200
dropout
dropout, 0.5
ip2
100
ip3
10
实验结果:
Iteration 60000, loss = 0.563472
Iteration 60000, Testing net (#0) Test net output #0: accuracy = 0.6728 Test net output #1: loss = 1.03333 从实验结果可以知道,加了Dropout层之后,虽然没有提高识别效果,但是降低了过拟合。因此,下一步增加FC层的输出看看。
模型10:(增加FC层的输出个数)
层类别
具体信息
conv1
output: 32, kernel: 5, stride: 1 pad: 2
pool1
pool: MAX, kernel: 3, stride: 2
relu1
ReLU
norm1
LRN
conv2
output: 16, kernel: 5, stride: 1 pad: 2
pool2
pool: MAX, kernel: 3, stride: 2
relu1
ReLU
ip1
400
dropout
dropout, 0.5
ip2
150
ip3
10
实验结果:
Iteration 60000, loss = 0.446714
Iteration 60000, Testing net (#0) Test net output #0: accuracy = 0.6903 Test net output #1: loss = 0.990431
模型11:(再增加一个Dropout)
层类别
具体信息
conv1
output: 32, kernel: 5, stride: 1 pad: 2
pool1
pool: MAX, kernel: 3, stride: 2
relu1
ReLU
norm1
LRN
conv2
output: 16, kernel: 5, stride: 1 pad: 2
pool2
pool: MAX, kernel: 3, stride: 2
relu1
ReLU
ip1
400
dropout
dropout, 0.5
ip2
200
dropout
dropout, 0.5
ip3
10
实验结果:
Iteration 60000, loss = 0.586936
Iteration 60000, Testing net (#0) Test net output #0: accuracy = 0.7013 Test net output #1: loss = 0.92605
模型12:(调整卷积层的输出)
层类别
具体信息
conv1
output: 48, kernel: 5, stride: 1 pad: 2
pool1
pool: MAX, kernel: 3, stride: 2
relu1
ReLU
norm1
LRN
conv2
output: 32, kernel: 5, stride: 1 pad: 2
pool2
pool: MAX, kernel: 3, stride: 2
relu1
ReLU
ip1
400
dropout
dropout, 0.5
ip2
200
dropout
dropout, 0.5
ip3
10
实验结果:
Iteration 60000, loss = 0.273988
Iteration 60000, Testing net (#0) Test net output #0: accuracy = 0.7088 Test net output #1: loss = 1.1117