经典模型之Lenet
模型背景
1985年,Rumelhart和Hinton等人提出了后向传播(BackPropagation,BP)算法[1](也有说1986年的,指的是他们另一篇paper:Learningrepresentations by back-propagating errors),使得神经网络的训练变得简单可行,这篇文章在GoogleScholar上的引用次数达到了19000多次,目前还是比Cortes和Vapnic的Support-Vector Networks稍落后一点,不过以Deep Learning最近的发展劲头来看,超越指日可待。
几年后,LeCun利用BP算法来训练多层神经网络用于识别手写邮政编码,这个工作就是CNN的开山之作,如图2所示,多处用到了5*5的卷积核,但在这篇文章中LeCun只是说把5*5的相邻区域作为感受野,并未提及卷积或卷积神经网络。
容易看出,Lenet网络结构由1个数据层、3个卷积层、2个池化层、2个全连接层和、1个SoftmaxLoss层和1个输出层组成。
solver.prototxt
# The train/test net protocol buffer definition
net: "examples/mnist/lenet_train_test.prototxt"
# test_iter specifies how many forward passes the test should carry out.
# In the case of MNIST, we have test batch size 100 and 100 test iterations,
# covering the full 10,000 testing images.
test_iter: 100
# Carry out testing every 500 training iterations.
test_interval: 500
# The base learning rate, momentum and the weight decay of the network.
base_lr: 0.01
momentum: 0.9
weight_decay: 0.0005
# The learning rate policy
lr_policy: "inv"
gamma: 0.0001
power: 0.75
# Display every 100 iterations
display: 100
# The maximum number of iterations
max_iter: 10000
# snapshot intermediate results
snapshot: 5000
snapshot_prefix: "examples/mnist/lenet"
# solver mode: CPU or GPU
solver_mode: CPU
~
参数说明
- net
训练/测试网络结构
- test_iter
表示测试的次数;比如,你的test阶段的batchsize=100,而你的测试数据为10000张图片,则你的测试次数为10000/100=100次;即,你的test_iter=100;