caffe教程笔记《Solver》

Solver可理解为模型舵手。因为它控制着模型的训练和测试阶段，并规定了学习的方法和学习的参数。
The solver orchestrates （安排）model optimization by coordinating（协调） the network’s forward inference and backward gradients to form（产生） parameter updates that attempt to improve the loss.
The Solver optimizes a model by：
（1）calls network forward to compute the output and loss（前向）
（2）calls network backward to compute the gradients（反向）
（3）incorporates the gradients into parameter updates according to the solver method （权重更新）
（4）updates the solver state according to learning rate, history, and method（分离解耦）

The Caffe solvers mathod are:

Stochastic Gradient Descent (type: "SGD"),#随机梯度下降
AdaDelta (type: "AdaDelta"),
Adaptive Gradient (type: "AdaGrad"),
Adam (type: "Adam"),
Nesterov’s Accelerated Gradient (type: "Nesterov") and
RMSprop (type: "RMSProp")

The solver 作用
1. 参数字典
2. 迭代优化参数
3. 周期性的得到测试结果
4. snapshots the model and solver state throughout the optimization（？）

Snapshotting and Resuming
The weight snapshots export the learned model while the solver snapshots allow training to be resumed from a given point.

Solver::Snapshot()和Solver::SnapshotSolverState()保存指定迭代次数中的参数并停止，相当于断点，由Solver::Restore()和Solver::RestoreSolverState()继续运行。
参见博文http://blog.csdn.net/u013508213/article/details/53208434#solver