Caffe下学习速率调整策略摘抄

摘抄自caffe.proto:
1. fixed: always return base_lr
2. step: return

$$base\_lr * gamma ^ {(floor(iter / step))}$$
3. exp: return $$base\_lr * gamma ^ {iter}$$
4. inv: return $$base\_lr * (1 + gamma * iter) ^ {(- power)}$$
5. multistep: similar to step but it allows non uniform steps defined by stepvalue
6. poly: the effective learning rate follows a polynomial decay, to be zero by the max_iter. return $$base\_lr* (1 - iter/max_iter) ^ {(power)}$$
7. sigmoid: the effective learning rate follows a sigmod decay. return $$base\_lr* ( 1/(1 + exp(-gamma * (iter - stepsize))))$$