caffe下学习速率调整策略

摘抄自caffe.proto:
1. fixed: always return base_lr
2. step: return

base_lr∗gamma(floor(iter/step))

3. exp: return

base_lr∗gammaiter

4. inv: return

base_lr∗(1+gamma∗iter)(−power)

5. multistep: similar to step but it allows non uniform steps defined by stepvalue
6. poly: the effective learning rate follows a polynomial decay, to be zero by the max_iter. return

base_lr∗(1−iter/maxiter)(power)

7. sigmoid: the effective learning rate follows a sigmod decay. return

base_lr∗(1/(1+exp(−gamma∗(iter−stepsize))))