an empirical study of learning rates in deep neural networks for speech recognition 总结

结论源自：google 论文《AN EMPIRICAL STUDY OF LEARNING RATES IN DEEP NEURAL NETWORKS FORSPEECH RECOGNITION 》

该基于与现有所有 learning rate schedulding schemes 对比提出 “AdaDec” 

1. 目前，最有效的 随机梯度下降算法是：minibatch stochastic gradient descent 

2. 论文中提出的 “AdaDec” 是 learning rate schedulding schemes 中最优的。