qq_34738572的博客

2 PRELIMINARIESPrimitives: L(θ) loss函数； L^(θ)近似器； 函数L的梯度；g; a; s; v; w存储缓冲区，都可以用下标t表示；g动量缓冲；Optimizationalgorithms：词：1.perform a sequence of steps 执行一系列步骤 2.rather than as fullpseudocode 而不是完整的伪代码总体更新规则：SGD: Momentum：，当时，即为SGD优化算法。控制...

ABSTRACT:词：1.encountered 遇到 2.exploits 利用 3.approximately 大约 4.intrinsically 固有的短语：1. in an intrinsically deterministic manner 以一种固有确定方式1 INTRODUCTION词：1. mission impossible 不可能完成的任务 2.milestones 里程碑 3.first-order 一阶 4.saddle points 鞍点 5.sharp...

题目：1.EXPONENTIAL 指数ABSTRACT:词：1.Intriguing 有趣的 2.exotic 异乎寻常的 3.ubiquitous 无处不在 4.blows up 炸毁短语：1.To the best of our knowledge 据我们所知 2.it is equivalent to 。。。。相当于 1INTRODUCTION词：1.rigorous 严格 2.trajectory轨迹3.invariance 不变性短语：1. it is fall...

ADASHIFT: DECORRELATION AND CONVERGENCE OF ADAPTIVE LEARNING RATE METHODS1INTRODUCTION词：1.respect to 关于 2.exponential 指数 3.alleviate 减轻 4.oscillation 震荡 5.counterexamples 反例 6.violation 违反7.subsequently 随后短语：1.the key issue。。。。。 lies in the 关键问...

题目：A CLOSER LOOK AT DEEP LEARNING HEURISTICS: LEARNING RATE RESTARTS, WARMUP AND DISTILLATIONABSTRACT:词：1.heuristics 启发式的 2.knowledge distillation 知识升华 3.underpinnings 基础 4.aid援助 5.empirical 经验 6.linear interpolation and visualizations with dimensio...

标题：ADAPTIVE GRADIENT METHODS WITH DYNAMIC BOUND OF LEARNING RATEAbstract：1.element-wise scaling term 逐元素缩放项 2.gradual and smooth transition 逐步平稳过渡 3.prototypes 原型4.dominant 优势短语：in spite of its simplicity 尽管很简单...

Abstract：优秀短语：1.SGD have become increasingly popular in optimization2.it is now widely recognized that3.play a key role in reaching "good" solutions that4.In an attempt to shed some light on why this is the case 试图阐明为什么会这样5.we also argue 我们也讨论了..