Optimizers 类进行迭代梯度修正

Training

Optimizers 类

Optimizer

The Optimizer base class provides methods to compute gradients for a loss and apply gradients to variables. A collection of subclasses implement classic optimization algorithms such as GradientDescent and Adagrad.

Optimizers这个基类提供函数用来计算梯度，并将梯度状态更新。该基类的一系类子类实现了一些经典的算法，比如说GradientDescent和Adagrad

class tf.train.Optimizer

• 1

Base class for optimizers.

This class defines the API to add Ops to train a model. You never use this class directly, but instead instantiate one of its subclasses such as GradientDescentOptimizer, AdagradOptimizer, or MomentumOptimizer.

用于优化的基类定义了一系列API，通过这些API可以在训练模型时添加一些操作。例如：

Usage
# Create an optimizer with the desired parameters.
opt = GradientDescentOptimizer(learning_rate=0.1)

# Add Ops to the graph to minimize a cost by updating a list of variables.
# "cost" is a Tensor, and the list of variables contains variables.Variable
# objects.
opt_op = opt.minimize(cost, <list of variables>)
Execute opt_op to do one step of training:
opt_op.run()

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10

Processing gradients before applying them.

Calling minimize() takes care of both computing the gradients and applying them to the variables. If you want to process the gradients before applying them you can instead use the optimizer in three steps： 
Compute the gradients with compute_gradients(). 
Process the gradients as you wish. 
Apply the processed gradients with apply_gradients(). 
可以通过minimize()函数来同时计算梯度并更新该梯度所对应的参数状态（其计算的就是对应的参数的梯度），若想先计算梯度，然后再将梯度对应参数状态更新，可以通过以下几步来使用：

* 利用 compute_gradients() 函数先计算梯度 
* 按照自己的需求来处理梯度 
* 调用apply_gradients() 函数来更新该梯度所对应的参数的状态。 
* 这样有一个好处是，梯度可以按照自己的需求来改动后再使用。 
例如：

# Create an optimizer.
opt = GradientDescentOptimizer(learning_rate=0.1)

# Compute the gradients for a list of variables.
grads_and_vars = opt.compute_gradients(loss, <list of variables>)

# grads_and_vars is a list of tuples (gradient, variable).  Do whatever you
# need to the 'gradient' part, for example cap them, etc.
capped_grads_and_vars = [(MyCapper(gv[0]), gv[1])) for gv in grads_and_vars]

# Ask the optimizer to apply the capped gradients.
opt.apply_gradients(capped_grads_and_vars)