tensorflow随笔-影子变量值_tensorflow 影子-CSDN博客

本文链接：https://blog.csdn.net/ai_lx/article/details/89053743

本文介绍了TensorFlow中的滑动平均模型，包括影子变量的更新公式和apply()方法的作用。影子变量的值通过衰减率进行更新，合理的衰减率通常接近1.0。apply()方法用于创建训练变量的副本并维护其滑动平均值。此外，还提到了可选参数num_updates，它允许动态调整衰减率，以控制训练初期的移动平均速度。

摘要由CSDN通过智能技术生成

#!/usr/bin/env python2
# -*- coding: utf-8 -*-
import tensorflow as tf
my_var=tf.Variable(0.)
step=tf.Variable(0,trainable=False)
ema=tf.train.ExponentialMovingAverage(0.99,step)
maintain_average_op=ema.apply([my_var])

with tf.Session() as sess:
    init_op=tf.global_variables_initializer()
    sess.run(init_op)
    decay=0.99
    #影子变量值变化
    for i in range(1,6):
        print sess.run([my_var,ema.average(my_var)])
        sess.run(my_var.assign_add(i))
        sess.run(maintain_average_op)
        print sess.run([my_var,ema.average(my_var)])
        print "==="

    print "----------------"
    #num_updates即step变化
    sess.run(my_var.assign(5.))
    for i in range(1,20,3):     
        print sess.run([my_var,ema.average(my_var)])
        sess.run(step.assign_add(i))
        sess.run(maintain_average_op)
        print sess.run([my_var,ema.average(my_var)])
        print "==="

滑动平均模型

shadow_variable= decay shadow_variable + (1 - decay) variable

Reasonable values for decay are close to 1.0, typically in themultiple-nines range: 0.999, 0.9999, etc.

The apply() methodadds shadow copies of trained variables and add ops that maintain a movingaverage of the trained variables in their shadow copies. It is used whenbuilding the training model.

The optional num_updates parameter allows one to tweak thedecay rate dynamically. It is typical to pass the count of training steps,usually kept in a variable that is incremented at each step, in which case thedecay rate is lower at the start of training. This makes moving averages movefaster. If passed, the actual decay rate used is:

min(decay, (1 +num_updates) / (10 + num_updates))