TensorFlow基础name_scope与variable_scope

name_scope与variable_scope

import tensorflow as tf

Outline

• 变量重用
• 总结

1.变量重用

在Tensorflow中，直接重用变量会出错，必须指明 reuse。 变量重用实现的就是RNN中的参数共享操作。此处的 with 和其他处的不太一样，执行完后，里面所申请的变量还驻留在内存中，所以打印语句在外面也是可以实现的。

# name_scope + Variable => 无法实现变量重用
with tf.name_scope('scope_1'): # 如果重复执行语句，scope 的名称也会被系统自动标上不同的数字加以区分
    var1 = tf.Variable(initial_value=[1.0], name='var1', dtype=tf.float32)
    var2 = tf.Variable(initial_value=[2.0], name='var2', dtype=tf.float32)
    var22 = tf.Variable(initial_value=[3.0], name='var2', dtype=tf.float32) # 这种方式并不能达到重用变量的目的

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print('Value = %f \t Name = %s' % (sess.run(var1), var1.name) )
    print('Value = %f \t Name = %s' % (sess.run(var2), var2.name) )
    print('Value = %f \t Name = %s' % (sess.run(var22), var22.name) )

Value = 1.000000     Name = scope_1/var1:0
Value = 2.000000     Name = scope_1/var2:0
Value = 3.000000     Name = scope_1/var2_1:0

# 使用 variable_scope + get_variable => 实现变量重用
with tf.variable_scope('scope_2') as scope:
    init_val = tf.constant_initializer(value=1.0)
    var1 = tf.get_variable(name='var1', initializer=init_val, dtype=tf.float32, shape=[1])
    var2 = tf.get_variable(name='var2', initializer=init_val, dtype=tf.float32, shape=[1])
    scope.reuse_variables() # 该语句之后的变量，可以重用，如果不加就会出错
    var3 = tf.get_variable(name='var2', initializer=init_val, dtype=tf.float32, shape=[1])

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print('Value = %f \t Name = %s' % (sess.run(var1), var1.name) )
    print('Value = %f \t Name = %s' % (sess.run(var2), var2.name) )
    print('Value = %f \t Name = %s' % (sess.run(var3), var3.name) )

Value = 1.000000     Name = scope_2/var1:0
Value = 1.000000     Name = scope_2/var2:0
Value = 1.000000     Name = scope_2/var2:0

# 使用 variable_scope + Variable => 无法实现变量重用
with tf.variable_scope('scope_3') as scope:
    var1 = tf.Variable(initial_value=[1.0], name='var1', dtype=tf.float32)
    var2 = tf.Variable(initial_value=[2.0], name='var2', dtype=tf.float32)
    var3 = tf.Variable(initial_value=[3.0], name='var2', dtype=tf.float32) # 想要实现 reuse 语句

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print('Value = %f \t Name = %s' % (sess.run(var1), var1.name) )
    print('Value = %f \t Name = %s' % (sess.run(var2), var2.name) )
    print('Value = %f \t Name = %s' % (sess.run(var3), var3.name) ) # 系统检测到变量重名，不会重用它，给是自动修改名称，加以区分

Value = 1.000000     Name = scope_3/var1:0
Value = 2.000000     Name = scope_3/var2:0
Value = 3.000000     Name = scope_3/var2_1:0

总结

• name_scope 会给在它自己作用域下的变量添加 prefix（前缀）。（通过 get_variable 创建的变量除外）；
• 重复执行 name_scope 包含的语句时, scope的名称也会自动添加标号，加以区分；
• variable_scope + get_variable 是实现变量 reuse 的方式

When you create a variable with tf.get_variable instead of tf.Variable, Tensorflow will start checking the names of the vars created with the same method to see if they collide. If they do, an exception will be raised. If you created a var with tf.get_variable and you try to change the prefix of your variable names by using the tf.name_scope context manager, this won’t prevent the Tensorflow of raising an exception. Only tf.variable_scope context manager will effectively change the name of your var in this case. Or if you want to reuse the variable you should call scope.reuse_variables() before creating the var the second time.

In summary, tf.name_scope just add a prefix to all tensor created in that scope (except the vars created with tf.get_variable), and tf.variable_scope add a prefix to the variables created with tf.get_variable[1].

Reference

1. https://stackoverflow.com/questions/34215746/difference-between-variable-scope-and-name-scope-in-tensorflow