今天在查阅tf.variable_scope()时看到一篇博文对tf中scope的解释很全面,特此转载记录。
1. tf.name_scope('scope_name')或tf.name_scope(named_scope)
主要与tf.Variable搭配使用;
当传入字符串时,用以给变量名添加前缀,类似于目录,如case1所示;
当传入已存在的name_scope对象时,则其范围内变量的前缀只与当前传入的对象有关,与更上层的name_scope无关,如case2所示。
import tensorflow as tf
# case 1:
with tf.name_scope('l1'):
with tf.name_scope('l2'):
wgt1 = tf.Variable([1,2,3], name='wgts')
bias1 = tf.Variable([0.1], name='biases')
print wgt1.name, bias1.name
# >>> l1/l2/wgts:0 l1/l2/biases:0
# case 2:
with tf.name_scope('l1') as l1_scp:
with tf.name_scope('l2'):
wgt0 = tf.Variable([1,2,3], name='wgts')
bias0 = tf.Variable([0.1], name='biases')
with tf.name_scope(l1_scp):
wgt1 = tf.Variable([1,2,3], name='wgts')
bias1 = tf.Variable([0.1], name='biases')
print wgt0.name, bias0.name, wgt1.name, bias1.name
# >>> l1_1/l2/wgts:0 l1_1/l2/biases:0 l1_1/wgts:0 l1_1/biases:0
2. tf.variable_scope('scope_name', reuse=None)
与name_scope一样:当传入字符串时,用以给变量名添加前缀,类似于目录;
当传入已存在的variable_scope对象时,则其范围内变量的前缀只与当前传入的对象有关,与更上层的variable_scope无关。
常于get_variable搭配使用,多用于变量共享;其中 reuse 参数可设为 None、tf.AUTO_REUSE、True、False;
当 reuse=None(默认情况)时,与上层variable_scope的reuse参数一样。
# case 1
with tf.variable_scope('lv1'):
with tf.variable_scope('lv2'):
init = tf.constant_initializer(0.1)
wgt1 = tf.get_variable('wgts', [2,2])
bias1 = tf.get_variable('biases', [2,2])
print wgt1.name, bias1.name
# >>> lv1/lv2/wgts:0 lv1/lv2/biases:0
当 reuse=tf.AUTO_REUSE 时,自动复用,如果变量存在则复用,不存在则创建。这是最安全的用法。
with tf.variable_scope('lv1'):
with tf.variable_scope('lv2'):
init = tf.constant_initializer(0.1)
wgt1 = tf.get_variable('wgts', [2,2])
bias1 = tf.get_variable('biases', [2,2])
print wgt1.name, bias1.name
# >>> lv1/lv2/wgts:0 lv1/lv2/biases:0
with tf.variable_scope('lv1', reuse=tf.AUTO_REUSE):
with tf.variable_scope('lv2'):
init = tf.constant_initializer(0.1)
wgt2 = tf.get_variable('wgts', [2,2])
bias2 = tf.get_variable('biases', [2,2])
print wgt2.name, bias2.name
# >>> lv1/lv2/wgts:0 lv1/lv2/biases:0
当 reuse=True 时,tf.get_variable会查找该命名变量,如果没有找到,则会报错;所以设置reuse=True之前,要保证该命名变量已存在。
with tf.variable_scope('lv1', reuse=True):
with tf.variable_scope('lv2'):
init = tf.constant_initializer(0.1)
wgt1 = tf.get_variable('wgts', [2,2])
bias1 = tf.get_variable('biases', [2,2])
print wgt1.name, bias1.name
# >>> ValueError: Variable lv1/lv2/wgts does not exist,
# or was not created with tf.get_variable(). Did you mean
# to set reuse=tf.AUTO_REUSE in VarScope?
命名变量已存在:
with tf.variable_scope('lv1'):
with tf.variable_scope('lv2'):
init = tf.constant_initializer(0.1)
wgt1 = tf.get_variable('wgts', [2,2])
bias1 = tf.get_variable('biases', [2,2])
print wgt1.name, bias1.name
# >>> lv1/lv2/wgts:0 lv1/lv2/biases:0
# case 2
with tf.variable_scope('lv1', reuse=True):
with tf.variable_scope('lv2'):
init = tf.constant_initializer(0.1)
wgt1 = tf.get_variable('wgts', [2,2])
bias1 = tf.get_variable('biases', [2,2])
print wgt1.name, bias1.name
# >>> lv1/lv2/wgts:0 lv1/lv2/biases:0
当 reuse=False 时,tf.get_variable会调用tf.Variable来创建变量,并检查创建的变量是否以存在,如果已存在,则报错;
with tf.variable_scope('lv1'):
with tf.variable_scope('lv2'):
init = tf.constant_initializer(0.1)
wgt1 = tf.get_variable('wgts', [2,2])
bias1 = tf.get_variable('biases', [2,2])
print wgt1.name, bias1.name
# >>> lv1/lv2/wgts:0 lv1/lv2/biases:0
# case 2
with tf.variable_scope('lv1', reuse=False):
with tf.variable_scope('lv2'):
init = tf.constant_initializer(0.1)
wgt1 = tf.get_variable('wgts', [2,2])
bias1 = tf.get_variable('biases', [2,2])
print wgt1.name, bias1.name
# ValueError: Variable lv1/lv2/wgts already exists, disallowed.
# Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope?