tf.variable_scope 参数

最近在看TensorFlow的变量管理，发现很多代码中tf.variable_scope()参数的数量及意义还不太清楚，特此记录：

  def __init__(self,
               name_or_scope,
               default_name=None,
               values=None,
               initializer=None,
               regularizer=None,
               caching_device=None,
               partitioner=None,
               custom_getter=None,
               reuse=None,
               dtype=None,
               use_resource=None,
               constraint=None,
               auxiliary_name_scope=True):
    """Initialize the context manager.

    Args:
      name_or_scope: `string` or `VariableScope`: the scope to open.
      default_name: The default name to use if the `name_or_scope` argument is
        `None`, this name will be uniquified. If name_or_scope is provided it
        won't be used and therefore it is not required and can be None.
      values: The list of `Tensor` arguments that are passed to the op function.
      initializer: default initializer for variables within this scope.
      regularizer: default regularizer for variables within this scope.
      caching_device: default caching device for variables within this scope.
      partitioner: default partitioner for variables within this scope.
      custom_getter: default custom getter for variables within this scope.
      reuse: `True`, None, or tf.AUTO_REUSE; if `True`, we go into reuse mode
        for this scope as well as all sub-scopes; if tf.AUTO_REUSE, we create
        variables if they do not exist, and return them otherwise; if None, we
        inherit the parent scope's reuse flag. When eager execution is enabled,
        this argument is always forced to be tf.AUTO_REUSE.
      dtype: type of variables created in this scope (defaults to the type
        in the passed scope, or inherited from parent scope).
      use_resource: If False, all variables will be regular Variables. If True,
        experimental ResourceVariables with well-defined semantics will be used
        instead. Defaults to False (will later change to True). When eager
        execution is enabled this argument is always forced to be True.
      constraint: An optional projection function to be applied to the variable
        after being updated by an `Optimizer` (e.g. used to implement norm
        constraints or value constraints for layer weights). The function must
        take as input the unprojected Tensor representing the value of the
        variable and return the Tensor for the projected value
        (which must have the same shape). Constraints are not safe to
        use when doing asynchronous distributed training.
      auxiliary_name_scope: If `True`, we create an auxiliary name scope with
        the scope. If `False`, we don't touch name scope.

    Returns:
      A scope that can be captured and reused.

    Raises:
      ValueError: when trying to reuse within a create scope, or create within
        a reuse scope.
      TypeError: when the types of some arguments are not appropriate.
    """

上面的代码是tf.variable函数的定义，其中

 name_or_scope: `string` or `VariableScope`: the scope to open. 是变量空间的名称
 default_name: 当name_or_scope 使用时它就可以忽略，基本没什么用
 values: 传入该scope的tensor参数      

initializer=None: 默认的参数初始化函数

regularizer: d默认的正则化函数
caching_device: default caching device for variables within this scope.
partitioner: default partitioner for variables within this scope.
custom_getter: default custom getter for variables within this scope.

reuse: `True`, None, or tf.AUTO_REUSE; if `True`, we go into reuse mode
        for this scope as well as all sub-scopes; if tf.AUTO_REUSE, we create
        variables if they do not exist, and return them otherwise; if None, we
        inherit the parent scope's reuse flag. When eager execution is enabled,
        this argument is always forced to be tf.AUTO_REUSE

reuse有三种取值，默认取值是None：

True: 参数空间使用reuse 模式，即该空间下的所有tf.get_variable()函数将直接获取已经创建的变量，如果参数不存在tf.get_variable()函数将会报错。

AUTO_REUSE：若参数空间的参数不存在就创建他们，如果已经存在就直接获取它们。

None 或者False 这里创建函数tf.get_variable()函数只能创建新的变量，当同名变量已经存在时，函数就报错

下面是几个例子：

with tf.variable_scope("foo"):
    v = tf.get_variable("v", [1], initializer=tf.constant_initializer(1.0))
                        
#with tf.variable_scope("foo"):
   # v = tf.get_variable("v", [1])
    
with tf.variable_scope("foo", reuse=True):
    v1 = tf.get_variable("v", [1])
print(v == v1)

#with tf.variable_scope("bar", reuse=True):
   # v = tf.get_variable("v", [1])

输出是True

with tf.variable_scope("root"):       # reuse 在默认情况下为false 或者和 上一层保持一致
    print(tf.get_variable_scope().reuse)
    
    with tf.variable_scope("foo", reuse=True):
        print(tf.get_variable_scope().reuse)
        
        with tf.variable_scope("bar"):
            print(tf.get_variable_scope().reuse)
            
    print(tf.get_variable_scope().reuse)

tf.name_scope和tf.variable_scope的关系

tf.name_scope和tf.variable_scope是两个作用域，主要与创建/调用变量函数tf.Variable() 和tf.get_variable()搭配使用。首先说tf.Variable() 和tf.get_variable()的区别：

    tf.Variable() 每次调用都会产生一个新的变量，他会自动检测命名冲突并自行处理，变量名称是一个可选参数，例如：

a1 = tf.Variable(tf.random_normal(shape=[2, 3], mean=0, stddev=1), name='a1')

a2 = tf.Variable(tf.random_normal(shape=[2, 3], mean=0, stddev=1), name='a1')

print(a1.name)

print(a2.name)

print(a1==a2)

运行结果：

f1/a1:0
f1/a1_1:0

False    # a2实际变成了a1_1并且和a1 不是同一个变量

而tf.get_variable()则不同，遇到重命名的变量创建且变量名没有设置成共享变量（所谓的共享是指在同一参数空间下的共享，参数空间名称不一样就不能共享了）时，就会报错 ；相对应的，变量名称这个参数是必填参数，tf.get_variable()会根据这个参数去创建或者获取变量。

tf.name_scope()主要用于管理图中各种op，而tf.variable_scope()主要用于管理图中变量的名字，在 tf.name_scope下时，tf.get_variable()创建的变量名不受 name_scope 的影响（不受它的约束）

import tensorflow as tf

with tf.variable_scope('f2'):
    #a1 = tf.get_variable(name='a1', shape=[1], initializer=tf.constant_initializer(1))
    a1 = tf.Variable(tf.random_normal(shape=[2, 3], mean=0, stddev=1), name='a1')
    a2 = tf.Variable(tf.random_normal(shape=[2, 3], mean=0, stddev=1), name='a1')
    a5 = tf.get_variable(name='a1', shape=[1], initializer=tf.constant_initializer(1))

with tf.variable_scope('f2', reuse=True):
    a3 = tf.get_variable(name='a1', shape=[1], initializer=tf.constant_initializer(1))

    a4 = tf.Variable(tf.random_normal(shape=[2, 3], mean=0, stddev=1), name='a2')

print(a1.name)
print(a2.name)
print(a1==a2)
print(a5.name)
print(a3.name)
print(a4.name)

运行结果：

f2/a1:0
f2/a1_1:0
False
f2/a1_2:0
f2/a1_2:0
f2_1/a2:0