tf.Variable_scope

最新推荐文章于 2022-07-04 21:28:19 发布

Yongqiang Cheng

最新推荐文章于 2022-07-04 21:28:19 发布

阅读量1.3k

点赞数 1

分类专栏： TensorFlow - Keras 文章标签： tf.Variable_scope

世上没有白读的书，每一页都算数。

本文链接：https://blog.csdn.net/chengyq116/article/details/94588608

版权

TensorFlow - Keras 专栏收录该内容

120 篇文章 2 订阅

订阅专栏

tf.Variable_scope

site/en/api_docs/api_docs/python/tf/variable_scope.md

1. Class `variable_scope`

A context manager for defining ops that creates variables (layers).
用于定义创建变量 (layers) 的操作的上下文管理器。

This context manager validates that the (optional) values are from the same graph, ensures that graph is the default graph, and pushes a name scope and a variable scope.
此上下文管理器验证 (可选) values 来自同一个 graph，确保该图是默认 graph，并推送名称范围和变量范围。

If name_or_scope is not None, it is used as is. If scope is None, then default_name is used. In that case, if the same name has been previously used in the same scope, it will be made unique by appending _N to it.
如果 name_or_scope 不是 None，则按原样使用。如果 scope 为 None，则使用 default_name。在这种情况下，如果先前在同一范围内使用了相同的名称，则通过向其添加 _N 将使其成为唯一名称。

Variable scope allows you to create new variables and to share already created ones while providing checks to not create or share by accident.
变量范围允许您创建新变量并共享已创建的变量，同时提供不会意外创建或共享的检查。

accident ['æksɪdənt]：n. 事故，意外，意外事件，机遇

1.1 init

__init__(
    name_or_scope,
    default_name=None,
    values=None,
    initializer=None,
    regularizer=None,
    caching_device=None,
    partitioner=None,
    custom_getter=None,
    reuse=None,
    dtype=None,
    use_resource=None,
    constraint=None
)

Initialize the context manager.
初始化上下文管理器。

Args
name_or_scope: string or VariableScope: the scope to open. (string 或者 VariableScope 表示打开的范围。)
default_name: The default name to use if the name_or_scope argument is None, this name will be uniquified. If name_or_scope is provided it won’t be used and therefore it is not required and can be None. (如果 name_or_scope 参数为 None，则使用默认的名称，该名称将是唯一的。如果提供了 name_or_scope，它将不会被使用，因此它不是必需的，并且可以是 None。)
values: The list of Tensor arguments that are passed to the op function. (传递给操作函数的 Tensor 参数列表。)
initializer: default initializer for variables within this scope. (此范围内变量的默认初始值设定项。)
regularizer: default regularizer for variables within this scope. (此范围内变量的默认正规化器。)
caching_device: default caching device for variables within this scope. (此范围内变量的默认缓存设备。)
partitioner: default partitioner for variables within this scope. (此范围内变量的默认分区程序。)
custom_getter: default custom getter for variables within this scope. (此范围内变量的默认自定义 getter。)
reuse: True, None, or tf.AUTO_REUSE; if True, we go into reuse mode for this scope as well as all sub-scopes; if tf.AUTO_REUSE, we create variables if they do not exist, and return them otherwise; if None, we inherit the parent scope’s reuse flag. In Eager mode, this argument is always forced to be tf.AUTO_REUSE. (可以是 True、None 或 tf.AUTO_REUSE。如果是 True，则我们进入此范围的重用模式以及所有子范围。如果是 tf.AUTO_REUSE，则我们创建变量 (如果它们不存在)，否则返回它们。如果是 None，则我们继承父范围的重用标志。当启用紧急执行时，该参数总是被强制为 tf.AUTO_REUSE。)
dtype: type of variables created in this scope (defaults to the type in the passed scope, or inherited from parent scope). (在此范围中创建的变量类型 (默认为传入范围中的类型，或从父范围继承)。)
use_resource: If False, all variables will be regular Variables. If True, experimental ResourceVariables with well-defined semantics will be used instead. Defaults to False (will later change to True). In Eager mode, this argument is always forced to be True. (如果为 False，则所有变量都将是常规变量。如果为 True，则将使用具有明确定义的语义的实验性 ResourceVariables。默认为 False (稍后将更改为 True)。当启用紧急执行时，该参数总是被强制为 True。)
constraint: An optional projection function to be applied to the variable after being updated by an Optimizer (e.g. used to implement norm constraints or value constraints for layer weights). The function must take as input the unprojected Tensor representing the value of the variable and return the Tensor for the projected value (which must have the same shape). Constraints are not safe to use when doing asynchronous distributed training. (一个可选的投影函数，在被 Optimizer (例如用于实现层权重的范数约束或值约束) 更新之后应用于该变量。该函数必须将代表变量值的未投影张量作为输入，并返回投影值的张量 (它必须具有相同的形状)。进行异步分布式培训时，约束条件的使用是不安全的)

Returns
A scope that can be captured and reused.
可以捕获和重用的范围。

getter ['getə]：n. 吸气剂，获得者，毒饵 vt. 消气，用吸气剂去除 vi. 使用杂质吸收剂

Raises
ValueError: when trying to reuse within a create scope, or create within a reuse scope. (尝试在创建范围内重用或在重用范围内创建时。)
TypeError: when the types of some arguments are not appropriate. (当某些参数的类型不合适时。)

1.2 enter

__enter__()

1.3 exit

__exit__(
    type_arg,
    value_arg,
    traceback_arg
)

Simple example of how to create a new variable:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf
import numpy as np

with tf.variable_scope("foo"):
    with tf.variable_scope("bar"):
        v = tf.get_variable("v", [1])
        print(v.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
foo/bar/v:0
strong@foreverstrong:~/git_workspace/MonoGRNet$

Basic example of sharing a variable AUTO_REUSE:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf
import numpy as np


def foo():
    with tf.variable_scope("foo", reuse=tf.AUTO_REUSE):
        v = tf.get_variable("v", [1])
    return v


v1 = foo()  # Creates v.
v2 = foo()  # Gets the same, existing v.

print(v1.name)
print(v2.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
foo/v:0
foo/v:0
strong@foreverstrong:~/git_workspace/MonoGRNet$

Basic example of sharing a variable with reuse=True:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf
import numpy as np

with tf.variable_scope("foo"):
    v = tf.get_variable("v", [1])

with tf.variable_scope("foo", reuse=True):
    v1 = tf.get_variable("v", [1])

print(v.name)
print(v1.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
foo/v:0
foo/v:0
strong@foreverstrong:~/git_workspace/MonoGRNet$

Sharing a variable by capturing a scope and setting reuse:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf
import numpy as np

with tf.variable_scope("foo") as scope:
    v = tf.get_variable("v", [1])
    scope.reuse_variables()
    v1 = tf.get_variable("v", [1])

print(v.name)
print(v1.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
foo/v:0
foo/v:0
strong@foreverstrong:~/git_workspace/MonoGRNet$

To prevent accidental sharing of variables, we raise an exception when getting an existing variable in a non-reusing scope.
为防止意外共享变量，我们在非重用范围内获取现有变量时引发异常。

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf
import numpy as np

with tf.variable_scope("foo"):
    v = tf.get_variable("v", [1])
    v1 = tf.get_variable("v", [1])
    # Raises ValueError("... v already exists ...").

print(v.name)
print(v1.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
Traceback (most recent call last):
  File "yongqiang.py", line 13, in <module>
    v1 = tf.get_variable("v", [1])
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1203, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1092, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 425, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 394, in _true_getter
    use_resource=use_resource, constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 742, in _get_single_variable
    name, "".join(traceback.format_list(tb))))
ValueError: Variable foo/v already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:

  File "yongqiang.py", line 12, in <module>
    v = tf.get_variable("v", [1])

strong@foreverstrong:~/git_workspace/MonoGRNet$

Similarly, we raise an exception when trying to get a variable that does not exist in reuse mode.
类似地，我们在尝试获取在重用模式下不存在的变量时引发异常。

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf
import numpy as np

with tf.variable_scope("foo", reuse=True):
    v = tf.get_variable("v", [1])
    #  Raises ValueError("... v does not exists ...").

print(v.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
Traceback (most recent call last):
  File "yongqiang.py", line 12, in <module>
    v = tf.get_variable("v", [1])
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1203, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1092, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 425, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 394, in _true_getter
    use_resource=use_resource, constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 760, in _get_single_variable
    "reuse=tf.AUTO_REUSE in VarScope?" % name)
ValueError: Variable foo/v does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?
strong@foreverstrong:~/git_workspace/MonoGRNet$

Note that the reuse flag is inherited: if we open a reusing scope, then all its sub-scopes become reusing as well.
请注意，重用标志是继承的：如果我们打开一个重用范围，那么它的所有子范围也会重用。

A note about name scoping: Setting reuse does not impact the naming of other ops such as mult.
关于名称范围的说明：设置重用不会影响其他操作 (如 mult) 的命名。

2. tf.Variable_scope

当传入字符串时，用以给变量名添加前缀，类似于目录。

当 reuse=None (默认情况) 时，与上层 variable_scope 的 reuse 参数一样。

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf

with tf.Variable_scope('k1'):
    with tf.Variable_scope('k2'):
        init = tf.constant_initializer(0.1)
        weights0 = tf.get_variable('weights', [2, 2])
        bias0 = tf.get_variable('biases', [2, 2])

print(weights0.name)
print(bias0.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
k1/k2/weights:0
k1/k2/biases:0
strong@foreverstrong:~/git_workspace/MonoGRNet$

当 reuse=tf.AUTO_REUSE 时，自动复用。如果变量存在则复用，不存在则创建。这是安全的用法。

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf

with tf.Variable_scope('k1', reuse=tf.AUTO_REUSE):
    with tf.Variable_scope('k2'):
        init = tf.constant_initializer(0.1)
        weights0 = tf.get_variable('weights', [2, 2])
        bias0 = tf.get_variable('biases', [2, 2])

print(weights0.name)
print(bias0.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
k1/k2/weights:0
k1/k2/biases:0
strong@foreverstrong:~/git_workspace/MonoGRNet$

如果变量存在则复用。

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf

with tf.Variable_scope('k1'):
    with tf.Variable_scope('k2'):
        init = tf.constant_initializer(0.1)
        weights0 = tf.get_variable('weights', [2, 2])
        bias0 = tf.get_variable('biases', [2, 2])

print(weights0.name)
print(bias0.name)

with tf.Variable_scope('k1', reuse=tf.AUTO_REUSE):
    with tf.Variable_scope('k2'):
        init = tf.constant_initializer(0.1)
        weights0 = tf.get_variable('weights', [2, 2])
        bias0 = tf.get_variable('biases', [2, 2])

print(weights0.name)
print(bias0.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
k1/k2/weights:0
k1/k2/biases:0
k1/k2/weights:0
k1/k2/biases:0
strong@foreverstrong:~/git_workspace/MonoGRNet$

当 reuse=True 时，tf.get_variable 会查找该命名变量。如果没有找到，则会报错。设置 reuse=True 之前，要保证该命名变量已存在。

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf

with tf.Variable_scope('k1', reuse=True):
    with tf.Variable_scope('k2'):
        init = tf.constant_initializer(0.1)
        weights0 = tf.get_variable('weights', [2, 2])
        bias0 = tf.get_variable('biases', [2, 2])

print(weights0.name)
print(bias0.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
Traceback (most recent call last):
  File "yongqiang.py", line 13, in <module>
    weights0 = tf.get_variable('weights', [2, 2])
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1203, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1092, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 425, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 394, in _true_getter
    use_resource=use_resource, constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 760, in _get_single_variable
    "reuse=tf.AUTO_REUSE in VarScope?" % name)
ValueError: Variable k1/k2/weights does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?
strong@foreverstrong:~/git_workspace/MonoGRNet$

变量已存在示例。

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf

with tf.Variable_scope('kv1'):
    with tf.Variable_scope('kv2'):
        init = tf.constant_initializer(0.1)
        weights1 = tf.get_variable('weights', [2, 2])
        bias1 = tf.get_variable('biases', [2, 2])

print(weights1.name)
print(bias1.name)

with tf.Variable_scope('kv1', reuse=True):
    with tf.Variable_scope('kv2'):
        init = tf.constant_initializer(0.1)
        weights1 = tf.get_variable('weights', [2, 2])
        bias1 = tf.get_variable('biases', [2, 2])

print(weights1.name)
print(bias1.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
kv1/kv2/weights:0
kv1/kv2/biases:0
kv1/kv2/weights:0
kv1/kv2/biases:0
strong@foreverstrong:~/git_workspace/MonoGRNet$

当 reuse=False 时，tf.get_variable会调用tf.Variable来创建变量，并检查创建的变量是否以存在，如果已存在，则报错。

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf

with tf.Variable_scope('kv1'):
    with tf.Variable_scope('kv2'):
        init = tf.constant_initializer(0.1)
        weights1 = tf.get_variable('weights', [2, 2])
        bias1 = tf.get_variable('biases', [2, 2])

print(weights1.name)
print(bias1.name)

with tf.Variable_scope('kv1', reuse=False):
    with tf.Variable_scope('kv2'):
        init = tf.constant_initializer(0.1)
        weights1 = tf.get_variable('weights', [2, 2])
        bias1 = tf.get_variable('biases', [2, 2])

print(weights1.name)
print(bias1.name)

strong@foreverstrong:~/git_workspace/MonoGRNet$ python yongqiang.py 
kv1/kv2/weights:0
kv1/kv2/biases:0
Traceback (most recent call last):
  File "yongqiang.py", line 22, in <module>
    weights1 = tf.get_variable('weights', [2, 2])
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1203, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1092, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 425, in get_variable
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 394, in _true_getter
    use_resource=use_resource, constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 742, in _get_single_variable
    name, "".join(traceback.format_list(tb))))
ValueError: Variable kv1/kv2/weights already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:

  File "yongqiang.py", line 13, in <module>
    weights1 = tf.get_variable('weights', [2, 2])

strong@foreverstrong:~/git_workspace/MonoGRNet$