关于Tensorflow Variables以及get_variable的使用方法

关于Tensorflow Variables

首先，我们认识到tf.variable_scope(): 可以让变量拥有相同的变量命名。
它可以返回一个用于定义创建variable层的操作的上下文管理器。

可变范围允许创建新的variable并分享已经创建的variable，同时提供检查，不会意外创建或共享。

如何创建variable的简单实例

with tf.variabel_scop("foo"):
	with tf.variable_scope("bar"):
		v = tf.get_variabel("v", [1])
		asser v.name == "foo/bar/v:0"

共享variable的基本示例：

with tf.variable_scope("foo"):
	v = tf.get_variable("v", [1])
with tf.variable_scope("foo", reuse=True):
	v1 = tf.get_variabel("v", [1])
assert v1 == v

譬如说，我们在写一个加法函数的时候，你的代码是

# function to  Add two variables 
# tf.Tensor is implicitly named like <OP_NAME>:<i>
# <OP_NAME>: the name of operation produce the tensor.
# <i>:  integer representing the index of the tensor among the operation's outputs.
def add_function():
    x = tf.Variable(3, name="x_scalar")
    y = tf.Variable(2, name="y_scalar")
    addition = tf.add(x,  y, name="add_function")
    print("=== checking Variables ===")
    print("x:", x, "\ny:", y, "\n")
    return addition

# To check whether or noe result1 and result2 is different
# First Call creates one set of 2 variables.
result1 = add_function()
# Second Call creates another set of 2 variables.
result2 = add_function()

这里，我们可以看到，您已经有两个不同的变量在 函数中，每当你叫这个函数，他都会创建 这样两个变量 在add_function中。叫这个函数生成的结果是

# === result1 = add_function() ===
x: <tf.Variable 'x_scalar:0' shape=() dtype=int32_ref> 
y: <tf.Variable 'y_scalar:0' shape=() dtype=int32_ref> 

# === result2 = add_function() ===
x: <tf.Variable 'x_scalar_1:0' shape=() dtype=int32_ref> 
y: <tf.Variable 'y_scalar_1:0' shape=() dtype=int32_ref> 

# === the result1 and result2 ===
result1: Tensor("add_function:0", shape=(), dtype=int32) 
result2: Tensor("add_function_1:0", shape=(), dtype=int32)

正如你看到的，每个变量的名称都不一样。如果你想要在函数中分享变量。一个基本的方式是在代码的另一个地方创建这些变量，并且把他们传入进函数并进行使用。例如

dictionary = {x: tf.Varibale('x', name='x'), 
              y: tf.Variable('y', name='y')}

def add_function(x, y):
    addition = tf.add(x,  y, name="add_function")
    print("=== checking Variables ===")
    print("x:", x, "\ny:", y, "\n")
    return addition

# To check whether or noe result1 and result2 is the same
# First Call creates one set of 2 variables.
result1 = add_function(variables_dict["x_scalar"], variables_dict["y_scalar"])
# Second Call also creates the same set of 2 variables.
result2 = add_function(variables_dict["x_scalar"], variables_dict["y_scalar"])
print("=== checking Variables ===")
print("result1:", result1, "\nresult2:", result2, "\n")

# To initialize all variables in this default graph.
global_init_op = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(global_init_op)
    print("=== checking Variables in a Session ===")
    print("result1:", result1, "\nresult2:", result2, "\n")
    result1_, result2_ = sess.run([result1, result2])
    print("=== the value each tensor has ===")
    print("result1:", result1_, "\nresult2:", result2_)
    print("[result1, result2]:", sess.run([result1, result2]))

结果如下，

=== checking Variables ===
x: <tf.Variable 'x_scalar_2:0' shape=() dtype=int32_ref> 
y: <tf.Variable 'y_scalar_2:0' shape=() dtype=int32_ref> 

=== checking Variables ===
x: <tf.Variable 'x_scalar_2:0' shape=() dtype=int32_ref> 
y: <tf.Variable 'y_scalar_2:0' shape=() dtype=int32_ref> 

=== checking Variables ===
result1: Tensor("add_function_2:0", shape=(), dtype=int32) 
result2: Tensor("add_function_3:0", shape=(), dtype=int32) 

这里唯一需要注意的就是res1和res2是不相通的。另外一条路是建立静态的类变量。用他们来作为分享的变量。

When you use class, one is to create a model using class. the other is to create class to only include variables. i.e. the classes take care of managing the variables they need.

为了不小心使用到重复的variable，我们可以在非重用范围内获取现有variable时，引发异常。

with tf.variable_scope("foo"):
    v = tf.get_variable("v", [1])
    v1 = tf.get_variable("v", [1])
    #  Raises ValueError("... v already exists ...").

请注意，重用标志是继承的：如果我们打开一个重用的范围，那么它的所有子范围也会变得重用。

ARGS：…

返回：
可以捕获和重复使用的范围。

get_variable的使用方法

参量以及functionality

该函数共有十一个参数， 常用的是，name, shape, dtype, 变量初始化方法 initializer, 所属于的集合collections.在这里插入代码片

 def get_variable(name,
                  shape=None,
                  dtype=None,
                  initializer=None,
                  regularizer=None,
                  trainable=True,
                  collections=None,
                  caching_device=None,
                  partitioner=None,
                  validate_shape=True,
                  use_resource=None,
                  custom_getter=None):

 import tensorflow as tf;  
 import numpy as np;  
 
 #常量初始化器
 v1_cons = tf.get_variable('v1_cons', shape=[1,4], initializer=tf.constant_initializer())
 v2_cons = tf.get_variable('v2_cons', shape=[1,4], initializer=tf.constant_initializer(9))
 #正太分布初始化器
 v1_nor = tf.get_variable('v1_nor', shape=[1,4], initializer=tf.random_normal_initializer())
 v2_nor = tf.get_variable('v2_nor', shape=[1,4], initializer=tf.random_normal_initializer(mean=0, stddev=5, seed=0))#均值、方差、种子值
 #截断正态分布初始化器
 v1_trun = tf.get_variable('v1_trun', shape=[1,4], initializer=tf.truncated_normal_initializer())
 v2_trun = tf.get_variable('v2_trun', shape=[1,4], initializer=tf.truncated_normal_initializer(mean=0, stddev=5, seed=0))#均值、方差、种子值
 #均匀分布初始化器
 v1_uni = tf.get_variable('v1_uni', shape=[1,4], initializer=tf.random_uniform_initializer())
 v2_uni = tf.get_variable('v2_uni', shape=[1,4], initializer=tf.random_uniform_initializer(maxval=-1., minval=1., seed=0))#最大值、最小值、种子值
 
 
 with tf.Session() as sess:
     sess.run(tf.global_variables_initializer())
     print("常量初始化器v1_cons:",sess.run(v1_cons))
     print("常量初始化器v2_cons:",sess.run(v2_cons))
     print("正太分布初始化器v1_nor:",sess.run(v1_nor))
     print("正太分布初始化器v2_nor:",sess.run(v2_nor))
     print("截断正态分布初始化器v1_trun:",sess.run(v1_trun))
     print("截断正态分布初始化器v2_trun:",sess.run(v2_trun))
     print("均匀分布初始化器v1_uni:",sess.run(v1_uni))
     print("均匀分布初始化器v2_uni:",sess.run(v2_uni))

如何理解tf.get_variable()

自此，让我们理解 tf.get_variable()， 我们只要知道 scope name 和 variable name，我们就可以重复使用。当我们用以下方式叫这个方法的时候

v = tf.get_variable(name, shape, dtype, initializer)

## How tf.getvarialbe works
# First Call create a tensor named "x_scalar"
x = tf.get_variable("x_scalar", [])
# Tensorflow aslo knows creating a tensor name "x_sclar" in a Second Call
# But, at this time, error happen like this:
# That's why reuse option is False 
y = tf.get_variable("x_scalar", []) # error message -> Variable x_scalar already exists, disallowed ...

我们注意到， 无论什么时候叫tf.get_variable, 他都尝试创建一个新的变量，named as specified in the name argument.
有两种情况，根据不同的scope范围，
Case 1: The scope is set to create new variable as evidenced by tf.get_variable_scope() == False.

Case 2: The scope is set to reuse the already existing variable as evidenced by tf.get_variable_scope() == True.

第一种情况，tf.get_variable_scope() == False. 如果如下声明

# By default, resue optiong is set to False
test = tf.get_variable("test_vector", [2]) # Variable scope "variable name" -> test_vector
print("=== variable scope ===") # test.op.name: test_vector 
print("test.name:", test.name) # test.name: test_vector:0
print("test.op.name:", test.op.name, "\n") # test.op.name: test_vector 
assert test.name == "test_vector:0"

# with context statement
with tf.variable_scope("test"):
    v = tf.get_variable("v_vector", [2]) # Variable scope "scope/variable name" -> test/v_vector
print("=== variable scope ===") # === variable scope ===
print("v.name:", v.name) # v.name: test/v_vector:0
print("v.op.name:", v.op.name) # v.op.name: test/v_vector
assert v.name == "test/v_vector:0"

在创建变量之前，我们给这些变量以scope name + name, tf.get_varible的方式是先检查一个变量是不是存在 full name 等于现有的 scope name 和 scope名称。
如果不存在，创立新的，如果存在，就raise value error。

接下来，我们看另外一种情况

tf.get_variable_scope() == True.

# with context statement
with tf.variable_scope("test"):
    var = tf.get_variable("var_vector", [2]) # Variable scope -> test/var_vector
with tf.variable_scope("test", reuse=True):
    var1 = tf.get_variable("var_vector", [2])
print("=== variable scope ===")
print("var.name:", var.name, "| var1.name:", var1.name)
print("var.op.name:", var.op.name, "| var1.op.name:", var1.op.name)
assert var.name == "test/var_vector:0"
assert var1.name == "test/var_vector:0"
assert var1 is var

在这种情况下， tf.get_variable()将会寻找一个已经存在的变量，其scope name + name 是现存的变量范围名称和名称。那么，如果不存在的话，tf.get_variable() 就会相应的产生错误。如果variable找到了，get_variable方法就return那个variable。

另外一种设定tf.get_variable()的reuse特性的方式。

让我们介绍一些tf.variable_scope()的用法，

# with context statement
with tf.variable_scope("test"):
    with tf.variable_scope("simple"):
        obj = tf.get_variable("obj_vector", [2]) # Variable scope -> test/simple/obj_vector
print("=== variable scope ===")
print("obj.name:", obj.name)
print("obj.op.name:", obj.op.name)
assert obj.name == "test/simple/obj_vector:0"

正如你所见到的，把变量范围给网罗起来，会将他们的scope名称叠加成一个目录结构。当你把“重复使用” 的旗立起来的时候，你可以通过叫resue_variables做上述同样的事情。这是引文叫reuse_variables可以召回当前的变量范围。
When you set the reuse flag up, you can do the same thing by calling tf.get_variable_scope().resue_variables(). That is because calling tf.get_variable_scope().resue_variables() can retrieve the current variable scope.

所以我们有

# with context statement
with tf.variable_scope("test"):
    t = tf.get_variable("t_vector", [2]) # Variable scope  -> test/t_vector
    # instead of with tf.variable_scope("test", reuse=True):
    tf.get_variable_scope().reuse_variables()
    t1 = tf.get_variable("t_vector", [2])
print("=== variable scope ===")
print("t.name:", t.name, "| t1.name:", t1.name)
print("t.op.name:", t.op.name, "| t1.op.name:", t1.op.name)
assert t.name == "test/t_vector:0"
assert t1.name == "test/t_vector:0"
assert t1 is t

那么要注意的地方就是，如果我们不叫reuse_variables，那么，就会报错，因为，我们使用了同样一个variable name，在同样一个variable scope “test”下。
所以，scope.reuse_variables()在建立LSTM模型的过程当中非常重要。因为dynamiclly建立cell的时候，要重复不断的使用同样模型参数变量。