Notes on Tensorflow（四）Variables

https://www.tensorflow.org/programmers_guide/variables

Variable

变量，通常用于表示和存储模型的参数，使用接口tf.Variable类来定义。 它有以下几个特点：
1. 使用前必须显示初始化
2. 值可以变， 但shape一般不能变。除非validate_shape = False
3. 从物理结构上讲， 它在内存中就是一段缓存，里面存储着一个tensor
4. 它可以被保存到硬盘上

创建Variable

创建Variable时必须提供初始值，这个初始值可以是list， numpy array或另一个tf tensor

import tensorflow as tf
import numpy as np
a = tf.Variable([1, 1, 1], name = 'a')
b = tf.Variable(np.ones((1, 3)), name = 'b')
c = tf.Variable(tf.ones(1, 3), name = 'c')

指定device

在创建变量时可以指定变量存储的设备

Operations that mutate a variable, such as tf.Variable.assign and the parameter update operations in a tf.train.Optimizer must run on the same device as the variable. Incompatible device placement directives will be ignored when creating these operations
TODO

with tf.device('/cpu:0'):
    d = tf.Variable([1] * 3, name = 'd')

with tf.device('/gpu:0'):
    e = tf.Variable([1] * 3, name = 'e')

初始化

创建Variable时需要指定它的初始值， 但这个操作并没有真正执行初始操作。 初始化操作由初始化操作子(initialization operator)完成.
在run graph之前必须执行初始化操作。

init_op = tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init_op)
    print e.eval()

如果在执行上面的代码时报Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available， 则使用下面的代码。原因还没仔细去看， 这个问题在github上有人提到过https://github.com/tensorflow/tensorflow/issues/2292 。

init_op = tf.global_variables_initializer()
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
    sess.run(init_op)
    print e.eval()

[1 1 1]

一个Variable可以使用另一个Variable作为它的初始值

w = tf.Variable(tf.random_normal([100, 100], stddev = 1))
w2 = tf.Variable(w.initialized_value() * 2)

保存和加载

使用tf.train.Saver保存和加载Variables。

保存

import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')
init_op = tf.global_variables_initializer()
saver = tf.train.Saver()
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
    sess.run(init_op)
    saver.save(sess, '/tmp/model.ckpt')

加载

import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')

saver = tf.train.Saver()
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
    saver.restore(sess, '/tmp/model.ckpt')
    print bias.eval()

[[ 0.]
 [ 0.]]

定制保存和加载

创建Saver时， 可以指定保存/加载哪些参数， 及以什么名字保存/从什么变量加载

import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')
init_op = tf.global_variables_initializer()
saver = tf.train.Saver({'w':weights})
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
    sess.run(init_op)
    saver.save(sess, '/tmp/model.ckpt')

import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')

saver = tf.train.Saver({'w': weights})
config = tf.ConfigProto(allow_soft_placement = True)
init_op = tf.global_variables_initializer()
with tf.Session(config = config) as sess:
    sess.run(init_op)
    saver.restore(sess, '/tmp/model.ckpt')
    print bias.eval()
    print weights.eval().shape

[[ 0.]
 [ 0.]]
(2, 2)

若是所有参数都被restored， 就不用sess.run(init_op)了。但这里bias没有restore, 所以还需要sess.run(init_op)