https://www.tensorflow.org/programmers_guide/variables
Variable
变量,通常用于表示和存储模型的参数,使用接口tf.Variable
类来定义。 它有以下几个特点:
1. 使用前必须显示初始化
2. 值可以变, 但shape一般不能变。除非validate_shape = False
3. 从物理结构上讲, 它在内存中就是一段缓存,里面存储着一个tensor
4. 它可以被保存到硬盘上
创建Variable
创建Variable时必须提供初始值,这个初始值可以是list, numpy array或另一个tf tensor
import tensorflow as tf
import numpy as np
a = tf.Variable([1, 1, 1], name = 'a')
b = tf.Variable(np.ones((1, 3)), name = 'b')
c = tf.Variable(tf.ones(1, 3), name = 'c')
指定device
在创建变量时可以指定变量存储的设备
Operations that mutate a variable, such as tf.Variable.assign and the parameter update operations in a tf.train.Optimizer must run on the same device as the variable. Incompatible device placement directives will be ignored when creating these operations
TODO
with tf.device('/cpu:0'):
d = tf.Variable([1] * 3, name = 'd')
with tf.device('/gpu:0'):
e = tf.Variable([1] * 3, name = 'e')
初始化
创建Variable时需要指定它的初始值, 但这个操作并没有真正执行初始操作。 初始化操作由初始化操作子(initialization operator)完成.
在run graph之前必须执行初始化操作。
init_op = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init_op)
print e.eval()
如果在执行上面的代码时报Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available
, 则使用下面的代码。原因还没仔细去看, 这个问题在github上有人提到过https://github.com/tensorflow/tensorflow/issues/2292 。
init_op = tf.global_variables_initializer()
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
sess.run(init_op)
print e.eval()
[1 1 1]
一个Variable可以使用另一个Variable作为它的初始值
w = tf.Variable(tf.random_normal([100, 100], stddev = 1))
w2 = tf.Variable(w.initialized_value() * 2)
保存和加载
使用tf.train.Saver
保存和加载Variables。
保存
import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')
init_op = tf.global_variables_initializer()
saver = tf.train.Saver()
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
sess.run(init_op)
saver.save(sess, '/tmp/model.ckpt')
加载
import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')
saver = tf.train.Saver()
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
saver.restore(sess, '/tmp/model.ckpt')
print bias.eval()
[[ 0.]
[ 0.]]
定制保存和加载
创建Saver
时, 可以指定保存/加载哪些参数, 及以什么名字保存/从什么变量加载
import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')
init_op = tf.global_variables_initializer()
saver = tf.train.Saver({'w':weights})
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
sess.run(init_op)
saver.save(sess, '/tmp/model.ckpt')
import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')
saver = tf.train.Saver({'w': weights})
config = tf.ConfigProto(allow_soft_placement = True)
init_op = tf.global_variables_initializer()
with tf.Session(config = config) as sess:
sess.run(init_op)
saver.restore(sess, '/tmp/model.ckpt')
print bias.eval()
print weights.eval().shape
[[ 0.]
[ 0.]]
(2, 2)
若是所有参数都被restored, 就不用sess.run(init_op)
了。但这里bias
没有restore, 所以还需要sess.run(init_op)