tensorflow quene queueRunner

转自：http://blog.csdn.net/lujiandong1/article/details/53369961

队列本身也是图中的一个节点。其他节点（enqueue, dequeue）可以修改队列节点中的内容。

#-*- coding:utf-8 -*-  
import tensorflow as tf  

#创建的图:一个先入先出队列,以及初始化,出队,+1,入队操作  
q = tf.FIFOQueue(3, "float")  
init = q.enqueue_many(([0.1, 0.2, 0.3],))  #一次往队列中输入多个值
x = q.dequeue()  #从队列中取出一个值
y = x + 1  
q_inc = q.enqueue([y])  #往队列中push一个值

#开启一个session,session是会话,会话的潜在含义是状态保持,各种tensor的状态保持  
with tf.Session() as sess:  
        sess.run(init)  

        for i in range(2):  
                sess.run(q_inc)  

        quelen =  sess.run(q.size())  
        for i in range(quelen):  
                print (sess.run(q.dequeue()))  

---

https://indico.io/blog/tensorflow-data-inputs-part1-placeholders-protobufs-queues/
队列

A TensorFlow queue is quite similar to a regular queue, they are
symbolic and only performed on Tensorflow’s graph. As part of the API,
a TFRecordReader always acts on a queue of filenames. It will pop a
filename off the queue and use that filename until the tfrecord is
empty. At this point it will grab the next filename off the filename
queue. Both the queue and the TFRecordReader have some state to keep
track of where they are. The only piece missing is how the filename
queue is actually fed. On initialization it is empty. This is where
the concept of QueueRunners comes in. There’s nothing magical about
it. At its core, it is simply a thread that uses a session and calls
an enqueue op over and over again.

TFRecordReader是在一个文件队列上操作。从队列中取出一个文件名，读取完这个文件里内容后又取出另一个文件名，直接队列为空。 队列最开始是空的，队列的填充是通过QueueRunners来实现的。QueueRunners是一个线程，他会重复调用入队列操作来填充队列。

We do, however, have to signal to TensorFlow to start these threads.
Without this, the queue will be blocked indefinitely, waiting for data
to be enqueued. To start the QueueRunners you can call
tf.train.start_queue_runners(sess=sess). This call is not symbolic. It
goes and creates threads. It is important to note that this must be
called after the initialization op is run. These threads try to
enqueue data to queues. If the queues are not initialized, TensorFlow
will rightly throw an error.

在代码里需要显示的启动tensorflow里的线程，否则队列一直为空，处于阻塞状态。可以在初始化操作函数执行后调用tf.train.start_queue_runners(sess=sess)来启动线程，这个函数会启动图里面的QueueRunners集合。

---

batch的实现

Great! Now we have two Tensors that represents a single example. We
could train with this using gradient descent, but that is rather
silly. It has been shown that training with batches of examples work
far better than training with a single examples.

# get single examples
label, image = read_and_decode_single_example("mnist.tfrecords")
# 返回的images_batch是一个list，代表一个batch，batch_size=128
images_batch, labels_batch = tf.train.shuffle_batch(
    [image, label], batch_size=128,
    capacity=2000,
    min_after_dequeue=1000)

sess = tf.Session()
init = tf.initialize_all_variables()
sess.run(init)
tf.train.start_queue_runners(sess=sess)#启动图中的QueueRunners集合
labels, images= sess.run([labels_batch, images_batch])

As discussed above, if label and image represent a single example, how can we make a batch of different examples? The answer lies in another set of queues and QueueRunners. shuffle_batch constructs a RandomShuffleQueue and proceeds to fill it with individual image and labels. This filling is done on a separate thread with a QueueRunner. The RandomShuffleQueue accumulates examples sequentially until it contains batch_size +min_after_dequeue examples are present. It then selects batch_size random elements from the queue to return. The value actually returned by shuffle_batch is the result of a dequeue_many call on the RandomShuffleQueue.