Tensorfow中使用tf.identity()的作用

先来看TensorFlow中对identity的定义：

####################################################################################################

def identity(input, name=None):  # pylint: disable=redefined-builtin
  r"""Return a tensor with the same shape and contents as input.
  Args:
    input: A `Tensor`.
    name: A name for the operation (optional).
  Returns:
    A `Tensor`. Has the same type as `input`.
  """
  if context.in_graph_mode():
    return gen_array_ops.identity(input, name=name)
  else:
    if context.context().device_name != input.device:
      return input._copy()  # pylint: disable=protected-access
    return input

####################################################################################################

它返回一个和输入的 tensor 大小和数值都一样的 tensor ,类似于 y=x 操作，我们通常可以查到以下使用示例：

import tensorflow as tf

g = tf.Graph()
with g.as_default():
    x = tf.Variable(1.0, name='x')
    x_plus_1 = tf.assign_add(x, 1, name='x_plus')

    with tf.control_dependencies([x_plus_1]):
        y = x
        z=tf.identity(x,name='z_added')

    init = tf.global_variables_initializer()

    with tf.Session() as sess:
        sess.run(init)
        for i in range(5):
            print(sess.run(z))
            # 输出 2,3,4,5,6

        # 如果改为输出 print(sess.run(y)) ,则结果为 1,1,1,1,1

但是所有的博客资料都没有详细说明为什么会这样，以及在 Graph 的构建中，这个方法应该怎么使用。

起初一看，这个方法好像并没有什么用，只是把输入原样复制了一遍，但是实际上，tf.identity在计算图内部创建了两个节点，send / recv节点，用来发送和接受两个变量，如果两个变量在不同的设备上，比如 CPU 和 GPU，那么将会复制变量，如果在一个设备上，将会只是一个引用。

之所以会出现上面代码的情况，就是因为 y = x 并没有在计算图中占有一席之地，所以每次sess.run(y) 的时候都没有进行它的上一步 tf.control_dependencies 的操作，而 z 的计算则不同，它是计算图内部的节点，所以每次sess.run(z) 的时候都会进行 tf.control_dependencies 的操作，所以它输出的值每次都会更新； 并且，我们可以想见，如果我们同时输出 y 和 z，那么 y 的值也会同步更新为2，3，4，5，6。

我们打印该计算图内部所有的 OP，可以看到也是没有 y 这样一个计算节点存在的：

[<tf.Operation 'x/initial_value' type=Const>, <tf.Operation 'x' type=VariableV2>, <tf.Operation 'x/Assign' type=Assign>, <tf.Operation 'x/read' type=Identity>, <tf.Operation 'x_plus/value' type=Const>, <tf.Operation 'x_plus' type=AssignAdd>, <tf.Operation 'z_added' type=Identity>, <tf.Operation 'init' type=NoOp>]

那么在什么时候使用这个方法呢？

它是通过在计算图内部创建 send / recv节点来引用或复制变量的，最主要的用途就是更好的控制在不同设备间传递变量的值；

另外，它还有一种常见的用途，就是用来作为一个虚拟节点来控制流程操作，比如我们希望强制先执行loss_averages_op或updata_op，然后更新相关变量。这可以实现为：

with tf.control_dependencies（[loss_averages_op]）：
  total_loss = tf.identity（total_loss）

或者：

with tf.control_dependencies（[updata_op]）：
  train_tensor = tf.identity（total_loss,name='train_op'）

在这里，tf.identity除了在执行 loss_averages_op之后标记total_loss张量被执行之外没有做任何有用的事情。

 

 

（1）与control_dependencies配套使用。主要解释见博文：https://blog.csdn.net/hu_guan_jie/article/details/78495297

（2）tf.identity is useful when you want to explicitly transport tensor between devices (like, from GPU to a CPU). The op adds send/recv nodes to the graph, which make a copy when the devices of the input and the output are different.

A default behavior is that the send/recv nodes are added implicitly when the operation happens on a different device but you can imagine some situations (especially in a multi-threaded/distributed settings) when it might be useful to fetch the value of the variable multiple times within a single execution of the session.run. tf.identity allows for more control with regard to when the value should be read from the source device. Possibly a more appropriate name for this op would be read.

Also, please note that in the implementation of tf.Variable link, the identity op is added in the constructor, which makes sure that all the accesses to the variable copy the data from the source only once. Multiple copies can be expensive in cases when the variable lives on a GPU but it is read by multiple CPU ops (or the other way around). Users can change the behavior with multiple calls to tf.identity when desired.

EDIT: Updated answer after the question was edited.

In addition, tf.identity can be used used as a dummy node to update a reference to the tensor. This is useful with various control flow ops. In the CIFAR case we want to enforce that the ExponentialMovingAverageOp will update relevant variables before retrieving the value of the loss. This can be implemented as:

with tf.control_dependencies([loss_averages_op]):
  total_loss = tf.identity(total_loss)

Here, the tf.identity doesn't do anything useful aside of marking the total_loss tensor to be ran after evaluating loss_averages_op.

（3）为没有名称参数的op分配name

In addition to the above, I simply use it when I need to assign a name to ops that do not have a name argument, just like when initializing a state in RNN's:

rnn_cell = tf.contrib.rnn.MultiRNNCell([cells])
# no name arg
initial_state = rnn_cell.zero_state(batch_size,tf.float32)
# give it a name with tf.identity()
initial_state = tf.identity(input=initial_state,name="initial_state")

 （4）当我们的输入数据以字节为单位进行序列化时，我们想要从该数据集中提取特征。 我们可以关键字格式执行此操作，然后为其获取占位符。 当有多个功能并且每个功能必须以不同的格式读取时，它的好处更加明显。

  #read the entire file in this placeholder      
  serialized_tf_example = tf.placeholder(tf.string, name='tf_example')

  #Create a pattern in which data is to be extracted from input files
  feature_configs = {'image': tf.FixedLenFeature(shape=[256], dtype=tf.float32),/
                     'text': tf.FixedLenFeature(shape=[128], dtype=tf.string),/
                     'label': tf.FixedLenFeature(shape=[128], dtype=tf.string),}

  #parse the example in key: tensor dictionary
  tf_example = tf.parse_example(serialized_tf_example, feature_configs)

  #Create seperate placeholders operation and tensor for each feature
  image = tf.identity(tf_example['image'], name='image')
  text  = tf.identity(tf_example['text'], name='text')
  label = tf.identity(tf_example['text'], name='label')

（4）In distribution training, we should use tf.identity or the workers will hang at waiting for initialization of the chief worker:

vec = tf.identity(tf.nn.embedding_lookup(embedding_tbl, id)) * mask
with tf.variable_scope("BiRNN", reuse=None):
    out, _ = tf.nn.bidirectional_dynamic_rnn(fw, bw, vec, sequence_length=id_sz, dtype=tf.float32)

For details, without identity, the chief worker would treat some variables as local variables inappropriately and the other workers wait for an initialization operation that can not end。 

 

（5）I found another application of tf.identity in Tensorboard. If you use tf.shuffle_batch, it returns multiple tensors at once, so you see messy picture when visualizing the graph, you can't split tensor creation pipeline from actiual input tensors：

But with tf.identity you can create duplicate nodes, which don't affect computation flow: