Tensorflow框架学习（三）

Introduction to Graphs

1. Intorduction to TensorFlow graphs

长期以来，人们对 TensorFlow 的最大抱怨是调试不灵活，随着 TensorFlow 2.0 的出现，情况发生了巨大变化。

现在 TensorFlow 允许快速运行和操作。这意味着，可以通过 Python 运行 TensorFlow 操作并将输出再次返回给 Python。这样创造了很大的灵活性，特别是对于项目调试。

但是不使用快速操作选项有一些优点。我们可以在 TensorFlow 图上进行操作，这样在某些情况下会显著提高操作速度。

援引Tensorflow原话：

• Graphs are data structures that contain a set of tf.Operation objects, which represent units of computation; and tf.Tensor objects, which represent the units of data that flow between operations. They are defined in a tf.Graph context. Since these graphs are data structures, they can be saved, run, and restored all without the original Python code.（图是包含一系列 tf.Operation 对象的数据结构，代表计算单元和 tf.Tensor 对象，它们表示在操作之间流动的数据单元。它们在 tf.Graph 上下文中定义。由于这些图是数据结构，因此无需原始 Python 代码即可保存、运行和恢复它们。）

关于Graph和Session相关知识，请参考这里

2. Operation

让我们举一些将函数转换为图形的例子，我们可以使用 @tf.function 装饰器在图形上使用 Python 函数。

# Loading necessary libraries
import tensorflow as tf
import numpy as np
import timeit

@tf.function
def multiply_fn(a, b):
  return tf.matmul(a, b)

# Create some tensors
a = tf.constant([[0.5, 0.5]])
b = tf.constant([[10.0], [1.0]])

# Check function
print('Multiple a of shape {} with b of shape {}'.format(a.shape, b.shape))
print(multiply_fn(a, b).numpy())

# Output
'''
Multiple a of shape (1, 2) with b of shape (2, 1)
[[10.5 10.5]
 [ 1.5  1.5]]
'''

# Function without neing take to graph, i.e., with eager execution.
def add_fn(a, b):
  return tf.add(a, b)

# Create some tensors
a = tf.constant([[0.5, 0.5]])
b = tf.constant([[10.0], [1.0]])

# Check function
print('Add a of shape {} with b of shape {}'.format(a.shape, b.shape))
print(add_fn(a, b).numpy())

# Output
'''
Add a of shape (1, 2) with b of shape (2, 1)
[[5.5]]
'''

简单来看没有什么特别的区别，请看下一节的例子

3. Speedup

现在让我们定义一个模型，并且利用快速模式和图方法来运行

class ModelShallow(tf.keras.Model):

  def __init__(self):
    super(ModelShallow, self).__init__()
    self.dense1 = tf.keras.layers.Dense(10, activation=tf.nn.relu)
    self.dense2 = tf.keras.layers.Dense(20, activation=tf.nn.relu)
    self.dense3 = tf.keras.layers.Dense(30, activation=tf.nn.softmax)
    self.dropout = tf.keras.layers.Dropout(0.5)

  def call(self, inputs, training=False):
    x = self.dense1(inputs)
    if training:
      x = self.dropout(x, training=training)
    x = self.dense2(x)
    out = self.dense3(x)
    return out

class ModelDeep(tf.keras.Model):

  def __init__(self):
    super(ModelDeep, self).__init__()
    self.dense1 = tf.keras.layers.Dense(1000, activation=tf.nn.relu)
    self.dense2 = tf.keras.layers.Dense(2000, activation=tf.nn.relu)
    self.dense3 = tf.keras.layers.Dense(3000, activation=tf.nn.softmax)
    self.dropout = tf.keras.layers.Dropout(0.5)

  def call(self, inputs, training=False):
    x = self.dense1(inputs)
    if training:
      x = self.dropout(x, training=training)
    x = self.dense2(x)
    out = self.dense3(x)
    return out

# Create the model with eager esxecution by default
model_shallow_with_eager = ModelShallow()

# Take model to graph. 
# NOTE: Instead of using decorators, we can ditectly operate tf.function on the model.
model_shallow_on_graph = tf.function(ModelShallow())

# Model deep
model_deep_with_eager = ModelDeep()
model_deep_on_graph = tf.function(ModelDeep())

# sample input
sample_input = tf.random.uniform([60, 28, 28])

# Check time for shallow model
print("Shallow Model - Eager execution time:", timeit.timeit(lambda: model_shallow_with_eager(sample_input), number=1000))
print("Shallow Model - Graph-based execution time:", timeit.timeit(lambda: model_shallow_on_graph(sample_input), number=1000))

# Check time for deep model
print("Deep Model - Eager execution time:", timeit.timeit(lambda: model_deep_with_eager(sample_input), number=1000))
print("Deep Model - Graph-based execution time:", timeit.timeit(lambda: model_deep_on_graph(sample_input), number=1000))

# Output
'''
Shallow Model - Eager execution time: 4.13806810799997
Shallow Model - Graph-based execution time: 1.575126482000087
Deep Model - Eager execution time: 517.5000654989999
Deep Model - Graph-based execution time: 510.01053104599987
'''

明显能发现，使用快速方法要比使用图方法速度慢，耗时长，并且这个差异却随着计算量减小而扩大。原因是，eager 模式就是类似于 python 这样的命令式编程，写好程序之后，不需要编译，就可以直接运行了，而且非常直观；而之前的静态图模式则类似于 c/c++ 的声明式编程。写好程序之后要先编译，然后才能运行。