Tensorflow (2): tf.slim库解读

官方文档: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/slim
tf.contrib.slim是一个轻量级的库,目的是方便地定义, 训练和评估tensorflow中复杂的模型
版本: 2018.6.14

主要作用

• 对于一些固定的代码进行封装,主要是一些高级的层和变量,方便用户写出一些更加紧凑的代码
• 包含多个广泛使用的CV模型(VGG, AlexNet)
• 提供训练(损失,学习), 评估一些高级方法

主要组成

标注[o]的将进行介绍,标注[x]的官方暂时没有文档

• [o]variables: provides convenience wrappers for variable creation and manipulation.
• [o]layers: contains high level layers for building models using tensorflow.
• [o]arg_scopes
• [o]losses: contains commonly used loss functions.
• [o]learning
• [o]metrics: contains popular evaluation metrics.
• [o]evaluation
• [o]data: contains TF-slim’s dataset definition, data providers, parallel_reader, and decoding utilities.
• [x]nets: contains popular network definitions such as VGG and AlexNet models.
• [x]queues: provides a context manager for easily and safely starting and closing QueueRunners.
• [x]regularizers: contains weight regularizers.

定义模型

Variable

1. 封装variable
   举例:

weights = slim.variable('weights',
                             shape=[10, 10, 3 , 3],
                             initializer=tf.truncated_normal_initializer(stddev=0.1),
                             regularizer=slim.l2_regularizer(0.05),
                             device='/CPU:0')

1. 管理model variable
   在原生的tf中,有两种变量:regular variable和local variable,第一种就是就是可以使用saver进行save的变量,第二中是只存在于session中,不能saved的变量.
   在slim中,区分变量为两种类型: model variable和non-model variable.mdoel variable就是可以学习的参数,并且在评估或预测的时候需要加载的参数,例如 slim.fully_connected or slim.conv2d层的参数,non-model variable就是在训练或评估的时候需要,但是在inference的时候不需要,例如global_step.
   举例:

# Model Variables
weights = slim.model_variable('weights',
                              shape=[10, 10, 3 , 3],
                              initializer=tf.truncated_normal_initializer(stddev=0.1),
                              regularizer=slim.l2_regularizer(0.05),
                              device='/CPU:0')
model_variables = slim.get_model_variables()

# Regular variables
my_var = slim.variable('my_var',
                       shape=[20, 1],
                       initializer=tf.zeros_initializer())
regular_variables_and_model_variables = slim.get_variables()

slim.get_model_variables()工作原理: 当使用slim创建一个model_varialbe的时候, slim会把这个变量添加到tf.GraphKeys.MODEL_VARIABLES collection.如果是自己创建的变量想要被slim进行管理,可以采用如下方式:

my_model_variable = CreateViaCustomCode()
# Letting TF-Slim know about the additional variable.
slim.add_model_variable(my_model_variable)

Layer

slim封装层
原生的tf创建一个卷积层,需要如下多个低级操作:
* 创建weight和bias variable
* convolve the weight with last tensor
* add the baise to last result
* activate
举例:

input = ...
with tf.name_scope('conv1_1') as scope:
  kernel = tf.Variable(tf.truncated_normal([3, 3, 64, 128], dtype=tf.float32,
                                           stddev=1e-1), name='weights')
  conv = tf.nn.conv2d(input, kernel, [1, 1, 1, 1], padding='SAME')
  biases = tf.Variable(tf.constant(0.0, shape=[128], dtype=tf.float32),
                       trainable=True, name='biases')
  bias = tf.nn.bias_add(conv, biases)
  conv1 = tf.nn.relu(bias, name=scope)

slim封装了一个简单的上边代码的替代品:

input = ...
net = slim.conv2d(input, 128, [3, 3], scope='conv1_1')

还封装常用的其它层: slim.batch_norm, slim.fully_connected等…

Scope

为了使创建的计算图更加模块化和方便管理,引入了Scope的概念,来对变量通过增加前缀进行划分.原生的tf提供了name_scope和variable_scope.
区别在于: 使用tf.Variable()的时候，tf.name_scope()和tf.variable_scope() 都会给 Variable 和