tf.contrib.slim学习之Training Models

TF-slim提供了一系列的常用损失函数和其他便捷的辅助函数去帮助执行训练和评估过程；

一 Training Models

(1)TF-slim定义loss函数

TF-slim预定义了一些常用的损失函数，如交叉熵损失：

import tensorflow as tf
import tensorflow.contrib.slim.nets as nets
vgg = nets.vgg

# Load the images and labels.
images, labels = ...

# Create the model.
predictions, _ = vgg.vgg_16(images)

# Define the loss functions and get the total loss.
loss = slim.losses.softmax_cross_entropy(predictions, labels)

使用slim.loss.定义的损失函数会被默认的加入到loss-function的集合中，使用slim.losses.get_total_loss(add_regularization_losses=False)会自动的计算multi-task的multi-loss的总和；下面两种方式计算总损失的效果是一样的；

# Load the images and labels.
images, scene_labels, depth_labels = ...

# Create the model.
scene_predictions, depth_predictions = CreateMultiTaskModel(images)

# Define the loss functions and get the total loss.
classification_loss = slim.losses.softmax_cross_entropy(scene_predictions, scene_labels)
sum_of_squares_loss = slim.losses.sum_of_squares(depth_predictions, depth_labels)

# The following two lines have the same effect:
total_loss = classification_loss + sum_of_squares_loss
total_loss = slim.losses.get_total_loss(add_regularization_losses=False)

 如果有自己定义的损失函数，也可以手动的将这个损失加入到loss-function集合中,方便slim集中管理loss

# Load the images and labels.
images, scene_labels, depth_labels, pose_labels = ...

# Create the model.
scene_predictions, depth_predictions, pose_predictions = CreateMultiTaskModel(images)

# Define the loss functions and get the total loss.
classification_loss = slim.losses.softmax_cross_entropy(scene_predictions, scene_labels)
sum_of_squares_loss = slim.losses.sum_of_squares(depth_predictions, depth_labels)
pose_loss = MyCustomLossFunction(pose_predictions, pose_labels)
slim.losses.add_loss(pose_loss) # Letting TF-Slim know about the additional loss.

# The following two ways to compute the total loss are equivalent:
regularization_loss = tf.add_n(slim.losses.get_regularization_losses())
total_loss1 = classification_loss + sum_of_squares_loss + pose_loss + regularization_loss

# (Regularization Loss is included in the total loss by default).
total_loss2 = slim.losses.get_total_loss()

tf.add_n(slim.losses.get_regularization_losses() )得到的正则化损失的总和；

（2）循环训练Training

TF-slim提供了一套简单但功能强大的工具用于训练，它们包含了反复计算losses,计算梯度，保存模型到磁盘，和一些方便的操控梯度的函数的Train函数，一旦指定了模型，损失，和优化方案后，可以直接调用slim.learning.create_train_op和slim.learning.train来执行优化过程。

g = tf.Graph()

# Create the model and specify the losses...
...

total_loss = slim.losses.get_total_loss()
optimizer = tf.train.GradientDescentOptimizer(learning_rate)

# create_train_op ensures that each time we ask for the loss, the update_ops
# are run and the gradients being computed are applied too.
train_op = slim.learning.create_train_op(total_loss, optimizer)
logdir = ... # Where checkpoints are stored.

slim.learning.train(
    train_op,
    logdir,
    number_of_steps=1000,
    save_summaries_secs=300,
    save_interval_secs=600):

 slim.learning.train用提供的train_op计算loss,并将梯度应用到weights;

相当于用slim.learning.train()来统一管理训练过程；

1. 定义网络模型，损失函数，和优化方案
2. 用损失函数和优化方案创建训练操作train_op；在每次训练的过程中，train_op主要有两个任务①计算loss②将计算出的loss按优化方案定义的方式去更新权重
3. 将train_op提供给slim.learning.train()；slim.training.train()会反复的执行train_op直达max_steps;并且会自动的保存checkpoint,summaries;

(3)训练VGG16模型的例子

import tensorflow as tf
import tensorflow.contrib.slim.nets as nets

slim = tf.contrib.slim
vgg = nets.vgg

...

train_log_dir = ...
if not tf.gfile.Exists(train_log_dir):
  tf.gfile.MakeDirs(train_log_dir)

with tf.Graph().as_default():
  # Set up the data loading:
  images, labels = ...

  # Define the model:
  predictions = vgg.vgg_16(images, is_training=True)

  # Specify the loss function:
  slim.losses.softmax_cross_entropy(predictions, labels)

  total_loss = slim.losses.get_total_loss()
  tf.summary.scalar('losses/total_loss', total_loss)

  # Specify the optimization scheme:
  optimizer = tf.train.GradientDescentOptimizer(learning_rate=.001)

  # create_train_op that ensures that when we evaluate it to get the loss,
  # the update_ops are done and the gradient updates are computed.
  train_tensor = slim.learning.create_train_op(total_loss, optimizer)

  # Actually runs training.
  slim.learning.train(train_tensor, train_log_dir)