tf.constrib.slim计算损失函数

Tensorflow的模型训练需要模型，损失函数，梯度计算，以及根据loss的梯度迭代更新参数,使用tf.constrib.slim很方便。
（1）losses
使用现有的loss：

loss = slim.losses.softmax_cross_entropy(predictions, labels)
对于多任务学习的loss，可以使用：

# Define the loss functions and get the total loss.
classification_loss = slim.losses.softmax_cross_entropy(scene_predictions, scene_labels)
sum_of_squares_loss = slim.losses.sum_of_squares(depth_predictions, depth_labels)

# The following two lines have the same effect: total_loss = classification_loss + sum_of_squares_loss total_loss = slim.losses.get_total_loss(add_regularization_losses=False) 如果使用了自己定义的loss，而又想使用slim的loss管理机制，可以使用： 

pose_loss = MyCustomLossFunction(pose_predictions, pose_labels)
slim.losses.add_loss(pose_loss) 
total_loss = slim.losses.get_total_loss()
＃total_loss中包涵了pose_loss
(2) 训练循环
slim在learning.py中提供了一个简单而有用的训练模型的工具。我们只需调用slim.learning.create_train_op 和slim.learning.train就可以完成优化过程。

g = tf.Graph()

# Create the model and specify the losses...
... total_loss = slim.losses.get_total_loss() optimizer = tf.train.GradientDescentOptimizer(learning_rate) # create_train_op ensures that each time we ask for the loss, the update_ops # are run and the gradients being computed are applied too. train_op = slim.learning.create_train_op(total_loss, optimizer) logdir = ... # Where checkpoints are stored. slim.learning.train( train_op, logdir, number_of_steps=1000,＃迭代次数 save_summaries_secs=300,＃存summary间隔秒数 save_interval_secs=600)＃存模型建个秒数 （3）训练的例子：

import tensorflow as tf

slim = tf.contrib.slim
vgg = tf.contrib.slim.nets.vgg

...

train_log_dir = ... if not tf.gfile.Exists(train_log_dir): tf.gfile.MakeDirs(train_log_dir) with tf.Graph().as_default(): # Set up the data loading: images, labels = ... # Define the model: predictions = vgg.vgg16(images, is_training=True) # Specify the loss function: slim.losses.softmax_cross_entropy(predictions, labels) total_loss = slim.losses.get_total_loss() tf.summary.scalar('losses/total_loss', total_loss) # Specify the optimization scheme: optimizer = tf.train.GradientDescentOptimizer(learning_rate=.001) # create_train_op that ensures that when we evaluate it to get the loss, # the update_ops are done and the gradient updates are computed. train_tensor = slim.learning.create_train_op(total_loss, optimizer) # Actually runs training. slim.learning.train(train_tensor, train_log_dir)
5. 根据已有模型进行微调
（1）利用tf.train.Saver()从checkpoint恢复模型

# Create some variables.
v1 = tf.Variable(..., name="v1") v2 = tf.Variable(..., name="v2") ... # Add ops to restore all the variables. restorer = tf.train.Saver() # Add ops to restore some variables. restorer = tf.train.Saver([v1, v2]) # Later, launch the model, use the saver to restore variables from disk, and # do some work with the model. with tf.Session() as sess: # Restore variables from disk. restorer.restore(sess, "/tmp/model.ckpt") print("Model restored.") # Do some work with the model ...
（2）部分恢复模型参数

# Create some variables.
v1 = slim.variable(name="v1", ...) v2 = slim.variable(name="nested/v2", ...) ... # Get list of variables to restore (which contains only 'v2'). These are all # equivalent methods: variables_to_restore = slim.get_variables_by_name("v2") # or variables_to_restore = slim.get_variables_by_suffix("2") # or variables_to_restore = slim.get_variables(scope="nested") # or variables_to_restore = slim.get_variables_to_restore(include=["nested"]) # or variables_to_restore = slim.get_variables_to_restore(exclude=["v1"]) # Create the saver which will be used to restore the variables. restorer = tf.train.Saver(variables_to_restore) with tf.Session() as sess: # Restore variables from disk. restorer.restore(sess, "/tmp/model.ckpt") print("Model restored.") # Do some work with the model ...
（3）当图的变量名与checkpoint中的变量名不同时，恢复模型参数
    当从checkpoint文件中恢复变量时，Saver在checkpoint文件中定位到变量名，并且把它们映射到当前图中的变量中。之前的例子中，我们创建了Saver，并为其提供了变量列表作为参数。这时，在checkpoint文件中定位的变量名，是隐含地从每个作为参数给出的变量的var.op.name而获得的。这一方式在图与checkpoint文件中变量名字相同时，可以很好的工作。而当名字不同时，必须给Saver提供一个将checkpoint文件中的变量名映射到图中的每个变量的字典，例子见下：

# Assuming that 'conv1/weights' should be restored from 'vgg16/conv1/weights'
def name_in_checkpoint(var): return 'vgg16/' + var.op.name # Assuming that 'conv1/weights' and 'conv1/bias' should be restored from 'conv1/params1' and 'conv1/params2' def name_in_checkpoint(var): if "weights" in var.op.name: return var.op.name.replace("weights", "params1") if "bias" in var.op.name: return var.op.name.replace("bias", "params2") variables_to_restore = slim.get_model_variables() variables_to_restore = {name_in_checkpoint(var):var for var in variables_to_restore} restorer = tf.train.Saver(variables_to_restore) with tf.Session() as sess: # Restore variables from disk. restorer.restore(sess, "/tmp/model.ckpt") （4）在一个不同的任务上对网络进行微调 比如我们要将1000类的imagenet分类任务应用于20类的Pascal VOC分类任务中，我们只导入部分层，见下例： 

image, label = MyPascalVocDataLoader(...)
images, labels = tf.train.batch([image, label], batch_size=32) # Create the model predictions = vgg.vgg_16(images) train_op = slim.learning.create_train_op(...) # Specify where the Model, trained on ImageNet, was saved. model_path = '/path/to/pre_trained_on_imagenet.checkpoint' # Specify where the new model will live: log_dir = '/path/to/my_pascal_model_dir/' # Restore only the convolutional layers: variables_to_restore = slim.get_variables_to_restore(exclude=['fc6', 'fc7', 'fc8']) init_fn = assign_from_checkpoint_fn(model_path, variables_to_restore) # Start training. slim.learning.train(train_op, log_dir, init_fn=init_fn)