tensorflow ：GraphKeys.REGULARIZATION_LOSSES

tensorflow Regularizers

tensorflow中对参数使用正则项分为两步:
1. 创建一个正则方法(函数/对象)
2. 将这个正则方法(函数/对象),应用到参数上

参考：https://blog.csdn.net/u012436149/article/details/70264257

• 就下面的这里改一下，举个例子：

在使用tf.get_variable()和tf.variable_scope()的时候,你会发现,它们俩中有regularizer形参.如果传入这个参数的话,那么variable_scope内的weights的正则化损失,会被添加到GraphKeys.REGULARIZATION_LOSSES中.

import tensorflow as tf
from tensorflow.contrib import layers
reset_graph()
regularizer = layers.l1_regularizer(0.1)

#这里设置了变量空间variable_scope的regularizer参数,结果名称下的变量都默认正则化；
with tf.variable_scope('var', initializer=tf.random_normal_initializer(), 
regularizer=regularizer):
    weight = tf.get_variable('weight', shape=[8], initializer=tf.ones_initializer())
#这里没有设置变量空间variable_scope的regularizer参数,结果名称下的变量不会默认被正则化；
with tf.variable_scope('var2', initializer=tf.random_normal_initializer()):
    weight2 = tf.get_variable('weight', shape=[8], initializer=tf.ones_initializer())

regularization_loss = tf.reduce_sum(tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES))
with tf.Session() as sess:
    print(tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES))

结果验证：

[<tf.Tensor 'var/weight/Regularizer/l1_regularizer:0' shape=() dtype=float32>]

weight2 并没有在tf.GraphKeys.REGULARIZATION_LOSSES中。

如果修改这一行：

    weight2 = tf.get_variable('weight', shape=[8], regularizer=regularizer,initializer=tf.ones_initializer())

加入正则化参数中：

[<tf.Tensor 'var/weight/Regularizer/l1_regularizer:0' shape=() dtype=float32>, <tf.Tensor 'var2/weight/Regularizer/l1_regularizer:0' shape=() dtype=float32>]