tensorflow:Scale of 0 disables regularizer原因
- 在运行tensorflow时候,遇到日志输出:
INFO:tensorflow:Scale of 0 disables regularizer.
开始以为是哪里出错了,仔细检查了代码后发现,原因是:使用tf.contrib.layers.l2_regularizer
时候将scale
参数,也就是l2正则强度设为了0.0,看以下源码:
def l2_regularizer(scale, scope=None):
"""Returns a function that can be used to apply L2 regularization to weights.
Small values of L2 can help prevent overfitting the training data.
Args:
scale: A scalar multiplier `Tensor`. 0.0 disables the regularizer.
scope: An optional scope name.
Returns:
A function with signature `l2(weights)` that applies L2 regularization.
Raises:
ValueError: If scale is negative or if scale is not a float.
"""
if isinstance(scale, numbers.Integral):
raise ValueError('scale cannot be an integer: %s' % (scale,))
if isinstance(scale, numbers.Real):
if scale < 0.:
raise ValueError('Setting a scale less than 0 on a regularizer: %g.' %
scale)
if scale == 0.:
logging.info('Scale of 0 disables regularizer.')
return lambda _: None
def l2(weights):
"""Applies l2 regularization to weights."""
with ops.name_scope(scope, 'l2_regularizer', [weights]) as name:
my_scale = ops.convert_to_tensor(scale,
dtype=weights.dtype.base_dtype,
name='scale')
return standard_ops.multiply(my_scale, nn.l2_loss(weights), name=name)
return l2
可见最终原因就是这个。
- 解决方案:
from tensorflow.contrib.layers import l2_regularizer
l2_reg = l2_regularizer(0.001)
也就是将正则系数设为大于0的数,或者不需要正则化的时候取消正则操作。