tf.clip_by_global_norm函数解析

#coding=utf-8
import tensorflow as tf
import numpy as np
def test_clip_by_global_norm():
    x = [1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0]
    r, use_norm = tf.clip_by_global_norm(x, 10)
    print(r,use_norm)
    #--------use_norm----------
    x_l2 = np.array(x)**2
    print("self use_norm", np.sqrt(sum(x_l2)))
    
if __name__ == '__main__':
    test_clip_by_global_norm()

clip_norm: 10 
use_norm: tf.Tensor(14.282857, shape=(), dtype=float32) 
1.0 / use_norm: tf.Tensor(0.07001401, shape=(), dtype=float32) 
1 / clip_norm: 0.1 
scale_for_finite: tf.Tensor(0.70014006, shape=(), dtype=float32) 
scale: tf.Tensor(0.70014006, shape=(), dtype=float32)
[<tf.Tensor: shape=(), dtype=float32, numpy=0.70014006>, <tf.Tensor: shape=(), dtype=float32, numpy=1.4002801>, <tf.Tensor: shape=(), dtype=float32, numpy=2.1004202>, <tf.Tensor: shape=(), dtype=float32, numpy=2.8005602>, <tf.Tensor: shape=(), dtype=float32, numpy=3.5007002>, <tf.Tensor: shape=(), dtype=float32, numpy=4.2008405>, <tf.Tensor: shape=(), dtype=float32, numpy=4.9009805>, <tf.Tensor: shape=(), dtype=float32, numpy=5.6011205>]
tf.Tensor(14.282857, shape=(), dtype=float32)
self use_norm 14.2828568570857

代码执行过程：如果没有user_norm则使用 use_norm = global_norm(t_list, name) 这个函数指定，它指的是 梯度的均方和 即math_ops.reduce_sum(array_ops.stack(half_squared_norms))，
然后取scale = clip_norm*min(1/use_norm, 1/clip_norm)，作为系数，对梯度的每一个分量进行缩放
tf源码：

tf.clip_by_global_norm的理解
tf.clip_by_global_norm()
tf.clip_by_global_norm详解

tf.clip_by_global_norm理解