bn层初始化参数_Day208:BN层代码解析

BN层介绍

代码解析

input_data = mx.nd.arange(1,9).reshape((1,2,2,2))print(input_data)            # [[[[1. 2.]            #    [3. 4.]]            #            #   [[5. 6.]            #    [7. 8.]]]]            # gamma = mx.nd.ones(2)   # 初始化BN层的γ参数，为了计算方便，将其初始化为1print(gamma)            # [1. 1.]            # beta = mx.nd.ones(2)    # 初始化BN层的β参数，为了计算方便，将其初始化为1print(beta)            # [1. 1.]            # moving_mean = mx.nd.ones(2)*3   # 初始化全局均值print(moving_mean)            # [3. 3.]            # moving_var = mx.nd.ones(2)*2    # 初始化全局方差print(moving_var)            # [2. 2.]            # out_data = mx.nd.BatchNorm(data=input_data, gamma=gamma, beta=beta, moving_mean=moving_mean, moving_var=moving_var,                           momentum=0.9, fix_gamma=1, use_global_stats=1)print(out_data)            # [[[[-0.4138602  0.2930699]            #    [ 1.         1.7069302]]            #            #   [[ 2.4138603  3.1207902]            #    [ 3.8277204  4.534651 ]]]]            # 

L2Normalization

• L2 Norma

• L2 Normalize

L2归一化，是对单个向量的各个元素做归一化的手段，使得向量x变换后的结果x'的L2 norm为1