Single Shot MultiBox Detector（MXNet）源码阅读笔记（2）

最新推荐文章于 2020-07-17 22:10:24 发布

Ellin_

最新推荐文章于 2020-07-17 22:10:24 发布

阅读量709

点赞数

分类专栏： SSD 文章标签：阅读

本文链接：https://blog.csdn.net/Ellin_/article/details/56009440

版权

SSD 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

这篇博客详细介绍了Single Shot MultiBox Detector（SSD）在MXNet框架中使用Xavier方法进行参数初始化的过程，阐述了Xavier初始化在卷积层应用的重要性。

摘要由CSDN通过智能技术生成

Single Shot MultiBox Detector（MXNet）源码阅读笔记（2）

初始化的方法Xavier

We initialize the parameters for all the newly added convolutional layers with the “xavier” method.

论文中所述，我们对新增加的卷积层使用“xavier”的方法进行初始化。
Xavier Initialization很简单，其实就是

V a r (W i) = 2 n i n + n o u t

$Var(W_i) = \dfrac{2}{n_{in}+n_{out}}$

nin $n_{in}$ -输入神经元的数量

nout $n_{out}$ -输出神经元的数量
则

s c a l e = m a g n i t u d e V a r ( W i ) - - - - - - - - - - \sqrt

$scale=\sqrt{\dfrac{magnitude}{Var(W_i)}}$
具体解释见 andy’sblog
原理出处见论文 Understanding the difficulty of training deep feedforward neural networks
MXnet中代码 initializer.py部分

class Xavier(Initializer):
    """Initialize the weight with Xavier or similar initialization scheme.
    Parameters
    ----------
    rnd_type: str, optional
        Use ```gaussian``` or ```uniform``` to init
    factor_type: str, optional
        Use ```avg```, ```in```, or ```out``` to init
    magnitude: float, optional
        scale of random number range
    """
    def __init__(self, rnd_type="uniform", factor_type="avg", magnitude=3):
        super(Xavier, self).__init__(rnd_type=rnd_type, factor_type=factor_type,magnitude=magnitude)
        self.rnd_type = rnd_type
        self.factor_type = factor_type
        self.magnitude = float(magnitude)


    def _init_weight(self, _, arr):
        shape = arr.shape
        hw_scale = 1.
        if len(shape) > 2:
            hw_scale = np.prod(shape[2:])
        fan_in, fan_out = shape[1] * hw_scale, shape[0] * hw_scale
        factor = 1.
        if self.factor_type == "avg":
            factor = (fan_in + fan_out) / 2.0
            # 计算方差
        elif self.factor_type == "in":
            factor = fan_in
        elif self.factor_type == "out":
            factor = fan_out
        else:
            raise ValueError("Incorrect factor type")
        scale = np.sqrt(self.magnitude / factor)
        # 计算范围
        if self.rnd_type == "uniform":
            random.uniform(-scale, scale, out=arr)
        elif self.rnd_type == "gaussian":
            random.normal(0, scale, out=arr)
        else:
            raise ValueError("Unknown random type")