-
无偏差的加入噪音
对 x \mathbf{x} x加入噪音得到 x ′ \mathbf{x}^{\prime} x′,我们希望
E [ x ′ ] = x \mathbf{E}\left[\mathbf{x}^{\prime}\right]=\mathbf{x} E[x′]=x
丢弃法对每个元素进行如下扰动
x i ′ = { 0 with probablity p x i 1 − p otherise x_{i}^{\prime}= \begin{cases}0 & \text { with probablity } p \\ \frac{x_{i}}{1-p} & \text { otherise }\end{cases} xi′={01−pxi with probablity p otherise -
通常将丢弃法作用在隐藏全连接层的输出上。
-
实现
dropout_layer
函数import torch from torch import nn def dropout_layer(X, dropout): assert 0 <= dropout <= 1 if dropout == 1: return torch.zeros_like(X) if dropout == 0: return X mask = (torch.randn(X.shape) > dropout).float() return mask * X / (1.0 - dropout)
13 丢弃法 [动手学深度学习v2]
最新推荐文章于 2024-09-09 09:47:53 发布