torch、(三) Random sampling

这篇博客介绍了PyTorch中用于随机采样的各种函数,包括设置随机种子、均匀分布、正态分布、多模分布以及准随机序列的SobolEngine。这些函数可用于生成不同分布的随机数,对于训练深度学习和机器学习模型至关重要。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

参考  torch、(三) Random sampling - 云+社区 - 腾讯云

目录

torch.seed()[source]

torch.manual_seed(seed)[source]

torch.initial_seed()[source]

torch.get_rng_state()[source]

torch.set_rng_state(new_state)[source]

torch.default_generator Returns the default CPU torch.Generator

torch.bernoulli(input, *, generator=None, out=None) → Tensor

torch.multinomial(input, num_samples, replacement=False, out=None) → LongTensor

torch.normal()

torch.normal(mean=0.0, std, out=None) → Tensor

torch.normal(mean, std=1.0, out=None) → Tensor

torch.normal(mean, std, size, *, out=None) → Tensor

torch.rand(*size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor

torch.rand_like(input, dtype=None, layout=None, device=None, requires_grad=False) → Tensor

torch.randint(low=0, high, size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor

torch.randint_like(input, low=0, high, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor

torch.randn(*size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor

torch.randn_like(input, dtype=None, layout=None, device=None, requires_grad=False) → Tensor

torch.randperm(n, out=None, dtype=torch.int64, layout=torch.strided, device=None, requires_grad=False) → LongTensor

In-place random sampling

Quasi-random sampling

class torch.quasirandom.SobolEngine(dimension, scramble=False, seed=None)[source]

draw(n=1, out=None, dtype=torch.float32)[source]

fast_forward(n)[source]

reset()[source]


torch.seed()[source]

Sets the seed for generating random numbers to a non-deterministic random number. Returns a 64 bit number used to seed the RNG.

torch.manual_seed(seed)[source]

Sets the seed for generating random numbers. Returns a torch.Generator object.

Parameters

seed (int) – The desired seed.

torch.initial_seed()[source]

Returns the initial seed for generating random numbers as a Python long.

torch.get_rng_state()[source]

Returns the random number generator state as a torch.ByteTensor.

torch.set_rng_state(new_state)[source]

Sets the random number generator state.

Parameters

new_state (torch.ByteTensor) – The desired state

torch.default_generator Returns the default CPU torch.Generator

torch.bernoulli(input, *, generator=None, out=None) → Tensor

Draws binary random numbers (0 or 1) from a Bernoulli distribution.

The input tensor should be a tensor containing probabilities to be used for drawing the binary random number. Hence, all values in input have to be in the range: 0≤inputi≤10 \leq \text{input}_i \leq 10≤inputi​≤1 .

The ith\text{i}^{th}ith element of the output tensor will draw a value 111 according to the ith\text{i}^{th}ith probability value given in input.

outi∼Bernoulli(p=inputi)\text{out}_{i} \sim \mathrm{Bernoulli}(p = \text{input}_{i}) outi​∼Bernoulli(p=inputi​)

The returned out tensor only has values 0 or 1 and is of the same shape as input.

out can have integral dtype, but input must have floating point dtype.

Parameters

  • input (Tensor) – the input tensor of probability values for the Bernoulli distribution

  • out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.empty(3, 3).uniform_(0, 1)  # generate a uniform random matrix with range [0, 1]
>>> a
tensor([[ 0.1737,  0.0950,  0.3609],
        [ 0.7148,  0.0289,  0.2676],
        [ 0.9456,  0.8937,  0.7202]])
>>> torch.bernoulli(a)
tensor([[ 1.,  0.,  0.],
        [ 0.,  0.,  0.],
        [ 1.,  1.,  1.]])

>>> a = torch.ones(3, 3) # probability of drawing "1" is 1
>>> torch.bernoulli(a)
tensor([[ 1.,  1.,  1.],
        [ 1.,  1.,  1.],
        [ 1.,  1.,  1.]])
>>> a = torch.zeros(3, 3) # probability of drawing "1" is 0
>>> torch.bernoulli(a)
tensor([[ 0.,  0.,  0.],
        [ 0.,  0.,  0.],
        [ 0.,  0.,  0.]])

torch.multinomial(input, num_samples, replacement=False, out=None) → LongTensor

Returns a tensor where each row contains num_samples indices sampled from the multinomial probability distribution located in the corresponding row of tensor input.

Note

The rows of input do not need to sum to one (in which case we use the values as weights), but must be non-negative, finite and have a non-zero sum.

Indices are ordered from left to right according to when each was sampled (first samples are placed in first column).

If input is a vector, out is a vector of size num_samples.

If input is a matrix with m rows, out is an matrix of shape (m×num_samples)(m \times \text{num\_samples})(m×num_samples) .

If replacement is True, samples are drawn with replacement.

If not, they are drawn without replacement, which means that when a sample index is drawn for a row, it cannot be drawn again for that row.

Note

When drawn without replacement, num_samples must be lower than number of non-zero elements in input (or the min number of non-zero elements in each row of input if it is a matrix).

Parameters

  • input (Tensor) – the input tensor containing probabilities

  • num_samples (int) – number of samples to draw

  • replacement (bool, optional) – whether to draw with replacement or not

  • out (Tensor, optional) – the output tensor.

Example:

>>> weights = torch.tensor([0, 10, 3, 0], dtype=torch.float) # create a tensor of weights
>>> torch.multinomial(weights, 2)
tensor([1, 2])
>>> torch.multinomial(weights, 4) # ERROR!
RuntimeError: invalid argument 2: invalid multinomial distribution (with replacement=False,
not enough non-negative category to sample) at ../aten/src/TH/generic/THTensorRandom.cpp:320
>>> torch.multinomial(weights, 4, replacement=True)
tensor([ 2,  1,  1,  1])

torch.normal()

torch.normal(mean, std, out=None) → Tensor

Returns a tensor of random numbers drawn from separate normal distributions whose mean and standard deviation are given.

The mean is a tensor with the mean of each output element’s normal distribution

The std is a tensor with the standard deviation of each output element’s normal distribution

The shapes of mean and std don’t need to match, but the total number of elements in each tensor need to be the same.

Note

When the shapes do not match, the shape of mean is used as the shape for the returned output tensor

Parameters

  • mean (Tensor) – the tensor of per-element means

  • std (

``` # Compute padding. pad_total = (self.out_size - 1) * self.down_factor + 1 # Desired output size before downsampling. pad_total -= (self.in_size + self.conv_kernel - 1) * self.up_factor # Input size after upsampling. pad_total += self.up_taps + self.down_taps - 2 # Size reduction caused by the filters. pad_lo = (pad_total + self.up_factor) // 2 # Shift sample locations according to the symmetric interpretation (Appendix C.3). pad_hi = pad_total - pad_lo self.padding = [int(pad_lo[0]), int(pad_hi[0]), int(pad_lo[1]), int(pad_hi[1])] def forward(self, x, w, noise_mode='random', force_fp32=False, update_emas=False): assert noise_mode in ['random', 'const', 'none'] # unused misc.assert_shape(x, [None, self.in_channels, int(self.in_size[1]), int(self.in_size[0])]) misc.assert_shape(w, [x.shape[0], self.w_dim]) # Track input magnitude. if update_emas: with torch.autograd.profiler.record_function('update_magnitude_ema'): magnitude_cur = x.detach().to(torch.float32).square().mean() self.magnitude_ema.copy_(magnitude_cur.lerp(self.magnitude_ema, self.magnitude_ema_beta)) input_gain = self.magnitude_ema.rsqrt() # Execute affine layer. styles = self.affine(w) if self.is_torgb: weight_gain = 1 / np.sqrt(self.in_channels * (self.conv_kernel ** 2)) styles = styles * weight_gain # Execute modulated conv2d. dtype = torch.float16 if (self.use_fp16 and not force_fp32 and x.device.type == 'cuda') else torch.float32 x = modulated_conv2d(x=x.to(dtype), w=self.weight, s=styles, padding=self.conv_kernel-1, demodulate=(not self.is_torgb), input_gain=input_gain) # Execute bias, filtered leaky ReLU, and clamping. gain = 1 if self.is_torgb else np.sqrt(2) slope = 1 if self.is_torgb else 0.2 x = filtered_lrelu.filtered_lrelu(x=x, fu=self.up_filter, fd=self.down_filter, b=self.bias.to(x.dtype), up=self.up_factor, down=self.down_factor, padding=self.padding, gain=gain, slope=slope, clamp=self.conv_clamp) # Ensure correct shape and dtype. misc.assert_shape(x, [None, self.out_channels, int(self.out_size[1]), int(self.out_size[0])]) assert x.dtype == dtype return x```解释这段代码
最新发布
03-11
<think>好的,我现在要解释用户提供的这段代码。首先,我需要整体看一下代码的结构和功能。看起来这段代码是PyTorch的一个自定义模块,可能用于生成对抗网络(GAN)中的上采样或下采样操作,特别是StyleGAN之类的模型,因为里面有提到modulated_conv2d和affine层,这些在StyleGAN中常见。 首先,代码分为两部分:padding的计算和forward方法。我需要分别理解这两个部分的作用。 关于padding的计算部分,用户给出了一个注释,说明这是在计算填充量。首先,pad_total的计算涉及输出尺寸、输入尺寸、卷积核大小、上采样和下采样因子,以及滤波器的tap数。这看起来像是在根据输入输出尺寸和上下采样的需求,计算需要填充的总量,然后分配到pad_lo和pad_hi,这可能是在左右或上下两侧分配填充量,保持对称。这部分可能需要进一步分析每个步骤的数学推导。 然后是forward方法,参数包括输入x、样式w、噪声模式等。在forward中,首先进行了一些断言检查,确保输入的尺寸正确。接着,如果update_emas为真,会更新magnitude_ema,这可能是在跟踪输入的大小,用于归一化。然后计算input_gain,作为EMA的逆平方根,用于后续调整输入幅度。 执行affine层,将w转换为styles。如果是toRGB层(即生成RGB图像的最后层),会应用权重增益。然后执行modulated_conv2d,这里使用了调制卷积,这可能是指StyleGAN2中的权重调制机制。数据类型可能根据是否使用fp16来决定。 接下来是偏置、过滤的leaky ReLU和钳位操作。这里使用了filtered_lrelu,这可能是一种结合上采样和下采样滤波器的激活函数,带有增益和斜率参数。最后确保输出的形状和数据类型正确。 现在需要详细解释每个步骤,尤其是padding的计算部分。可能需要用公式表示各个步骤,比如pad_total的计算。例如: pad_total = (out_size -1)*down_factor +1 - (in_size + conv_kernel -1)*up_factor + up_taps + down_taps -2 这可能涉及到确保在经过上采样、卷积、下采样等操作后,输出尺寸正确,因此需要计算所需的填充量。这里的每个项可能需要更详细的解释,例如为什么这样组合。 在forward过程中,modulated_conv2d是StyleGAN2中的关键部分,它根据样式向量调整卷积权重。filtered_lrelu可能结合了上采样或下采样的滤波操作,比如在激活时进行抗锯齿处理。 需要注意用户提到的系统级指令,比如数学表达式需要用$...$,独立公式用$$...$$,并且用中文回答,结构清晰。需要确保解释准确,不遗漏关键点,如EMA的作用、affine变换、调制卷积、滤波激活等。 可能的疑问点:pad_total的计算是否考虑了上采样和下采样的步长?如何保证最终的输出尺寸正确?up_taps和down_taps具体指什么?这部分可能需要进一步查阅相关论文或代码注释,但根据当前代码,假设这些参数是滤波器的大小或抽头数,影响填充量。 总之,需要将代码拆解为各个功能模块,逐步解释每个变量的作用,数学计算的意义,以及整个前向传播流程。</think>以下是对代码的逐层解析: ### 一、Padding计算(核心对齐逻辑) 该模块通过数学公式计算卷积前后的尺寸对齐,保证上/下采样后的尺寸匹配。主要步骤: 1. **目标尺寸计算** $$pad_{\text{total}} = (\text{out\_size} - 1) \times \text{down\_factor} + 1$$ - 表示下采样前需要的中间尺寸 2. **输入尺寸扣除** $$pad_{\text{total}} -= (\text{in\_size} + \text{conv\_kernel} - 1) \times \text{up\_factor}$$ - 计算上采样后的输入尺寸 3. **滤波器尺寸补偿** $$pad_{\text{total}} += (\text{up\_taps} + \text{down\_taps} - 2)$$ - 补偿上下采样滤波器的尺寸缩减 4. **对称填充分配** $$pad_{\text{lo}} = \frac{pad_{\text{total}} + \text{up\_factor}}{2}$$ $$pad_{\text{hi}} = pad_{\text{total}} - pad_{\text{lo}}$$ ### 二、前向传播流程(关键操作链) ```python def forward(...): ``` 1. **输入验证** - 检查noise_mode合法性 - 验证输入张量x的尺寸格式:[batch, in_channels, height, width] - 验证样式向量w的尺寸:[batch, w_dim] 2. **动态归一化** ```python self.magnitude_ema.copy_(magnitude_cur.lerp(...)) input_gain = self.magnitude_ema.rsqrt() ``` - 使用指数移动平均(EMA)跟踪输入幅度的平方均值 - 通过逆平方根实现自适应幅值归一化 3. **样式调制** ```python styles = self.affine(w) # 全连接层映射样式向量 if self.is_torgb: styles *= 1/√(in_channels×conv_kernel²) # 稳定RGB输出 ``` - 将潜在向量w转换为卷积权重调制参数 4. **调制卷积核** ```python x = modulated_conv2d(...) # 核心操作 ``` - 实现StyleGAN2的特征解耦机制 - 根据styles动态调整卷积核权重 5. **非线性激活** ```python filtered_lrelu(...) # 带滤波的激活 ``` - 结合上/下采样滤波器的leaky ReLU - 关键参数: - `gain=√2`(保持方差稳定) - `slope=0.2`(标准leaky参数) - `clamp`防止数值爆炸 ### 、设计特点分析 1. **抗锯齿处理** - 通过`up_filter`和`down_filter`实现频域控制 - 配合padding策略消除混叠现象 2. **数值稳定性** - EMA动态归一化 - weight_gain显式控制 - clamp截断保护 3. **硬件适配** - 自动选择float16/float32 - CUDA设备优化 ### 四、典型应用场景 该模块常见于: 1. StyleGAN系列生成器的特征变换层 2. 渐进式生成结构中的分辨率切换模块 3. 需要频域控制的超分辨率网络 建议结合附录C.3的对称解释原则,理解padding分配的数学依据。核心思想是通过精确的几何级数计算,实现任意缩放倍率下的尺寸对齐。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

Wanderer001

ROIAlign原理

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值