nn.Linear & Tensor.masked_fill

1. nn.Linear

• torch.nn.Linear(in_features, out_features, bias=True, device=None, dtype=None)

  • Applies a linear transformation to the incoming data: $y=x{A}^T+b$
  • This module supports TensorFloat32.
  • On certain ROCm devices, when using float16 inputs this module will use different precision for backward.

• Parameters

  • in_features – size of each input sample
  • out_features – size of each output sample
  • bias – If set to False, the layer will not learn an additive bias. Default: True

• Shape:

  • Input: $(\ast ,H_{in})$, where * means any number of dimensions including none and $H_{in}=\text{in\_features}$
  • Output: $(\ast ,H_{out})$, where all but the last dimension are the same shape as the input and $H_{out}=\text{out\_features}$

2. Tensor.masked_fill

Tensor.masked_fill_(mask, value)

Fills elements of self tensor with value where mask is True. The shape of mask must be broadcastable with the shape of the underlying tensor.

• Parameters
  • mask (BoolTensor) – the boolean mask
  • value (float) – the value to fill in with