pytorch使用小技巧，亦是工程小技巧

将一个函数或者类（python中统称为对象）作为不同身份使用

• 以下是Linear class

class Linear(Module):
    r"""Applies a linear transformation to the incoming data: :math:`y = xA^T + b`

    This module supports :ref:`TensorFloat32<tf32_on_ampere>`.

    Args:
        in_features: size of each input sample
        out_features: size of each output sample
        bias: If set to ``False``, the layer will not learn an additive bias.
            Default: ``True``

    Shape:
        - Input: :math:`(N, *, H_{in})` where :math:`*` means any number of
          additional dimensions and :math:`H_{in} = \text{in\_features}`
        - Output: :math:`(N, *, H_{out})` where all but the last dimension
          are the same shape as the input and :math:`H_{out} = \text{out\_features}`.

    Attributes:
        weight: the learnable weights of the module of shape
            :math:`(\text{out\_features}, \text{in\_features})`. The values are
            initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where
            :math:`k = \frac{1}{\text{in\_features}}`
        bias:   the learnable bias of the module of shape :math:`(\text{out\_features})`.
                If :attr:`bias` is ``True``, the values are initialized from
                :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where
                :math:`k = \frac{1}{\text{in\_features}}`

    Examples::

        >>> m = nn.Linear(20, 30)
        >>> input = torch.randn(128, 20)
        >>> output = m(input)
        >>> print(output.size())
        torch.Size([128, 30])
    """
    __constants__ = ['in_features', 'out_features']
    in_features: int
    out_features: int
    weight: Tensor

    def __init__(self, in_features: int, out_features: int, bias: bool = True,
                 device=None, dtype=None) -> None:
        factory_kwargs = {'device': device, 'dtype': dtype}
        super(Linear, self).__init__()
        self.in_features = in_features
        self.out_features = out_features
        self.weight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))
        if bias:
            self.bias = Parameter(torch.empty(out_features, **factory_kwargs))
        else:
            self.register_parameter('bias', None)
        self.reset_parameters()

    def reset_parameters(self) -> None:
        init.kaiming_uniform_(self.weight, a=math.sqrt(5))
        if self.bias is not None:
            fan_in, _ = init._calculate_fan_in_and_fan_out(self.weight)
            bound = 1 / math.sqrt(fan_in) if fan_in > 0 else 0
            init.uniform_(self.bias, -bound, bound)

    def forward(self, input: Tensor) -> Tensor:
        return F.linear(input, self.weight, self.bias)

    def extra_repr(self) -> str:
        return 'in_features={}, out_features={}, bias={}'.format(
            self.in_features, self.out_features, self.bias is not None
        )

它可以在所有支持的类中使用：如先初始化，再使用，像下面这样：

class Attention(nn.Module):
	def __init__(self,
			...):
		...省略代码...
		self.out_proj= Linear(in_features, out_features, bias=bias,
                         device=device, dtype=dtype)
		   ...省略代码...
    def forward(self, x):
		...省略代码...
		self.out_proj()

此处self.out_proj用于将attention计算得到的结果进行汇总。这个名称或者说这个做的事情的含义，其实可以将Linear class进行可以再封装，给一个适合做这件事情的名称，之后在其他构建的attention中就可以使用它所代表的含义，虽然还是Linear class本身。

如下面对Linear class进行再封装：

class NonDynamicallyQuantizableLinear(Linear):
    def __init__(self, in_features: int, out_features: int, bias: bool = True,
                 device=None, dtype=None) -> None:
        super().__init__(in_features, out_features, bias=bias,
                         device=device, dtype=dtype)

接下来看它在多个地方的使用，你就会通过它的名称显而易见知道它是在做什么，那它就是专注于做一件事，即我们对它进行了归类，这样看上去会让整个工程清晰明了：

class Attention1(nn.Module):
	def __init__(self,
			...):
		...省略代码...
		self.out_proj = NonDynamicallyQuantizableLinear(embed_dim, embed_dim, bias=bias, **factory_kwargs)
		   ...省略代码...
    def forward(self, x):
		...省略代码...
		self.out_proj()

class Attention2(nn.Module):
	def __init__(self,
			...):
		...省略代码...
		self.out_proj = NonDynamicallyQuantizableLinear(embed_dim, embed_dim, bias=bias, **factory_kwargs)
		   ...省略代码...
    def forward(self, x):
		...省略代码...
		self.out_proj()

其实只是对官方固有名称的延续归类,只是猫叫了咪，它就换了几个身份。
来自官方代码的芝麻大小的感悟： https://github.com/pytorch/pytorch/blob/main/torch/nn/modules/activation.py