CNN、RNN中Pytorch参数详细介绍

CNN、RNN中Pytorch参数详细介绍

最近在用PyTorch写CNN和RNN网络，对里面的参数构造非常的不清楚；专门写个文章记录下

1. ReLu

很好理解，就是是否inplace修改值；

#  inplace: can optionally do the operation in-place.
def __init__(self, inplace: bool = False):

• inplace: 一般都会设置True，我个人认为是因为可以减少空间。

2. Linear

全连接层

"""
in_features: size of each input sample
out_features: size of each output sample
bias: If set to ``False``, the layer will not learn an additive bias.
"""
def __init__(self, in_features: int, out_features: int, bias: bool = True) -> None:

• Bias：一般就按默认的设置就好，不要动他！
• in_features: 如果前面输入的是矩阵，一般会用flatten压平再输入；

全连接层的连续使用很难规定(224,n)->(n,10)中间的参数怎么选；至少我还没搞懂。

3. BatchNormal2d

这里不讲1d,2d,3d的区别

"""
num_features: :math:`C` from an expected input of size :math:`(N, C, H, W)`
eps: a value added to the denominator for numerical stability. Default: 1e-5
momentum: the value used for the running_mean and running_var computation. Can be set to ``None`` for cumulative moving average (i.e. simple average). Default: 0.1
affine: a boolean value that when set to ``True``, this module has learnable affine parameters. Default: ``True``
track_running_stats: a boolean value that when set to ``True``, this module tracks the running mean and variance, and when set to ``False``, this module does not track such statistics, and initializes statistics buffers :attr:`running_mean` and :attr:`running_var` as ``None``. then these buffers are ``None``, this module always uses batch statistics. in both training and eval modes. Default: ``True``
"""
def __init__(self, num_features, eps=1e-5, momentum=0.1, affine=True,
                 track_running_stats=True):

• num_features: 就是channel的值，比如有N张图片，每张图片有C个channel，每个channel是个(H,W)的矩阵，而归一化就是将(H,W)的矩阵做归一化，因此是需要遍历每个channel；而torch.nn中的处理是无需考虑N(batch_size)的，CNN中默认第一个参数就是batch_size。
• 其他参数: 默认就行！

4. MaxPool2d

"""
kernel_size: the size of the window to take a max over
stride: the stride of the window. Default value is :attr:`kernel_size`
padding: implicit zero padding to be added on both sides
dilation: a parameter that controls the stride of elements in the window
return_indices: if ``True``, will return the max indices along with the outputs. Useful for :class:`torch.nn.MaxUnpool2d` later
ceil_mode: when True, will use `ceil` instead of `floor` to compute