CNN、RNN中Pytorch参数详细介绍
最近在用PyTorch写CNN和RNN网络,对里面的参数构造非常的不清楚;专门写个文章记录下
1. ReLu
很好理解,就是是否inplace修改值;
# inplace: can optionally do the operation in-place.
def __init__(self, inplace: bool = False):
- inplace: 一般都会设置True,我个人认为是因为可以减少空间。
2. Linear
全连接层
"""
in_features: size of each input sample
out_features: size of each output sample
bias: If set to ``False``, the layer will not learn an additive bias.
"""
def __init__(self, in_features: int, out_features: int, bias: bool = True) -> None:
- Bias:一般就按默认的设置就好,不要动他!
- in_features: 如果前面输入的是矩阵,一般会用flatten压平再输入;
全连接层的连续使用很难规定(224,n)->(n,10)中间的参数怎么选;至少我还没搞懂。
3. BatchNormal2d
这里不讲1d,2d,3d的区别
"""
num_features: :math:`C` from an expected input of size :math:`(N, C, H, W)`
eps: a value added to the denominator for numerical stability. Default: 1e-5
momentum: the value used for the running_mean and running_var computation. Can be set to ``None`` for cumulative moving average (i.e. simple average). Default: 0.1
affine: a boolean value that when set to ``True``, this module has learnable affine parameters. Default: ``True``
track_running_stats: a boolean value that when set to ``True``, this module tracks the running mean and variance, and when set to ``False``, this module does not track such statistics, and initializes statistics buffers :attr:`running_mean` and :attr:`running_var` as ``None``. then these buffers are ``None``, this module always uses batch statistics. in both training and eval modes. Default: ``True``
"""
def __init__(self, num_features, eps=1e-5, momentum=0.1, affine=True,
track_running_stats=True):
- num_features: 就是channel的值,比如有N张图片,每张图片有C个channel,每个channel是个(H,W)的矩阵,而归一化就是将(H,W)的矩阵做归一化,因此是需要遍历每个channel;而torch.nn中的处理是无需考虑N(batch_size)的,CNN中默认第一个参数就是batch_size。
- 其他参数: 默认就行!
4. MaxPool2d
"""
kernel_size: the size of the window to take a max over
stride: the stride of the window. Default value is :attr:`kernel_size`
padding: implicit zero padding to be added on both sides
dilation: a parameter that controls the stride of elements in the window
return_indices: if ``True``, will return the max indices along with the outputs. Useful for :class:`torch.nn.MaxUnpool2d` later
ceil_mode: when True, will use `ceil` instead of `floor` to compute