Pytorch 搭建卷积神经网络基础

Pytorch搭建卷积神经网络基础

感谢pytorch详解nn.Module类,children和modules方法区别

一,卷积层构建

1,1D卷积层构建

torch.nn.Conv1d(in_channels, out_channels, kernel_size, stride:= 1, padding= 0, dilation= 1, groups1, bias= True, padding_mode = 'zeros')

对输入信号上应用一维卷积 。输入信号尺寸(N, C_{\text{in}}, L),卷积后的输出信号尺寸(N, C_{\text{out}}, L_{\text{out}})。N 是batch size, C表示通道数,L信号的序列长度。

卷积运算过程:

\text{out}(N_i, C_{\text{out}_j}) = \text{bias}(C_{\text{out}_j}) + \sum_{k = 0}^{C_{in} - 1} \text{weight}(C_{\text{out}_j}, k) \star \text{input}(N_i, k)

group:设置卷积的分组

group=1:所有的输入信号卷积为输出

group=2:等效于两个并排的卷积层,每一个看到一半的输入通道,产生一半的输出通道。最后 concatenated.

groups= in_channels:每一个输入通道都和一个滤波器卷积,滤波器的尺寸为\left\lfloor\frac{out\_channels}{in\_channels}\right\rfloor

一维卷积需要合适的padding来保证所有的输入 

如果groups=in_channels,一维卷积是depthwise convolution,输出通道一定是输入通道的K倍。如果输入信号尺寸为(N, C_{in}, L_{in}),depthwise 乘法器K的深度卷积的输出信号尺寸为(C_\text{in}=C_{in}, C_\text{out}=C_{in} \times K, ..., \text{groups}=C_{in})

输入信号:(N, C_{in}, L_{in})

输出信号:(N, C_{out}, L_{out})

ID卷积运算:

L_{out} = \left\lfloor\frac{L_{in} + 2 \times \text{padding} - \text{dilation} \times (\text{kernel\_size} - 1) - 1}{\text{stride}} + 1\right\rfloor

2,2D卷积层构建

class torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)

参数:

  • in_channels(int) – 输入信号的通道
  • out_channels(int) – 卷积产生的通道
  • kerner_size(int or tuple) - 卷积核的尺寸
  • stride(int or tupleoptional) - 卷积步长,卷积核在图像窗口上每次平移的间隔

padding(int or tupleoptional) - 输入的每一条边补充0的层数,填充包括图像的上下左右,以padding = 1为例,若原始图像大小为32x32,那么padding后的图像大小就变成了34x34,而不是33x33。

  • dilation(int or tupleoptional) – 卷积核元素之间的间距
  • groups(intoptional) – 从输入通道到输出通道的阻塞连接数
  • bias(booloptional) - 如果bias=True,添加偏置

 参数kernel_sizestride,paddingdilation也可以是一个int的数据,此时卷积height和width值相同;也可以是一个tuple数组,tuple的第一维度表示height的数值,tuple的第二维度表示width的数值

(eg:kernel_size=3,卷积核为3×3;kernel_size=(3,5),卷积核为3×5.

对图像卷积,N一次输入的Batch数

3,Conv3d

torch.nn.Conv3d(in_channels: int, out_channels: int, kernel_size: Union[T, Tuple[T, T, T]], stride: Union[T, Tuple[T, T, T]] = 1, padding: Union[T, Tuple[T, T, T]] = 0, dilation: Union[T, Tuple[T, T, T]] = 1, groups: int = 1, bias: bool = True, padding_mode: str = 'zeros')

3D卷积层的输入尺寸(N, C_{in}, D, H, W),输出尺寸 (N, C_{out}, D_{out}, H_{out}, W_{out})

out(N_i, C_{out_j}) = bias(C_{out_j}) + \sum_{k = 0}^{C_{in} - 1} weight(C_{out_j}, k) \star input(N_i, k)

\star3D互相关运算

Input: (N, C_{in}, D_{in}, H_{in}, W_{in})

Output:(N, C_{out}, D_{out}, H_{out}, W_{out})

D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - \text{dilation}[0] \times (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor

H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - \text{dilation}[1] \times (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor

W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - \text{dilation}[2] \times (\text{kernel\_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor

二,定义神经网络 

1,搭建基础自定义模型

Pytyotch中继承nn.Module类实现自定义模型类有两种方式:

1)高层API方法:使用torch.nn.****来实现;,这些接口都是类,类可以存储参数,比如全连接层的权值矩阵、偏置矩阵等都可以作为类的属性存储着,当对这些接口类创建对象后再作为自定义模型类的属性,就实现了存储参数

2)低层API方法:使用低层函数方法,torch.nn.functional.****来实现;,从名称就看出是一些函数接口,实现函数的运算功能,没办法保存这些信息,若是用它创建有学习参数的层,需要自己再实现保存参数的部分。

在自定义模型时必须重新实现构造函数__init__构造函数和forward这两个方法。

  • 一般把网络中具有可学习参数的层(如全连接层、卷积层等)放在构造函数__init__()方法中,当然也可以把不具有参数的层也放在__init__方法里面;
  • 不具有可学习参数的层(如ReLU、dropout、BatchNormanation层)可放在构造函数__init__中,也可不放在构造函数__init__中,如果不放在构造函数__init__里面,则在forward方法里面可以使用nn.functional来代替, 因为搭建时将没有训练参数的层 没有放在构造函数里面了(当然就没有这些属性了),所以这些层就不会出现在model里面(打印或可视化model)
  • forward方法是必须要重写的,它是实现模型的功能,实现各个层之间的连接关系的核心。
class Net1(torch.nn.Module):
    def __init__(self):
        # 必须调用父类的构造函数,因为想要使用父类的方法,这也是继承Module的目的
        super(Net, self).__init__()
        self.conv1 = torch.nn.Conv2d(3, 32, 3, 1, 1)
        self.relu1 = torch.nn.ReLU()
        self.max_pooling1 = torch.nn.MaxPool2d(2, 1)
        self.conv2 = torch.nn.Conv2d(3, 32, 3, 1, 1)
        self.relu2 = torch.nn.ReLU()
        self.max_pooling2 = torch.nn.MaxPool2d(2, 1)
        self.dense1 = torch.nn.Linear(32 * 3 * 3, 128)
        self.dense2 = torch.nn.Linear(128, 10)

    def forward(self, x):
        x = self.conv1(x)
        x = self.relu1(x)
        x = self.max_pooling1(x)
        x = self.conv2(x)
        x = self.relu2(x)
        x = self.max_pooling2(x)
        x = self.dense1(x)
        x = self.dense2(x)
        return x

 2,利用torch.nn.Sequential()容器

利用torch.nn.Sequential()容器,模型的各层被顺序添加到容器中。

class Net2(torch.nn.Module):
    def __init__(self):
        super(Net2, self).__init__()
        self.conv = torch.nn.Sequential(
            torch.nn.Conv2d(3, 32, 3, 1, 1),
            torch.nn.ReLU(),
            torch.nn.MaxPool2d(2))
        self.dense = torch.nn.Sequential(
            torch.nn.Linear(32 * 3 * 3, 128),
            torch.nn.ReLU(),
            torch.nn.Linear(128, 10)
        )

    def forward(self, x):
        conv_out = self.conv1(x)
        res = conv_out.view(conv_out.size(0), -1)
        out = self.dense(res)
        return out


print("Method 2:")
model2 = Net2()
print(model2)
print(model2.conv[2])  # 先获取属性,再下标索引,
'''
Net2(
  (conv): Sequential(
    (0): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU()
    (2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (dense): Sequential(
    (0): Linear(in_features=288, out_features=128, bias=True)
    (1): ReLU()
    (2): Linear(in_features=128, out_features=10, bias=True)
  )
)
可看出模型有两个属性分别是conv和dense, 且他们都是Sequential类型

3,参考Module.children() vs Module.modules()

  • children()返回的是最外层的元素
  • modules方法将整个模型的所有构成(包括包装层Sequential、单独的层、自定义层等)由浅入深依次遍历出来,直到最深处的单层
class Net3(torch.nn.Module):
    def __init__(self):
        super(Net3, self).__init__()
        self.conv = torch.nn.Sequential(
            OrderedDict(
                [
                    ("conv1", torch.nn.Conv2d(3, 32, 3, 1, 1)),
                    ("relu1", torch.nn.ReLU()),
                    ("pool1", torch.nn.MaxPool2d(2))
                ]
            ))

        self.dense = torch.nn.Sequential(
            OrderedDict([
                ("dense1", torch.nn.Linear(32 * 3 * 3, 128)),
                ("relu2", torch.nn.ReLU()),
                ("dense2", torch.nn.Linear(128, 10))
            ])
        )

    def forward(self, x):
        conv_out = self.conv1(x)
        res = conv_out.view(conv_out.size(0), -1)
        out = self.dense(res)
        return out


print("Method :")
model = Net3()
Net3(
  (conv): Sequential(
    (conv1): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (relu1): ReLU()
    (pool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (dense): Sequential(
    (dense1): Linear(in_features=288, out_features=128, bias=True)
    (relu2): ReLU()
    (dense2): Linear(in_features=128, out_features=10, bias=True)
  )
)
==============================




# children()方法其实就是获取模型的属性,可看到构造函数有两个属性
for i in model.children():
    print(i)
    print(type(i))
print('==============================')
# named_children()方法也是获取模型的属性,同时获取属性的名字
for i in model.named_children():
    print(i)
    print(type(i))


'''
Sequential(
  (conv1): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (relu1): ReLU()
  (pool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
<class 'torch.nn.modules.container.Sequential'>
Sequential(
  (dense1): Linear(in_features=288, out_features=128, bias=True)
  (relu2): ReLU()
  (dense2): Linear(in_features=128, out_features=10, bias=True)
)
<class 'torch.nn.modules.container.Sequential'>

==============================-----------------------------------------------
('conv', Sequential(
  (conv1): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (relu1): ReLU()
  (pool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
))
<class 'tuple'>
('dense', Sequential(
  (dense1): Linear(in_features=288, out_features=128, bias=True)
  (relu2): ReLU()
  (dense2): Linear(in_features=128, out_features=10, bias=True)
))
<class 'tuple'>
'''



for i in model.modules():
    print(i)
    print('==============================')
print('=============华丽分割线=================')
# named_modules()同上,但是返回的每一个元素是一个元组,第一个元素是名称,第二个元素才是层对象本身。
for i in model.named_modules():
    print(i)
    print('==============================')
'''
离模型最近即最浅处,就是模型本身

由浅入深,到模型的属性
Sequential(
  (conv1): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (relu1): ReLU()
  (pool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
==============================
由浅入深,再到模型的属性的内部
Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
==============================
由浅入深,再到模型的属性的内部,依次将这个属性遍历结束
ReLU()
==============================
MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
由浅入深,再到模型的属性的内部,依次将这个属性遍历结束,再遍历另个属性
==============================
Sequential(
  (dense1): Linear(in_features=288, out_features=128, bias=True)
  (relu2): ReLU()
  (dense2): Linear(in_features=128, out_features=10, bias=True)
)
==============================
Linear(in_features=288, out_features=128, bias=True)
==============================
ReLU()
==============================
Linear(in_features=128, out_features=10, bias=True)
==============================
=============华丽分割线=================------------------------------------
('', Net4(
  (conv): Sequential(
    (conv1): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (relu1): ReLU()
    (pool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (dense): Sequential(
    (dense1): Linear(in_features=288, out_features=128, bias=True)
    (relu2): ReLU()
    (dense2): Linear(in_features=128, out_features=10, bias=True)
  )
))
==============================
('conv', Sequential(
  (conv1): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (relu1): ReLU()
  (pool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
))
==============================
('conv.conv1', Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
==============================
('conv.relu1', ReLU())
==============================
('conv.pool1', MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False))
==============================
('dense', Sequential(
  (dense1): Linear(in_features=288, out_features=128, bias=True)
  (relu2): ReLU()
  (dense2): Linear(in_features=128, out_features=10, bias=True)
))
==============================
('dense.dense1', Linear(in_features=288, out_features=128, bias=True))
==============================
('dense.relu2', ReLU())
==============================
('dense.dense2', Linear(in_features=128, out_features=10, bias=True))
==============================
'''

 

 

©️2020 CSDN 皮肤主题: 大白 设计师:CSDN官方博客 返回首页