nn.ModuleList和nn.Sequential的区别

nn.ModuleList，nn.Sequential，这些类我们称为为容器。存放nn.Module的子类（如nn.Conv2d，nn.Linear等）。

nn.ModuleList 类似于List，但是也有很大的区别。

1.加入到nn.ModuleList里面的module是会自动注册到整个网络上的，同时module的parameters也会自动添加到整个网络中。若使用python的list，则会出问题。

class net1(nn.Module):
    def __init__(self):
        super(net1, self).__init__()
        self.linears = nn.ModuleList([nn.Linear(10,10) for i in range(2)])
    def forward(self, x):
        for m in self.linears:
            x = m(x)
        return x
 
net = net1()
print(net)
#这个网络权重(weights)和偏置(bias)都在这个网络之内

for param in net.parameters():
    print(type(param.data), param.size())

class net2(nn.Module):
    def __init__(self):
        super(net2, self).__init__()
        self.linears = [nn.Linear(10,10) for i in range(2)]
    def forward(self, x):
        for m in self.linears:
            x = m(x)
        return x
 
net = net2()
print(net)
print(list(net.parameters()))

卷积层和它们的parameters并没有自动注册到我们的网络中，所以网络无法训练。

2.nn.ModuleList并没有定义一个网络，它只是将不同的模块储存在一起，这些模块之间并没有什么先后顺序可言

class net3(nn.Module):
    def __init__(self):
        super(net3, self).__init__()
        self.linears = nn.ModuleList([nn.Linear(10,20), nn.Linear(20,30), nn.Linear(5,10)])
    def forward(self, x):
        x = self.linears[2](x)
        x = self.linears[0](x)
        x = self.linears[1](x) 
        return x
 
net = net3()
print(net)
input = torch.randn(32, 5)
print(net(input).shape)

3.既然ModuleList可以根据序号来调用，那么一个模型可以在forward函数中被调用多次。但需要注意的是，被调用多次的模块，是使用同一组parameters的，也就是它们是参数共享的。 

class net4(nn.Module):
    def __init__(self):
        super(net4, self).__init__()
        self.linears = nn.ModuleList([nn.Linear(5, 10), nn.Linear(10, 10)])
    def forward(self, x):
        x = self.linears[0](x)
        x = self.linears[1](x)
        x = self.linears[1](x)
        return x
 
net = net4()
print(net)
for name, param in net.named_parameters():
    print(name, param.size())

 

nn.Sequential

nn.Sequential已经实现了内部的forward函数，而且里面的模块必须是按照顺序进行排列的

class net5(nn.Module):
    def __init__(self):
        super(net5, self).__init__()
        self.block = nn.Sequential(nn.Conv2d(1,20,5),
                                    nn.ReLU(),
                                    nn.Conv2d(20,64,5),
                                    nn.ReLU())
    def forward(self, x):
        x = self.block(x)
        return x
 
net = net5()
print(net)

 

三个nn.Sequential初始化的例子

model1 = nn.Sequential(
          nn.Conv2d(1,20,5),
          nn.ReLU(),
          nn.Conv2d(20,64,5),
          nn.ReLU()
        )
print(model1)

 

import collections
model2 = nn.Sequential(collections.OrderedDict([
          ('conv1', nn.Conv2d(1,20,5)),
          ('relu1', nn.ReLU()),
          ('conv2', nn.Conv2d(20,64,5)),
          ('relu2', nn.ReLU())
        ]))
print(model2)

class net7(nn.Module):
    def __init__(self):
        super(net7, self).__init__()
        self.linear_list = [nn.Linear(10, 10) for i in range(3)]
        self.linears = nn.Sequential(*self.linear_list)
 
    def forward(self, x):
        x = self.linears(x)
        return x
 
net = net7()
print(net)