nn.Parameter和nn.Linear、nn.ModuleList和nn.Sequential区别

taoqick

已于 2024-11-21 10:50:34 修改

阅读量204

点赞数 2

文章标签： python 人工智能深度学习

于 2024-10-21 23:42:19 首次发布

本文链接：https://blog.csdn.net/taoqick/article/details/143136227

版权

self.weight = nn.Parameter(torch.FloatTensor(out_features, in_features)) 和self.weight = nn.Linear(in_features, out_features) 有什么区别？

初始化方式不一样，nn.Parameter初始化方式需要指定，nn.Linear默认是Kaiming初始化
nn.Linear默认有个bias

nn.ModuleList和nn.Sequential区别

nn.ModuleList不会自动连接子模块，需要在forward方法中显式地调用每个模块的前向传播。但是nn.Sequential是一种便捷的方式，用于定义按顺序执行的神经网络层

nn.ModuleList示例

import torch
import torch.nn as nn

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.layers = nn.ModuleList([nn.Linear(10, 10) for _ in range(3)])
    
    def forward(self, x):
        for layer in self.layers:
            x = layer(x)
        return x

# 实例化模型并进行前向传播
model = MyModel()
input_tensor = torch.randn(5, 10)
output = model(input_tensor)
print(output)

nn.Sequential通过有序的子模块：

import torch
import torch.nn as nn

# 定义模型
model = nn.Sequential(
    nn.Linear(10, 20),
    nn.ReLU(),
    nn.Linear(20, 10)
)

# 进行前向传播
input_tensor = torch.randn(5, 10)
output = model(input_tensor)
print(output)

nn.Sequential通过字典的方式：

# 使用OrderedDict
from collections import OrderedDict

model = nn.Sequential(OrderedDict([
    ('fc1', nn.Linear(10, 20)),
    ('relu', nn.ReLU()),
    ('fc2', nn.Linear(20, 10))
]))

# 进行前向传播
input_tensor = torch.randn(5, 10)
output = model(input_tensor)
print(output)