grad在python什么模块_Python中的参数类和模块类摘要,pytorch,parameter,Module,总结

看了pytorch官网parameters和Module部分。做个部分测试并记录。详细建议去

官网

查询

一、nn.Parameter类

是Tensor的子类。

唯一区别：在和Module一起使用时。当parameter赋值给module的属性时，会自动将parameter加入到模型的参数列表中。而tensor不会。

二、nn.Module是一个python容器。

是所有神经网络模块的基类。

Module可以包含其他的Module模块，支持用树结构来嵌入他们。

可以直接将模块赋值给Module的普通属性

"""可以在__init__构建函数中添加模块"""

class NeuralNet(nn.Module):

def __init__(self):

super(NeuralNet,self).__init__()

linear = nn.Linear(512,3)

self.linear1 = linear

self.linear2 = linear

self.test1 = nn.Parameter(torch.tensor([1,2,3],dtype=torch.float),requires_grad=True)

self.test2 = torch.tensor([4,5,6],dtype=torch.float)

self.add_module('linear3',nn.Linear(512,4)) #add_module方法等同于直接给model的属性赋值。

model.children(): 返回模型的所有子模块的迭代器

model.modules():返回模型的所有模块(不仅仅是子模块，还包含当前模块)

model.named_children():返回当前子模块的迭代器。名字：模块

model.named_modules():

测试代码：

model = NeuralNet()

print("\n*****children ****\n")

for i in model.children():

print(i)

print("\n*****modules ****\n")

for i in model.modules():

print(i)

print("\n*****named children ****\n")

for i in model.named_children():

print(i)

print("\n*****named modules ****\n")

for i in model.named_modules():

print(i)

测试结果：

*****children ****

Linear(in_features=512, out_features=3, bias=True)

Linear(in_features=512, out_features=4, bias=True)

*****modules ****

NeuralNet(

(linear1): Linear(in_features=512, out_features=3, bias=True)

(linear2): Linear(in_features=512, out_features=3, bias=True)

(linear3): Linear(in_features=512, out_features=4, bias=True)

)

Linear(in_features=512, out_features=3, bias=True)

Linear(in_features=512, out_features=4, bias=True)

*****named children ****

('linear1', Linear(in_features=512, out_features=3, bias=True))

('linear3', Linear(in_features=512, out_features=4, bias=True))

*****named modules ****

('', NeuralNet(

(linear1): Linear(in_features=512, out_features=3, bias=True)

(linear2): Linear(in_features=512, out_features=3, bias=True)

(linear3): Linear(in_features=512, out_features=4, bias=True)

))

('linear1', Linear(in_features=512, out_features=3, bias=True))

('linear3', Linear(in_features=512, out_features=4, bias=True))

model.buffers:返回模型所有缓存参数的迭代器(缓存参数：不需要训练的)，

model.parameters:返回模型所有参数的迭代器

module.named_buffers:

module.named_parameters:

module.parameters常和optimizer()一起使用，来制定需要优化的参数。

测试代码：

print("\n*****buffers ****\n")

for i in model.buffers():

print(i)

print("\n*****named buffers ****\n")

for i in model.named_buffers():

print(i)

print("\n*****parameters ****\n")

for i in model.parameters():

print(i)

print("\n*****named parameters ****\n")

for i in model.named_parameters():

print(i)

测试结果：

*****buffers ****

*****named buffers ****

*****parameters ****

Parameter containing:

tensor([1., 2., 3.], requires_grad=True)

Parameter containing:

tensor([[-0.0403, -0.0027, 0.0287, ..., -0.0113, -0.0403, -0.0343],

[-0.0389, -0.0086, -0.0035, ..., 0.0246, -0.0127, -0.0313],

[ 0.0275, -0.0339, -0.0109, ..., 0.0304, 0.0250, 0.0273]],

requires_grad=True)

Parameter containing:

tensor([-0.0349, 0.0117, -0.0313], requires_grad=True)

Parameter containing:

tensor([[-0.0166, -0.0328, 0.0354, ..., -0.0193, -0.0410, 0.0034],

[-0.0350, -0.0207, -0.0372, ..., -0.0130, -0.0279, 0.0041],

[-0.0320, 0.0410, 0.0157, ..., -0.0358, 0.0145, 0.0272],

[ 0.0214, 0.0276, 0.0103, ..., 0.0165, -0.0265, -0.0109]],

requires_grad=True)

Parameter containing:

tensor([ 0.0035, 0.0182, -0.0215, 0.0390], requires_grad=True)

*****named parameters ****

('test1', Parameter containing:

tensor([1., 2., 3.], requires_grad=True))

('linear1.weight', Parameter containing:

tensor([[-0.0403, -0.0027, 0.0287, ..., -0.0113, -0.0403, -0.0343],

[-0.0389, -0.0086, -0.0035, ..., 0.0246, -0.0127, -0.0313],

[ 0.0275, -0.0339, -0.0109, ..., 0.0304, 0.0250, 0.0273]],

requires_grad=True))

('linear1.bias', Parameter containing:

tensor([-0.0349, 0.0117, -0.0313], requires_grad=True))

('linear3.weight', Parameter containing:

tensor([[-0.0166, -0.0328, 0.0354, ..., -0.0193, -0.0410, 0.0034],

[-0.0350, -0.0207, -0.0372, ..., -0.0130, -0.0279, 0.0041],

[-0.0320, 0.0410, 0.0157, ..., -0.0358, 0.0145, 0.0272],

[ 0.0214, 0.0276, 0.0103, ..., 0.0165, -0.0265, -0.0109]],

requires_grad=True))

('linear3.bias', Parameter containing:

tensor([ 0.0035, 0.0182, -0.0215, 0.0390], requires_grad=True))

注意parameters和tensor的区别：parameters被自动加入到模型的参数列表中。而tensor没有。

model.cpu():所有模型parameters和buffers移到cpu

model.cuda():所有模型parameters和buffers移到gpu

model.train():设置模型到训练模式。只对部分模型有影响(包含Dropout，BatchNorm)。

model.eval():设置模型到测试模式。