Pytorch 模型参数访问和初始化
本方法总结自《动手学深度学习》(Pytorch版)github项目
网络构建
网络构建相关可参考 Pytorch 学习(七):Pytorch 网络模型创建,建立一个多层感知器
import torch
import torch.nn as nn
net = nn.Sequential(
nn.Linear(2, 2),
nn.ReLU(),
nn.Linear(2, 1),
)
参数访问
for param in net.parameters():
print(param)
for name, param in net.name_parameters():
print(name, param)
参数初始化
- 使用 init 模块进行初始化
import torch.nn.init as init
for name, param in net.named_parameters():
print(name, param)
if 'weight' in name:
init.normal_(param, mean=0, std=0.01)
elif 'bias' in name:
init.constant_(param, val=0)
- 使用自定义方法初始化,注意防止梯度回传
def normal_(tensor, mean=0, std=0.03):
with torch.no_grad(): # 关闭梯度
tensor.normal_(mean=mean, std=std)
for name, param in net.named_parameters():
print(name, param)
if 'weight' in name:
normal_(param, mean=0, std=0.01)
elif 'bias' in name:
param.data = torch.tensor(1.0) # 不改变梯度