Pytorch 模型分析

自定义模型

import torch
from torch import nn

class MyNet(nn.Module):
    def __init__(self):
        super(MyNet, self).__init__()
        self.forward1 = nn.Sequential(
            nn.Conv3d(1, 8, kernel_size=3, padding=1),
            nn.BatchNorm3d(8),
            nn.ReLU(inplace=True),
            nn.Conv3d(8, 8, kernel_size=3, padding=1),
            nn.BatchNorm3d(8),
            nn.ReLU(inplace=True))

        self.avgpool = nn.AvgPool3d(kernel_size=2, stride=2)
        self.maxpool = nn.MaxPool3d(kernel_size=2, stride=2)

    def forward(self, input):
        input = input.float()
        layer1 = self.forward1(input)

        down1 = self.maxpool(layer1)
        source1 = self.avgpool(input)
        cat1 = torch.cat((down1, source1), 1)
        return cat1

net = MyNet()
input = torch.randn([1,1,16,256,256])#(NCDHW)
if torch.cuda.is_available():
    net.cuda()#necessary for torchsummary, must to cuda
    input = input.cuda()
out = net(input)

模型可视化

方法一:Tensorboard in Pytorch

Pytorch 模型的网络结构可视化

方法二:pytorchviz

（很遗憾，in fact, this code has something wrong, so I hope you can improve it.）
使用pytorchviz和Github中的模型可视化函make_dot()

sudo apt-get install graphviz

then use the demo in the code package
Notice: I installed the ‘graphviz’ package in Pycharm but encounter this error:

graphviz.backend.ExecutableNotFound: failed to execute ['dot', '-Tpdf', '-O', 'Digraph.gv'], make sure the Graphviz executables are on your systems' PATH

In Ubuntu, you should install the graphviz package in your system (not just the python package): sudo apt-get install graphviz

打印模型参数

Attention: 参数的输出顺序按照模型定义中的 init(self)函数，而非forward(self, input)函数。

Method1: torchsummary

from torchsummary import summary
print('############summary#############')
summary(net, input_size=(1,16,256,256))#must remove the number of N

output:
############summary#############
----------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================
Conv3d-1 [-1, 8, 16, 256, 256] 224
BatchNorm3d-2 [-1, 8, 16, 256, 256] 16
ReLU-3 [-1, 8, 16, 256, 256] 0
Conv3d-4 [-1, 8, 16, 256, 256] 1,736
BatchNorm3d-5 [-1, 8, 16, 256, 256] 16
ReLU-6 [-1, 8, 16, 256, 256] 0
MaxPool3d-7 [-1, 8, 8, 128, 128] 0
AvgPool3d-8 [-1, 1, 8, 128, 128] 0
================================================================
Total params: 1,992 Trainable params: 1,992 Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 4.00
Forward/backward pass size (MB): 393.00
Params size (MB): 0.01
Estimated Total Size (MB): 397.01
---------------------------------------------------------------

Method2: net.named_parameters()

 print('############net.named_parameters()#############')
for name,param in net.named_parameters():
   print(name)

output
############net.named_parameters()#############
forward1.0.weight
forward1.0.bias
forward1.1.weight
forward1.1.bias
forward1.3.weight
forward1.3.bias
forward1.4.weight
forward1.4.bias

优点：简洁明了，方便结合模型的summary输出，check模型中的参数。

Method3: net.state_dict()

print('############net.state_dict()#############')
for name in net.state_dict():
    print(name)

output
############net.state_dict()#############
forward1.0.weight
forward1.0.bias
forward1.1.weight
forward1.1.bias
forward1.1.running_mean
forward1.1.running_var
forward1.1.num_batches_tracked
forward1.3.weight
forward1.3.bias
forward1.4.weight
forward1.4.bias
forward1.4.running_mean
forward1.4.running_var
forward1.4.num_batches_tracked

缺点：与named_parameters()相比，有许多冗余输出，不方便对参数进行check。

Method4: net.modules()

print('############net.modules()#############')
print(list(net.modules())[0])

output
############net.modules()#############
TNet(
(forward1): Sequential(
(0): Conv3d(1, 8, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1))
(1): BatchNorm3d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace)
(3): Conv3d(8, 8, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1))
(4): BatchNorm3d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU(inplace)
)
(avgpool): AvgPool3d(kernel_size=2, stride=2, padding=0)
(maxpool): MaxPool3d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)

优点：输出神经网络中每层的参数设置（包括默认参数）