深度学习每周学习总结J4（DPN 算法探索实践 - 鸟类识别）

• 🍨 本文为🔗365天深度学习训练营 中的学习记录博客
• 🍖 原作者：K同学啊 | 接辅导、项目定制

一：回顾与总结： 三种神经网络模型对比研究及尝试构成新的网络结构模型

卷积计算过程

对于卷积新的理解：

我之前的误解是错误以为有几个卷积核，就有几个权重矩阵。比如输入通道数为3，输出通道数为2，我误以为只有两个权重矩阵。但其实对于一个卷积层来说，卷积核的数量等同于输出通道数（也称为输出的特征图数）。如果输入通道数为3，输出通道数为2，那么根据卷积操作的定义，每个输出通道对应一个单独的卷积核组。每个卷积核组由数量等于输入通道数的权重矩阵组成，因此在这个例子中，实际上有 $3\times 2=6$ 个权重矩阵（即每个输出通道有3个权重矩阵，与输入通道数相对应）。

此外，在训练神经网络时，除了卷积核的参数矩阵（也称为权重矩阵）以外，偏置参数 $ b$ 也是会随着训练过程而改变的。偏置参数 $b$ 是每个神经元的一个独立参数。需要注意这个偏置是与整个卷积核相关的，而不是与单个权重矩阵相关，一个卷积核组可能有复数个权重矩阵。因此，对于每个输出通道，模型在计算完成卷积后，会加上各自的偏置来调整输出，它在每次反向传播过程中通过梯度下降或其他优化算法进行更新。

这样的更新机制是为了让神经网络模型能够更好地拟合训练数据。偏置参数$b$的存在使得每个神经元在没有任何输入时就可以有一个非零输出，从而增加了模型的灵活性和表达能力。

Source: http://cs231n.github.io/convolutional-networks/

ResNet-50 模型

1. 关于残差

目前残差结构主要有两种：

左边的单元为 ResNet 两层的残差单元，两层的残差单元包含两个相同输出的通道数的 3x3 卷积，只是用于较浅的 ResNet 网络，对较深的网络主要使用三层的残差单元。三层的残差单元又称为 bottleneck 结构，先用一个 1x1 卷积进行降维，然后 3x3 卷积，最后用 1x1 升维恢复原有的维度。另外，如果有输入输出维度不同的情况，可以对输入做一个线性映射变换维度，再连接后面的层。三层的残差单元对于相同数量的层又减少了参数量，因此可以拓展更深的模型。通过残差单元的组合有经典的 ResNet-50，ResNet-101 等网络结构。

2. 关于ConvBlock的思考

关于convblock的参数设置例如：[64,64,256]:

主要基于参数选择的经验性：第一个是常用的通道数64是常见的开始值，第二个值为64是为了保持相同的特征数保留和加强特征，最后一个256会大一些是为了增加特征维度和网络模型的表达能力

残差层（layer1 到 layer4）
每个残差层通过 _build_layer 函数构建，每个层的输出通道配置如 [64, 64, 256], [128, 128, 512], [256, 256, 1024], [512, 512, 2048]，仍然是基于经验法则和文献中的设计，确保了网络在进行深度学习时能够高效提取特征。

3. 关于IdentityBlock的思考

Identity Block 的特征

在经典的残差网络实现中，一个 Identity Block 通常具有以下特征：

无尺寸或通道变化：在 Identity Block 中，输入和输出的通道数相同，并且通常步幅也为 1。
直接使用输入作为快捷路径：在没有下采样和通道变换的情况下，快捷连接就直接是输入。

相关代码解析

self.shortcut = nn.Sequential()
if in_channels != filters3 or strides != 1:
    self.shortcut = nn.Sequential(
        nn.Conv2d(in_channels, filters3, kernel_size=1, stride=strides),
        nn.BatchNorm2d(filters3)
    )

1. shortcut的定义

• self.shortcut 是一个用于储存快捷连接的神经网络模块（Sequential）。
• 当输入层in_channels 与输出层 filters3 不同，或者 strides 不等于1（即需要下采样）时，表示需要调整输入特征图的尺寸或通道数。
  • 这种情况下，shortcut 被定义为一个卷积层（Conv2d）和一个批量归一化层（BatchNorm2d）的组合。
• 如果通道数和步幅满足条件（即相等且为1），那么 shortcut 直接使用输入 x，实际上不会发生任何变化。

2. Identity Block 的特性

• 身份连接：在残差网络中，身份连接使得输入能直接回馈到输出。当 in_channels 和 filters3 相等，且步幅为1时，shortcut 就会等于输入 x。这正是 Identity Block 的特性，即不对输入进行任何修改。

完整前向传播过程

在 forward 方法中，这段代码实现了身份连接：

def forward(self, x):
    shortcut = self.shortcut(x)  # 使用 shortcut 将 x 进行改动或保持不变

    x = F.relu(self.bn1(self.conv1(x)))  # 卷积操作和激活
    x = F.relu(self.bn2(self.conv2(x)))  # 另一个卷积操作和激活
    x = self.bn3(self.conv3(x))          # 最后的卷积操作

    x += shortcut  # 将调整（或不调整）的 shortcut 加到输出 x 上
    x = F.relu(x)  # 再次进行激活
    return x

• 加法操作（x += shortcut）：这一行实现了残差连接。根据前面定义的逻辑，如果 shortcut 是通过卷积操作得到的结果，那么输入信号经过了变换，以下采样的方式对信号进行匹配。
• 如果 shortcut 是直接使用的输入 x，那么就实现了身份连接，保持了输入特征图的维度和内容。

总结:

你的 ConvBlock 实现中已经巧妙地综合了 ConvBlock 和 Identity Block 的特性。通过对 shortcut 的定义和处理，你的代码在有需要时能够对信号进行变换，而在不需要时则能够保持信号不变。这种灵活性使得你的实现既可以实现残差学习的功能，也符合对身份块的定义。因此，不需要独立声明一个单独的 Identity Block。你的实现已经具备了这种能力。

ResNet-50v2 模型

1.与ResNet-50的区别

关键区别

1. 预激活结构：

   • ResNet50V2 使用了预激活方式（pre-activation block），在执行卷积操作之前先进行 Batch Normalization 和激活函数处理。这种结构使得网络在训练时更平稳，梯度传播更有效。

2. 快捷连接的实现：

   • 在 Block2 中，允许通过卷积层进行快捷连接（conv_shortcut），而不是简单地保持输入不变。这种方式在输出通道数需要变化时能够更好地对齐特征图。

3. 堆叠的结构：

   • Stack2 类负责管理多个 Block2 的堆叠，每个堆叠都在调整通道数和步幅时保持更清晰的结构。这在模型构建时，使得模块更易于组织和理解。

4. 模块化和可重用性：

   • ResNet50V2 更加模块化，各模块之间具有良好的隔离性，这样有助于模型的可读性和重用性。

5. 整体设计：

   • 整体的通道调整和层数设定使得 ResNet50V2 能够在特征提取上更加灵活，适应不同的输入特征。

总结

ResNet50V2 改进了原始 ResNet50 的结构，使之在处理深度学习任务时更加高效。通过预激活、灵活的快捷连接和模块化设计，使得网络能够更好地训练和泛化。这些改进为构建更深更复杂的神经网络打下了基础，也进一步解放了网络结构设计的限制。

2. 各模块分解

Block2 类

• 初始化方法 __init__：

  • 设定了 conv_shortcut 和 stride 等参数，用于控制是否使用卷积快捷连接以及步幅的设置。
  • 预激活（Pre-activation）：使用了 Batch Normalization 和 ReLU 激活函数。预激活是一种改进，有助于缓解深层网络的梯度消失问题。
  • 快捷路径（Shortcut Path）：
    • 如果 conv_shortcut 为真，通过 1x1 卷积调整通道数（4 * filters），以实现跨层连接。
    • 如果步幅不为 1 而 conv_shortcut 为假，则使用最大池化。
  • 该模块的三个卷积层结构沿用了标准的 ResNet 架构。

• 前向传播 forward：

  • 先进行预激活处理（Batch Normalization + ReLU）。
  • 根据设定的 conv_shortcut 决定快捷连接的实现。
  • 将经过卷积的特征图与快捷路径相加，然后返回结果。

Stack2 类

• 在该类中，多个 Block2 被堆叠在一起。
• 第一层使用卷积快捷连接，后续层保持输入和输出通道一致。

ResNet50V2 类

• 初始化方法 __init__：

  • 模型的输入从 3（RGB 图像）开始，经过一系列卷积层和堆栈结构。
  • stack1 到 stack4 各自包含不同数量的 Block2，每层都对应通道数和块数的调整。
  • 使用了标准的卷积后，增加了一个 Batch Normalization 层和 ReLU 激活。

• 前向传播 forward：

  • 输入数据经过了一系列处理，包括卷积、池化、堆叠处理、全局平均池化和最终的全连接层输出。

DenseNet 模型

它的基本思路与ResNet一致，但是它建立的是前面所有层与后面层的密集连接（dense connection），它的名称也是由此而来。DenseNet的另一大特色是通过特征在channel上的连接来实现特征重用（feature reuse）。这些特点让DenseNet在参数和计算成本更少的情形下实现比ResNet更优的性能

1. 设计理念

DenseNet的核心设计理念是“密集连接”（Dense Connectivity）。其目标是通过在每一层与所有前面层的特征图进行连接，来更有效地利用特征，缓解深度网络中的梯度消失问题。具体来说，DenseNet的主要优点包括：

• 特征重用：每一层都直接连接到前面的所有层，使得特征的重用变得更加高效，提高了信息流动性。
• 降低参数数量：通过密集连接，DenseNet显著减少了需要学习的参数数量，从而降低了过拟合的风险。
• 改善梯度流动：更好的梯度流动使得网络训练更加稳定，即使在极深的网络结构中也能保持良好的性能。

2. 网络结构

DenseNet的基本结构由多个稠密块（Dense Block）组成，每个稠密块内的每一层都与前面所有层进行连接。DenseNet的结构通常包含以下几个部分：

• 输入层：输入原始图像。
• 卷积层：初始卷积层用于提取特征。
• 稠密块：由多个卷积层组成，每个卷积层的输出都会被连接到下一层。
• 过渡层（Transition Layer）：通常用于降低特征图的尺寸，采用1x1卷积和平均池化。
• 分类层：最终的全局平均池化层和全连接层进行分类。

以下是DenseNet的一个简单结构示意图：

3. 与CNN和ResNet的对比

• CNN（卷积神经网络）：传统的CNN采用每层单独连接的结构，深度较大时会面临梯度消失和特征梯度消失问题。DenseNet通过密集连接，有效地解决了这些问题。

• ResNet（残差网络）：ResNet引入了残差学习，通过跳跃连接（skip connections）来缓解深度网络中的梯度消失问题。与ResNet不同，DenseNet将每一层都连接起来，这意味着每层的信息都可以从所有前面层中获得。DenseNet通常比同深度的ResNet具有更少的参数，从而提高了计算效率和分类精度。

以下是一个简单的对比图示：

DenseNet通过密集连接层间的特征图，有效地利用了信息流，降低了对参数的需求，并且改善了梯度流动。与传统的CNN和ResNet相比，DenseNet在许多应用上能够提供更好的性能和更高的效率。

4. 代码及关键点解析

让我们逐步分析这段 DenseNet 的代码实现，并详细解析模型的关键特点在代码中的体现。

1. 模块分解

_DenseLayer 类

• 初始化方法 __init__：

  • 创建一个基本的稠密层（Dense Layer），实现了瓶颈结构，主要由两个卷积层和两个 Batch Normalization 层组成。
  • 首先将输入特征通过 Batch Normalization 和 ReLU 激活，然后进行 1x1 卷积，接着经过 Batch Normalization 和 ReLU，再进行 3x3 的卷积。
  • drop_rate 是用来随机失活（Dropout）的概率。

• 前向传播 forward：

  • 先经过 BatchNorm 和 ReLU，再执行 Conv2d 操作。
  • 如果 drop_rate 大于 0，则对新特征进行随机失活。
  • 最后，使用 torch.cat 将原始输入 x 和新产生的特征 new_features 在通道维度上连接。这是 DenseNet 的核心思想之一：每层都可以访问前面所有层的特征。

_DenseBlock 类

• 初始化方法 __init__：
  • 创建一个稠密块（Dense Block），由多个 _DenseLayer 组成。
  • 根据传入的 num_layers 为每层创建一个 _DenseLayer，并将其添加到该块中。

_Transition 类

• 初始化方法 __init__：
  • 实现了稠密块之间的过渡层（Transition Layer），主要用于减少特征图的通道数和进行空间下采样。
  • 包括 Batch Normalization、ReLU、1x1 卷积和 2x2 的平均池化。

DenseNet 类

• 初始化方法 __init__：

  • 初始化 DenseNet 模型的各个部分。
  • 卷积层和最大池化：从输入的 RGB 图片开始，然后进行 7x7 的卷积和 3x3 的最大池化，生成初始特征。
  • DenseBlocks：根据 block_config 指定的结构，多个 _DenseBlock 被堆叠在一起。在每个 Block 之后，如果不是最后一个 Block，会加入一个过渡层。
  • 最终的 Batch Normalization 和 ReLU 处理经过所有 DenseBlocks 的特征。
  • 通过一个全连接层进行分类。

• 参数初始化：使用 Kaiming 初始化方法初始化卷积层，并将 Batch Normalization 和全连接层的偏置和权重进行初始化。

• 前向传播 forward：

  • 将输入 x 传递通过 features 串联的各层。
  • 在最后进行全局平均池化（7x7），将特征展平，并通过分类层输出最终的类概率。

2. DenseNet 关键特点分析

特点 1: 特征重用机制

• DenseNet 通过 torch.cat 将所有前面层的特征连接起来，允许每一层直接访问所有之前层的输出。这种设计能够有效减轻深层网络的退化问题，并增强特征重用的机制。

代码片段：

return torch.cat([x, new_features], 1)  # 在通道维度连接所有特征

特点 2: 瓶颈设计

• 在 _DenseLayer 中使用 1x1 卷积作为瓶颈层，这减少了在特征图上进一步运算需要的计算量和空间复杂度。

self.add_module("conv1", nn.Conv2d(num_input_features, bn_size*growth_rate, kernel_size=1, stride=1, bias=False))

特点 3: 过渡层

• 使用 _Transition 类来控制模型的复杂度和特征图的尺寸，使网络更具灵活性。通过平均池化来降低特征图尺寸，通过卷积减少通道数，防止过拟合。

self.add_module("pool", nn.AvgPool2d(2, stride=2))  # 下采样

特点 4: Dropout 机制

• 在每个稠密层之后可以使用 Dropout 来减少过拟合风险，增加了模型的泛化能力。

if self.drop_rate > 0:
    new_features = F.dropout(new_features, p=self.drop_rate, training=self.training)

特点 5: Globally Average Pooling

• 最终使用全局平均池化，这有助于减少模型的参数并提取全局特征，特别是在分类任务中表现更好。

3. 总结

DenseNet 在结构设计和数据流动态上与传统的卷积神经网络（如 ResNet）有显著差异，具体包括：

• 特征重用：通过所有层的特征连接，增强了特征的流动和重用。
• 瓶颈结构：减小了每个 DenseLayer 的计算负担，提高了计算效率。
• 稠密块和过渡层：通过更细致地控制特征图的生长，保持了模型的紧凑和高效。
• Dropout 和全局平均池化：增强了模型的泛化能力并减少了参数数量。

这些特性使 DenseNet 在处理复杂视觉任务时具有更好的性能，并在一定程度上克服了深度网络常见的退化和过拟合问题。

ResNet 和 DenseNet结合的框架（DPN）

你好！很高兴你决定尝试 Dual Path Network (DPN) 方案。DPN 是一种结合了 ResNet 和 DenseNet 特性的先进神经网络架构，它通过双重路径（残差路径和密集路径）来提高模型的性能和训练稳定性。下面，我将为你详细介绍 DPN 的基本概念，并提供一个简化的 PyTorch 实现代码，适合神经网络初学者理解和使用。

DPN 的基本概念

1. ResNet 的特点

• 残差连接（Residual Connections）：通过直接连接输入和输出，使得梯度能够更容易地流过深层网络，缓解梯度消失问题。
• 简化结构：使用较少的层数实现深度，便于训练。

2. DenseNet 的特点

• 特征重用（Feature Reuse）：每一层利用之前所有层的特征图，使得网络更加高效和有效，增强了特征的传递能力。
• 密集连接（Dense Connections）：每一层都与之前所有层直接相连，促进梯度流动和信息流动。

3. Dual Path Network (DPN) 的特点

• 双重路径（Dual Paths）：结合了 ResNet 的残差连接和 DenseNet 的密集连接，既保留了特征重用，又确保了梯度的高效传播。
• 分离的特征流（Dual Feature Paths）：在 DPN 中，每个模块同时维护两个特征流，一个用于残差连接（ResNet 风格），另一个用于密集连接（DenseNet 风格）。
• 高效的特征传递：通过双重路径，DPN 能够更好地传递和利用特征信息，提高模型的表达能力和性能。

DPN 的架构设计

DPN 的核心组件是 Dual Path Block，每个块包含两个特征流：

1. Residual Path（残差路径）：类似于 ResNet，通过相加操作连接输入和输出。
2. Dense Path（密集路径）：类似于 DenseNet，通过拼接操作将输入和输出连接起来。

Dual Path Block 的工作流程

1. 输入特征图：假设输入特征图的通道数为 C_in。
2. 分支卷积：对输入特征图分别进行 1x1 和 3x3 卷积操作，生成两部分输出：
   • 残差部分（Residual Part）：通过 1x1 卷积降维后，使用 3x3 卷积生成 C_res 通道的特征图。
   • 密集部分（Dense Part）：通过 1x1 卷积降维后，使用 3x3 卷积生成 C_dense 通道的特征图。
3. 特征融合：
   • 残差特征：将生成的 C_res 通道特征图与输入特征图的相应通道相加。
   • 密集特征：将生成的 C_dense 通道特征图与输入特征图在通道维度上拼接，得到新的特征图。
4. 输出特征图：将残差特征和密集特征组合在一起，作为下一层的输入。

DPN 的 PyTorch 实现

下面是一个简化版的 DPN 实现代码，适合初学者理解。代码包含了 DualPathBlock 和整体的 DPN 模型架构。

导入必要的库

import torch
import torch.nn as nn
import torch.nn.functional as F
from collections import OrderedDict

定义 Dual Path Block

class DualPathBlock(nn.Module):
    def __init__(self, in_channels, C_res, C_dense, bn_size=4, drop_rate=0):
        """
        Dual Path Block: 同时包含残差路径和密集路径
        Args:
            in_channels (int): 输入特征图的通道数
            C_res (int): 残差路径输出的通道数
            C_dense (int): 密集路径输出的通道数
            bn_size (int): 1x1 卷积的扩展倍数
            drop_rate (float): Dropout 的概率
        """
        super(DualPathBlock, self).__init__()
        self.drop_rate = drop_rate

        # 残差路径
        self.res_branch = nn.Sequential(
            nn.BatchNorm2d(in_channels),
            nn.ReLU(inplace=True),
            nn.Conv2d(in_channels, bn_size * C_res, kernel_size=1, bias=False),
            nn.BatchNorm2d(bn_size * C_res),
            nn.ReLU(inplace=True),
            nn.Conv2d(bn_size * C_res, C_res, kernel_size=3, padding=1, bias=False)
        )

        # 密集路径
        self.dense_branch = nn.Sequential(
            nn.BatchNorm2d(in_channels),
            nn.ReLU(inplace=True),
            nn.Conv2d(in_channels, bn_size * C_dense, kernel_size=1, bias=False),
            nn.BatchNorm2d(bn_size * C_dense),
            nn.ReLU(inplace=True),
            nn.Conv2d(bn_size * C_dense, C_dense, kernel_size=3, padding=1, bias=False)
        )

        # 如果残差输出通道数与输入不一致，定义一个1x1卷积调整输入通道数
        if C_res != in_channels:
            self.res_identity = nn.Conv2d(in_channels, C_res, kernel_size=1, bias=False)
        else:
            self.res_identity = nn.Identity()

    def forward(self, x):
        res = self.res_branch(x)
        dense = self.dense_branch(x)

        # 残差连接：输入与残差部分相加
        res_identity = self.res_identity(x)
        res_out = res + res_identity

        # 密集连接：输入与密集部分拼接
        dense_out = torch.cat([x, dense], dim=1)

        if self.drop_rate > 0:
            res_out = F.dropout(res_out, p=self.drop_rate, training=self.training)
            dense_out = F.dropout(dense_out, p=self.drop_rate, training=self.training)

        # 将残差输出和密集输出拼接在一起，作为下一个块的输入
        out = torch.cat([res_out, dense_out[:, x.size(1):, :, :]], dim=1)

        return out

定义 DPN 模型

class DPN(nn.Module):
    def __init__(self, num_classes=1000, growth_rate=32, block_config=(6, 12, 24, 16),
                 C_res=(16, 32, 24, 128), C_dense=(16, 32, 24, 128), bn_size=4, drop_rate=0):
        """
        Dual Path Network (DPN) 模型
        Args:
            num_classes (int): 分类类别数
            growth_rate (int): DenseNet 的增长率
            block_config (tuple): 每个阶段的 Dual Path Block 数量
            C_res (tuple): 每个阶段残差路径的输出通道数
            C_dense (tuple): 每个阶段密集路径的输出通道数
            bn_size (int): 1x1 卷积的扩展倍数
            drop_rate (float): Dropout 的概率
        """
        super(DPN, self).__init__()
        self.features = nn.Sequential(OrderedDict([
            ("conv0", nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)),
            ("norm0", nn.BatchNorm2d(64)),
            ("relu0", nn.ReLU(inplace=True)),
            ("pool0", nn.MaxPool2d(kernel_size=3, stride=2, padding=1))
        ]))

        num_channels = 64  # 初始通道数

        # 构造每个阶段的 Dual Path Blocks
        for i, num_blocks in enumerate(block_config):
            for j in range(num_blocks):
                block = DualPathBlock(
                    in_channels=num_channels,
                    C_res=C_res[i],
                    C_dense=C_dense[i],
                    bn_size=bn_size,
                    drop_rate=drop_rate
                )
                self.features.add_module(f"dpn_block{i+1}_{j+1}", block)
                # 更新通道数：残差输出 C_res[i] + 密集输出 C_dense[i]
                num_channels = C_res[i] + C_dense[i]

        # 全局平均池化
        self.features.add_module("avgpool", nn.AdaptiveAvgPool2d((1, 1)))

        # 分类器
        self.classifier = nn.Linear(num_channels, num_classes)

    def forward(self, x):
        x = self.features(x)
        x = torch.flatten(x, 1)  # 展平
        x = self.classifier(x)
        return x

详细解读

a. DualPathBlock 类

• 初始化方法 __init__：

  • 残差路径（Residual Path）：

    • 包含 BatchNorm、ReLU、1x1 卷积、BatchNorm、ReLU 和 3x3 卷积。
    • 输出通道数为 C_res。
    • 如果 C_res 与 in_channels 不一致，通过 1x1 卷积调整输入通道数，使其与 C_res 匹配，便于相加操作。

  • 密集路径（Dense Path）：

    • 包含 BatchNorm、ReLU、1x1 卷积、BatchNorm、ReLU 和 3x3 卷积。
    • 输出通道数为 C_dense。
    • 与输入特征图在通道维度上拼接，增加通道数。

• 前向传播方法 forward：

  • 输入特征图 x 分别通过残差路径和密集路径，生成 res 和 dense。
  • 残差连接：
    • 将生成的残差特征 res 与调整后的输入特征图相加，得到 res_out。
  • 密集连接：
    • 将生成的密集特征 dense 与输入特征图在通道维度上拼接，得到 dense_out。
  • Dropout：
    • 如果 drop_rate 大于 0，则在 res_out 和 dense_out 上应用 Dropout，以减少过拟合。
  • 特征组合：
    • 将 res_out 和 dense_out 的新增通道部分拼接在一起，作为下一个块的输入。

b. DPN 类

• 初始化方法 __init__：

  • 初始卷积层：

    • 包含 7x7 卷积、BatchNorm、ReLU 和最大池化，初始通道数为 64。

  • Dual Path Blocks：

    • 根据 block_config，在每个阶段添加相应数量的 DualPathBlock。
    • 每个块的输入通道数随着块的叠加而变化，等于 C_res[i] + C_dense[i]。

  • 全局平均池化：

    • 将特征图的空间维度缩小到 1x1，减少参数量并增强模型的泛化能力。

  • 分类器：

    • 一个全连接层，将最终的特征图映射到分类类别数。

• 前向传播方法 forward：

  • 输入特征图通过所有的卷积层和 Dual Path Blocks。
  • 通过全局平均池化后展平，最后通过分类器得到输出。

c. 测试模型

• 创建模型实例：初始化 DPN 模型，设置分类类别数为 1000。
• 打印模型结构：展示模型的各层结构，确保所有层都正确定义。
• 创建随机输入张量：模拟一个批量大小为 2 的 224x224 RGB 图像。
• 前向传播：通过模型进行前向传播，检查输出的形状是否正确（应为 [2, 1000]）。

进一步的建议

1. 学习更多 DPN 细节：

   • DPN 的具体实现可能比上述简化版本更复杂，包括更多的细节如不同阶段的通道数配置等。建议阅读相关论文：Dual Path Networks 以深入理解其设计理念。

2. 调试和优化：

   • 初始代码可能需要根据实际需求进行调整和优化。例如，调整 growth_rate、bn_size 和 drop_rate 等超参数以获得更好的性能。
   • 可以在前向传播过程中添加更多的 print 语句，帮助跟踪特征图的形状和通道数，确保模型的正确性。

3. 使用预训练模型：

   • 如果你对 DPN 的训练过程感兴趣，可以考虑使用预训练的 DPN 模型，或者在你的任务上进行微调。

4. 实验与实践：

   • 尝试在不同的数据集和任务上使用 DPN，观察其性能表现，并与其他模型（如 ResNet、DenseNet）进行比较。

总结

Dual Path Network (DPN) 通过结合 ResNet 的残差连接和 DenseNet 的密集连接，提供了一种高效且强大的神经网络架构。上述代码为你提供了一个简化的 DPN 实现示例，并详细解释了各个组件的功能和工作流程。希望这些内容能帮助你更好地理解和应用 DPN 架构。如果有进一步的问题，欢迎随时提问！

二：代码流程

0. 总结

数据导入及处理部分：本次数据导入没有使用torchvision自带的数据集，需要将原始数据进行处理包括数据导入，查看数据分类情况，定义transforms，进行数据类型转换等操作。

划分数据集：划定训练集测试集后，再使用torch.utils.data中的DataLoader()分别加载上一步处理好的训练及测试数据，查看批处理维度.

模型构建部分：Dual Path Network (DPN)

设置超参数：在这之前需要定义损失函数，学习率（动态学习率），以及根据学习率定义优化器（例如SGD随机梯度下降），用来在训练中更新参数，最小化损失函数。

定义训练函数：函数的传入的参数有四个，分别是设置好的DataLoader(),定义好的模型，损失函数，优化器。函数内部初始化损失准确率为0，接着开始循环，使用DataLoader()获取一个批次的数据，对这个批次的数据带入模型得到预测值，然后使用损失函数计算得到损失值。接下来就是进行反向传播以及使用优化器优化参数，梯度清零放在反向传播之前或者是使用优化器优化之后都是可以的，一般是默认放在反向传播之前。

定义测试函数：函数传入的参数相比训练函数少了优化器，只需传入设置好的DataLoader(),定义好的模型，损失函数。此外除了处理批次数据时无需再设置梯度清零、返向传播以及优化器优化参数，其余部分均和训练函数保持一致。

训练过程：定义训练次数，有几次就使用整个数据集进行几次训练，初始化四个空list分别存储每次训练及测试的准确率及损失。使用model.train()开启训练模式，调用训练函数得到准确率及损失。使用model.eval()将模型设置为评估模式，调用测试函数得到准确率及损失。接着就是将得到的训练及测试的准确率及损失存储到相应list中并合并打印出来，得到每一次整体训练后的准确率及损失。

结果可视化

模型的保存，调取及使用。在PyTorch中，通常使用 torch.save(model.state_dict(), ‘model.pth’) 保存模型的参数，使用 model.load_state_dict(torch.load(‘model.pth’)) 加载参数。

需要改进优化的地方：确保模型和数据的一致性，都存到GPU或者CPU;注意numclasses不要直接用默认的1000，需要根据实际数据集改进；实例化模型也要注意numclasses这个参数；此外注意测试模型需要用（3,224,224）3表示通道数，这和tensorflow定义的顺序是不用的（224,224,3），做代码转换时需要注意。

此外该模型调用时batch_size需要调整（32->4）后才能正常运行，如果遇到隐藏文件夹影响文件分类（通常是由数据不规范导入产生的）直接ls - a 查看,然后rm -rf .ipynb_checkpoints删除就好

还有就是需要继续调参，因为初步训练的效果并不好，还不到50%，需进一步研究测试

import torch
import torch.nn as nn
import torchvision
from torchvision import datasets,transforms
from torch.utils.data import DataLoader
import torchvision.models as models
import torch.nn.functional as F
from collections import OrderedDict 


import os,PIL,pathlib
import matplotlib.pyplot as plt
import warnings

warnings.filterwarnings('ignore') # 忽略警告信息

plt.rcParams['font.sans-serif'] = ['SimHei'] # 用来正常显示中文标签
plt.rcParams['axes.unicode_minus'] = False   # 用来正常显示负号
plt.rcParams['figure.dpi'] = 100 # 分辨率

1. 设置GPU

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
device

device(type='cuda')

2. 导入数据及处理部分

# 获取数据分布情况
path_dir = './data/bird_photos/'
path_dir = pathlib.Path(path_dir)

paths = list(path_dir.glob('*'))
# classNames = [str(path).split("\\")[-1] for path in paths] # ['Bananaquit', 'Black Skimmer', 'Black Throated Bushtiti', 'Cockatoo']
classNames = [path.parts[-1] for path in paths]
classNames

['Bananaquit', 'Black Skimmer', 'Black Throated Bushtiti', 'Cockatoo']

# 定义transforms 并处理数据
train_transforms = transforms.Compose([
    transforms.Resize([224,224]),      # 将输入图片resize成统一尺寸
    transforms.RandomHorizontalFlip(), # 随机水平翻转
    transforms.ToTensor(),             # 将PIL Image 或 numpy.ndarray 装换为tensor,并归一化到[0,1]之间
    transforms.Normalize(              # 标准化处理 --> 转换为标准正太分布（高斯分布），使模型更容易收敛
        mean = [0.485,0.456,0.406],    # 其中 mean=[0.485,0.456,0.406]与std=[0.229,0.224,0.225] 从数据集中随机抽样计算得到的。
        std = [0.229,0.224,0.225]
    )
])
test_transforms = transforms.Compose([
    transforms.Resize([224,224]),
    transforms.ToTensor(),
    transforms.Normalize(
        mean = [0.485,0.456,0.406],
        std = [0.229,0.224,0.225]
    )
])
total_data = datasets.ImageFolder('./data/bird_photos/',transform = train_transforms)
total_data

Dataset ImageFolder
    Number of datapoints: 565
    Root location: ./data/bird_photos/
    StandardTransform
Transform: Compose(
               Resize(size=[224, 224], interpolation=bilinear, max_size=None, antialias=True)
               RandomHorizontalFlip(p=0.5)
               ToTensor()
               Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
           )

total_data.class_to_idx

{'Bananaquit': 0,
 'Black Skimmer': 1,
 'Black Throated Bushtiti': 2,
 'Cockatoo': 3}

3. 划分数据集

# 划分数据集
train_size = int(len(total_data) * 0.8)
test_size = len(total_data) - train_size

train_dataset,test_dataset = torch.utils.data.random_split(total_data,[train_size,test_size])
train_dataset,test_dataset

(<torch.utils.data.dataset.Subset at 0x24b5c8a54e0>,
 <torch.utils.data.dataset.Subset at 0x24b5c8a5720>)

# 定义DataLoader用于数据集的加载

batch_size = 4
#batch_size = 32

train_dl = torch.utils.data.DataLoader(
    train_dataset,
    batch_size = batch_size,
    shuffle = True,
    num_workers = 1
)
test_dl = torch.utils.data.DataLoader(
    test_dataset,
    batch_size = batch_size,
    shuffle = True,
    num_workers = 1
)

# 观察数据维度
for X,y in test_dl:
    print("Shape of X [N,C,H,W]: ",X.shape)
    print("Shape of y: ", y.shape,y.dtype)
    break

Shape of X [N,C,H,W]:  torch.Size([32, 3, 224, 224])
Shape of y:  torch.Size([32]) torch.int64

4. 模型构建部分

class DualPathBlock(nn.Module):
    def __init__(self, in_channels, C_res, C_dense, bn_size=4, drop_rate=0):
        """
        Dual Path Block: 同时包含残差路径和密集路径
        Args:
            in_channels (int): 输入特征图的通道数
            C_res (int): 残差路径输出的通道数
            C_dense (int): 密集路径输出的通道数
            bn_size (int): 1x1 卷积的扩展倍数
            drop_rate (float): Dropout 的概率
        """
        super(DualPathBlock, self).__init__()
        self.drop_rate = drop_rate

        # 残差路径
        self.res_branch = nn.Sequential(
            nn.BatchNorm2d(in_channels),
            nn.ReLU(inplace=True),
            nn.Conv2d(in_channels, bn_size * C_res, kernel_size=1, bias=False),
            nn.BatchNorm2d(bn_size * C_res),
            nn.ReLU(inplace=True),
            nn.Conv2d(bn_size * C_res, C_res, kernel_size=3, padding=1, bias=False)
        )

        # 密集路径
        self.dense_branch = nn.Sequential(
            nn.BatchNorm2d(in_channels),
            nn.ReLU(inplace=True),
            nn.Conv2d(in_channels, bn_size * C_dense, kernel_size=1, bias=False),
            nn.BatchNorm2d(bn_size * C_dense),
            nn.ReLU(inplace=True),
            nn.Conv2d(bn_size * C_dense, C_dense, kernel_size=3, padding=1, bias=False)
        )

        # 如果残差输出通道数与输入不一致，定义一个1x1卷积调整输入通道数
        if C_res != in_channels:
            self.res_identity = nn.Conv2d(in_channels, C_res, kernel_size=1, bias=False)
        else:
            self.res_identity = nn.Identity()

    def forward(self, x):
        res = self.res_branch(x)
        dense = self.dense_branch(x)

        # 残差连接：输入与残差部分相加
        res_identity = self.res_identity(x)
        res_out = res + res_identity

        # 密集连接：输入与密集部分拼接
        dense_out = torch.cat([x, dense], dim=1)

        if self.drop_rate > 0:
            res_out = F.dropout(res_out, p=self.drop_rate, training=self.training)
            dense_out = F.dropout(dense_out, p=self.drop_rate, training=self.training)

        # 将残差输出和密集输出拼接在一起，作为下一个块的输入
        out = torch.cat([res_out, dense_out[:, x.size(1):, :, :]], dim=1)

        return out
    
class DPN(nn.Module):
    def __init__(self, num_classes=1000, growth_rate=32, block_config=(6, 12, 24, 16),
                 C_res=(16, 32, 24, 128), C_dense=(16, 32, 24, 128), bn_size=4, drop_rate=0):
        """
        Dual Path Network (DPN) 模型
        Args:
            num_classes (int): 分类类别数
            growth_rate (int): DenseNet 的增长率
            block_config (tuple): 每个阶段的 Dual Path Block 数量
            C_res (tuple): 每个阶段残差路径的输出通道数
            C_dense (tuple): 每个阶段密集路径的输出通道数
            bn_size (int): 1x1 卷积的扩展倍数
            drop_rate (float): Dropout 的概率
        """
        super(DPN, self).__init__()
        self.features = nn.Sequential(OrderedDict([
            ("conv0", nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)),
            ("norm0", nn.BatchNorm2d(64)),
            ("relu0", nn.ReLU(inplace=True)),
            ("pool0", nn.MaxPool2d(kernel_size=3, stride=2, padding=1))
        ]))

        num_channels = 64  # 初始通道数

        # 构造每个阶段的 Dual Path Blocks
        for i, num_blocks in enumerate(block_config):
            for j in range(num_blocks):
                block = DualPathBlock(
                    in_channels=num_channels,
                    C_res=C_res[i],
                    C_dense=C_dense[i],
                    bn_size=bn_size,
                    drop_rate=drop_rate
                )
                self.features.add_module(f"dpn_block{i+1}_{j+1}", block)
                # 更新通道数：残差输出 C_res[i] + 密集输出 C_dense[i]
                num_channels = C_res[i] + C_dense[i]

        # 全局平均池化
        self.features.add_module("avgpool", nn.AdaptiveAvgPool2d((1, 1)))

        # 分类器
        self.classifier = nn.Linear(num_channels, num_classes)

    def forward(self, x):
        x = self.features(x)
        x = torch.flatten(x, 1)  # 展平
        x = self.classifier(x)
        return x

model = DPN(len(classNames)).to(device)
# 4070运行可能爆显存？使用较小的模型或减少模型复杂度
# 初始化 DPN 时，调整参数：
# model = DPN(num_classes=len(classNames), growth_rate=16, block_config=(3, 6, 12, 8),
#             C_res=(16, 32, 24, 64), C_dense=(16, 32, 24, 64), bn_size=4, drop_rate=0).to(device)
print(model)

DPN(
  (features): Sequential(
    (conv0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (norm0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (relu0): ReLU(inplace=True)
    (pool0): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (dpn_block1_1): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block1_2): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block1_3): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block1_4): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block1_5): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block1_6): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(64, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block2_1): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(32, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Identity()
    )
    (dpn_block2_2): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block2_3): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block2_4): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block2_5): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block2_6): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block2_7): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block2_8): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block2_9): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block2_10): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block2_11): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block2_12): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_1): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(64, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(64, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_2): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_3): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_4): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_5): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_6): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_7): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_8): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_9): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_10): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_11): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_12): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_13): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_14): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_15): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_16): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_17): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_18): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_19): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_20): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_21): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_22): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_23): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block3_24): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(96, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_1): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(48, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(48, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_2): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_3): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_4): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_5): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_6): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_7): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_8): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_9): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_10): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_11): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_12): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_13): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_14): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_15): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (dpn_block4_16): DualPathBlock(
      (res_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (dense_branch): Sequential(
        (0): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (1): ReLU(inplace=True)
        (2): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (4): ReLU(inplace=True)
        (5): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      )
      (res_identity): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
  )
  (classifier): Linear(in_features=256, out_features=4, bias=True)
)

# 查看模型详情
import torchsummary as summary
summary.summary(model,(3,224,224))

----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1         [-1, 64, 112, 112]           9,408
       BatchNorm2d-2         [-1, 64, 112, 112]             128
              ReLU-3         [-1, 64, 112, 112]               0
         MaxPool2d-4           [-1, 64, 56, 56]               0
       BatchNorm2d-5           [-1, 64, 56, 56]             128
              ReLU-6           [-1, 64, 56, 56]               0
            Conv2d-7           [-1, 64, 56, 56]           4,096
       BatchNorm2d-8           [-1, 64, 56, 56]             128
              ReLU-9           [-1, 64, 56, 56]               0
           Conv2d-10           [-1, 16, 56, 56]           9,216
      BatchNorm2d-11           [-1, 64, 56, 56]             128
             ReLU-12           [-1, 64, 56, 56]               0
           Conv2d-13           [-1, 64, 56, 56]           4,096
      BatchNorm2d-14           [-1, 64, 56, 56]             128
             ReLU-15           [-1, 64, 56, 56]               0
           Conv2d-16           [-1, 16, 56, 56]           9,216
           Conv2d-17           [-1, 16, 56, 56]           1,024
    DualPathBlock-18           [-1, 32, 56, 56]               0
      BatchNorm2d-19           [-1, 32, 56, 56]              64
             ReLU-20           [-1, 32, 56, 56]               0
           Conv2d-21           [-1, 64, 56, 56]           2,048
      BatchNorm2d-22           [-1, 64, 56, 56]             128
             ReLU-23           [-1, 64, 56, 56]               0
           Conv2d-24           [-1, 16, 56, 56]           9,216
      BatchNorm2d-25           [-1, 32, 56, 56]              64
             ReLU-26           [-1, 32, 56, 56]               0
           Conv2d-27           [-1, 64, 56, 56]           2,048
      BatchNorm2d-28           [-1, 64, 56, 56]             128
             ReLU-29           [-1, 64, 56, 56]               0
           Conv2d-30           [-1, 16, 56, 56]           9,216
           Conv2d-31           [-1, 16, 56, 56]             512
    DualPathBlock-32           [-1, 32, 56, 56]               0
      BatchNorm2d-33           [-1, 32, 56, 56]              64
             ReLU-34           [-1, 32, 56, 56]               0
           Conv2d-35           [-1, 64, 56, 56]           2,048
      BatchNorm2d-36           [-1, 64, 56, 56]             128
             ReLU-37           [-1, 64, 56, 56]               0
           Conv2d-38           [-1, 16, 56, 56]           9,216
      BatchNorm2d-39           [-1, 32, 56, 56]              64
             ReLU-40           [-1, 32, 56, 56]               0
           Conv2d-41           [-1, 64, 56, 56]           2,048
      BatchNorm2d-42           [-1, 64, 56, 56]             128
             ReLU-43           [-1, 64, 56, 56]               0
           Conv2d-44           [-1, 16, 56, 56]           9,216
           Conv2d-45           [-1, 16, 56, 56]             512
    DualPathBlock-46           [-1, 32, 56, 56]               0
      BatchNorm2d-47           [-1, 32, 56, 56]              64
             ReLU-48           [-1, 32, 56, 56]               0
           Conv2d-49           [-1, 64, 56, 56]           2,048
      BatchNorm2d-50           [-1, 64, 56, 56]             128
             ReLU-51           [-1, 64, 56, 56]               0
           Conv2d-52           [-1, 16, 56, 56]           9,216
      BatchNorm2d-53           [-1, 32, 56, 56]              64
             ReLU-54           [-1, 32, 56, 56]               0
           Conv2d-55           [-1, 64, 56, 56]           2,048
      BatchNorm2d-56           [-1, 64, 56, 56]             128
             ReLU-57           [-1, 64, 56, 56]               0
           Conv2d-58           [-1, 16, 56, 56]           9,216
           Conv2d-59           [-1, 16, 56, 56]             512
    DualPathBlock-60           [-1, 32, 56, 56]               0
      BatchNorm2d-61           [-1, 32, 56, 56]              64
             ReLU-62           [-1, 32, 56, 56]               0
           Conv2d-63           [-1, 64, 56, 56]           2,048
      BatchNorm2d-64           [-1, 64, 56, 56]             128
             ReLU-65           [-1, 64, 56, 56]               0
           Conv2d-66           [-1, 16, 56, 56]           9,216
      BatchNorm2d-67           [-1, 32, 56, 56]              64
             ReLU-68           [-1, 32, 56, 56]               0
           Conv2d-69           [-1, 64, 56, 56]           2,048
      BatchNorm2d-70           [-1, 64, 56, 56]             128
             ReLU-71           [-1, 64, 56, 56]               0
           Conv2d-72           [-1, 16, 56, 56]           9,216
           Conv2d-73           [-1, 16, 56, 56]             512
    DualPathBlock-74           [-1, 32, 56, 56]               0
      BatchNorm2d-75           [-1, 32, 56, 56]              64
             ReLU-76           [-1, 32, 56, 56]               0
           Conv2d-77           [-1, 64, 56, 56]           2,048
      BatchNorm2d-78           [-1, 64, 56, 56]             128
             ReLU-79           [-1, 64, 56, 56]               0
           Conv2d-80           [-1, 16, 56, 56]           9,216
      BatchNorm2d-81           [-1, 32, 56, 56]              64
             ReLU-82           [-1, 32, 56, 56]               0
           Conv2d-83           [-1, 64, 56, 56]           2,048
      BatchNorm2d-84           [-1, 64, 56, 56]             128
             ReLU-85           [-1, 64, 56, 56]               0
           Conv2d-86           [-1, 16, 56, 56]           9,216
           Conv2d-87           [-1, 16, 56, 56]             512
    DualPathBlock-88           [-1, 32, 56, 56]               0
      BatchNorm2d-89           [-1, 32, 56, 56]              64
             ReLU-90           [-1, 32, 56, 56]               0
           Conv2d-91          [-1, 128, 56, 56]           4,096
      BatchNorm2d-92          [-1, 128, 56, 56]             256
             ReLU-93          [-1, 128, 56, 56]               0
           Conv2d-94           [-1, 32, 56, 56]          36,864
      BatchNorm2d-95           [-1, 32, 56, 56]              64
             ReLU-96           [-1, 32, 56, 56]               0
           Conv2d-97          [-1, 128, 56, 56]           4,096
      BatchNorm2d-98          [-1, 128, 56, 56]             256
             ReLU-99          [-1, 128, 56, 56]               0
          Conv2d-100           [-1, 32, 56, 56]          36,864
        Identity-101           [-1, 32, 56, 56]               0
   DualPathBlock-102           [-1, 64, 56, 56]               0
     BatchNorm2d-103           [-1, 64, 56, 56]             128
            ReLU-104           [-1, 64, 56, 56]               0
          Conv2d-105          [-1, 128, 56, 56]           8,192
     BatchNorm2d-106          [-1, 128, 56, 56]             256
            ReLU-107          [-1, 128, 56, 56]               0
          Conv2d-108           [-1, 32, 56, 56]          36,864
     BatchNorm2d-109           [-1, 64, 56, 56]             128
            ReLU-110           [-1, 64, 56, 56]               0
          Conv2d-111          [-1, 128, 56, 56]           8,192
     BatchNorm2d-112          [-1, 128, 56, 56]             256
            ReLU-113          [-1, 128, 56, 56]               0
          Conv2d-114           [-1, 32, 56, 56]          36,864
          Conv2d-115           [-1, 32, 56, 56]           2,048
   DualPathBlock-116           [-1, 64, 56, 56]               0
     BatchNorm2d-117           [-1, 64, 56, 56]             128
            ReLU-118           [-1, 64, 56, 56]               0
          Conv2d-119          [-1, 128, 56, 56]           8,192
     BatchNorm2d-120          [-1, 128, 56, 56]             256
            ReLU-121          [-1, 128, 56, 56]               0
          Conv2d-122           [-1, 32, 56, 56]          36,864
     BatchNorm2d-123           [-1, 64, 56, 56]             128
            ReLU-124           [-1, 64, 56, 56]               0
          Conv2d-125          [-1, 128, 56, 56]           8,192
     BatchNorm2d-126          [-1, 128, 56, 56]             256
            ReLU-127          [-1, 128, 56, 56]               0
          Conv2d-128           [-1, 32, 56, 56]          36,864
          Conv2d-129           [-1, 32, 56, 56]           2,048
   DualPathBlock-130           [-1, 64, 56, 56]               0
     BatchNorm2d-131           [-1, 64, 56, 56]             128
            ReLU-132           [-1, 64, 56, 56]               0
          Conv2d-133          [-1, 128, 56, 56]           8,192
     BatchNorm2d-134          [-1, 128, 56, 56]             256
            ReLU-135          [-1, 128, 56, 56]               0
          Conv2d-136           [-1, 32, 56, 56]          36,864
     BatchNorm2d-137           [-1, 64, 56, 56]             128
            ReLU-138           [-1, 64, 56, 56]               0
          Conv2d-139          [-1, 128, 56, 56]           8,192
     BatchNorm2d-140          [-1, 128, 56, 56]             256
            ReLU-141          [-1, 128, 56, 56]               0
          Conv2d-142           [-1, 32, 56, 56]          36,864
          Conv2d-143           [-1, 32, 56, 56]           2,048
   DualPathBlock-144           [-1, 64, 56, 56]               0
     BatchNorm2d-145           [-1, 64, 56, 56]             128
            ReLU-146           [-1, 64, 56, 56]               0
          Conv2d-147          [-1, 128, 56, 56]           8,192
     BatchNorm2d-148          [-1, 128, 56, 56]             256
            ReLU-149          [-1, 128, 56, 56]               0
          Conv2d-150           [-1, 32, 56, 56]          36,864
     BatchNorm2d-151           [-1, 64, 56, 56]             128
            ReLU-152           [-1, 64, 56, 56]               0
          Conv2d-153          [-1, 128, 56, 56]           8,192
     BatchNorm2d-154          [-1, 128, 56, 56]             256
            ReLU-155          [-1, 128, 56, 56]               0
          Conv2d-156           [-1, 32, 56, 56]          36,864
          Conv2d-157           [-1, 32, 56, 56]           2,048
   DualPathBlock-158           [-1, 64, 56, 56]               0
     BatchNorm2d-159           [-1, 64, 56, 56]             128
            ReLU-160           [-1, 64, 56, 56]               0
          Conv2d-161          [-1, 128, 56, 56]           8,192
     BatchNorm2d-162          [-1, 128, 56, 56]             256
            ReLU-163          [-1, 128, 56, 56]               0
          Conv2d-164           [-1, 32, 56, 56]          36,864
     BatchNorm2d-165           [-1, 64, 56, 56]             128
            ReLU-166           [-1, 64, 56, 56]               0
          Conv2d-167          [-1, 128, 56, 56]           8,192
     BatchNorm2d-168          [-1, 128, 56, 56]             256
            ReLU-169          [-1, 128, 56, 56]               0
          Conv2d-170           [-1, 32, 56, 56]          36,864
          Conv2d-171           [-1, 32, 56, 56]           2,048
   DualPathBlock-172           [-1, 64, 56, 56]               0
     BatchNorm2d-173           [-1, 64, 56, 56]             128
            ReLU-174           [-1, 64, 56, 56]               0
          Conv2d-175          [-1, 128, 56, 56]           8,192
     BatchNorm2d-176          [-1, 128, 56, 56]             256
            ReLU-177          [-1, 128, 56, 56]               0
          Conv2d-178           [-1, 32, 56, 56]          36,864
     BatchNorm2d-179           [-1, 64, 56, 56]             128
            ReLU-180           [-1, 64, 56, 56]               0
          Conv2d-181          [-1, 128, 56, 56]           8,192
     BatchNorm2d-182          [-1, 128, 56, 56]             256
            ReLU-183          [-1, 128, 56, 56]               0
          Conv2d-184           [-1, 32, 56, 56]          36,864
          Conv2d-185           [-1, 32, 56, 56]           2,048
   DualPathBlock-186           [-1, 64, 56, 56]               0
     BatchNorm2d-187           [-1, 64, 56, 56]             128
            ReLU-188           [-1, 64, 56, 56]               0
          Conv2d-189          [-1, 128, 56, 56]           8,192
     BatchNorm2d-190          [-1, 128, 56, 56]             256
            ReLU-191          [-1, 128, 56, 56]               0
          Conv2d-192           [-1, 32, 56, 56]          36,864
     BatchNorm2d-193           [-1, 64, 56, 56]             128
            ReLU-194           [-1, 64, 56, 56]               0
          Conv2d-195          [-1, 128, 56, 56]           8,192
     BatchNorm2d-196          [-1, 128, 56, 56]             256
            ReLU-197          [-1, 128, 56, 56]               0
          Conv2d-198           [-1, 32, 56, 56]          36,864
          Conv2d-199           [-1, 32, 56, 56]           2,048
   DualPathBlock-200           [-1, 64, 56, 56]               0
     BatchNorm2d-201           [-1, 64, 56, 56]             128
            ReLU-202           [-1, 64, 56, 56]               0
          Conv2d-203          [-1, 128, 56, 56]           8,192
     BatchNorm2d-204          [-1, 128, 56, 56]             256
            ReLU-205          [-1, 128, 56, 56]               0
          Conv2d-206           [-1, 32, 56, 56]          36,864
     BatchNorm2d-207           [-1, 64, 56, 56]             128
            ReLU-208           [-1, 64, 56, 56]               0
          Conv2d-209          [-1, 128, 56, 56]           8,192
     BatchNorm2d-210          [-1, 128, 56, 56]             256
            ReLU-211          [-1, 128, 56, 56]               0
          Conv2d-212           [-1, 32, 56, 56]          36,864
          Conv2d-213           [-1, 32, 56, 56]           2,048
   DualPathBlock-214           [-1, 64, 56, 56]               0
     BatchNorm2d-215           [-1, 64, 56, 56]             128
            ReLU-216           [-1, 64, 56, 56]               0
          Conv2d-217          [-1, 128, 56, 56]           8,192
     BatchNorm2d-218          [-1, 128, 56, 56]             256
            ReLU-219          [-1, 128, 56, 56]               0
          Conv2d-220           [-1, 32, 56, 56]          36,864
     BatchNorm2d-221           [-1, 64, 56, 56]             128
            ReLU-222           [-1, 64, 56, 56]               0
          Conv2d-223          [-1, 128, 56, 56]           8,192
     BatchNorm2d-224          [-1, 128, 56, 56]             256
            ReLU-225          [-1, 128, 56, 56]               0
          Conv2d-226           [-1, 32, 56, 56]          36,864
          Conv2d-227           [-1, 32, 56, 56]           2,048
   DualPathBlock-228           [-1, 64, 56, 56]               0
     BatchNorm2d-229           [-1, 64, 56, 56]             128
            ReLU-230           [-1, 64, 56, 56]               0
          Conv2d-231          [-1, 128, 56, 56]           8,192
     BatchNorm2d-232          [-1, 128, 56, 56]             256
            ReLU-233          [-1, 128, 56, 56]               0
          Conv2d-234           [-1, 32, 56, 56]          36,864
     BatchNorm2d-235           [-1, 64, 56, 56]             128
            ReLU-236           [-1, 64, 56, 56]               0
          Conv2d-237          [-1, 128, 56, 56]           8,192
     BatchNorm2d-238          [-1, 128, 56, 56]             256
            ReLU-239          [-1, 128, 56, 56]               0
          Conv2d-240           [-1, 32, 56, 56]          36,864
          Conv2d-241           [-1, 32, 56, 56]           2,048
   DualPathBlock-242           [-1, 64, 56, 56]               0
     BatchNorm2d-243           [-1, 64, 56, 56]             128
            ReLU-244           [-1, 64, 56, 56]               0
          Conv2d-245          [-1, 128, 56, 56]           8,192
     BatchNorm2d-246          [-1, 128, 56, 56]             256
            ReLU-247          [-1, 128, 56, 56]               0
          Conv2d-248           [-1, 32, 56, 56]          36,864
     BatchNorm2d-249           [-1, 64, 56, 56]             128
            ReLU-250           [-1, 64, 56, 56]               0
          Conv2d-251          [-1, 128, 56, 56]           8,192
     BatchNorm2d-252          [-1, 128, 56, 56]             256
            ReLU-253          [-1, 128, 56, 56]               0
          Conv2d-254           [-1, 32, 56, 56]          36,864
          Conv2d-255           [-1, 32, 56, 56]           2,048
   DualPathBlock-256           [-1, 64, 56, 56]               0
     BatchNorm2d-257           [-1, 64, 56, 56]             128
            ReLU-258           [-1, 64, 56, 56]               0
          Conv2d-259           [-1, 96, 56, 56]           6,144
     BatchNorm2d-260           [-1, 96, 56, 56]             192
            ReLU-261           [-1, 96, 56, 56]               0
          Conv2d-262           [-1, 24, 56, 56]          20,736
     BatchNorm2d-263           [-1, 64, 56, 56]             128
            ReLU-264           [-1, 64, 56, 56]               0
          Conv2d-265           [-1, 96, 56, 56]           6,144
     BatchNorm2d-266           [-1, 96, 56, 56]             192
            ReLU-267           [-1, 96, 56, 56]               0
          Conv2d-268           [-1, 24, 56, 56]          20,736
          Conv2d-269           [-1, 24, 56, 56]           1,536
   DualPathBlock-270           [-1, 48, 56, 56]               0
     BatchNorm2d-271           [-1, 48, 56, 56]              96
            ReLU-272           [-1, 48, 56, 56]               0
          Conv2d-273           [-1, 96, 56, 56]           4,608
     BatchNorm2d-274           [-1, 96, 56, 56]             192
            ReLU-275           [-1, 96, 56, 56]               0
          Conv2d-276           [-1, 24, 56, 56]          20,736
     BatchNorm2d-277           [-1, 48, 56, 56]              96
            ReLU-278           [-1, 48, 56, 56]               0
          Conv2d-279           [-1, 96, 56, 56]           4,608
     BatchNorm2d-280           [-1, 96, 56, 56]             192
            ReLU-281           [-1, 96, 56, 56]               0
          Conv2d-282           [-1, 24, 56, 56]          20,736
          Conv2d-283           [-1, 24, 56, 56]           1,152
   DualPathBlock-284           [-1, 48, 56, 56]               0
     BatchNorm2d-285           [-1, 48, 56, 56]              96
            ReLU-286           [-1, 48, 56, 56]               0
          Conv2d-287           [-1, 96, 56, 56]           4,608
     BatchNorm2d-288           [-1, 96, 56, 56]             192
            ReLU-289           [-1, 96, 56, 56]               0
          Conv2d-290           [-1, 24, 56, 56]          20,736
     BatchNorm2d-291           [-1, 48, 56, 56]              96
            ReLU-292           [-1, 48, 56, 56]               0
          Conv2d-293           [-1, 96, 56, 56]           4,608
     BatchNorm2d-294           [-1, 96, 56, 56]             192
            ReLU-295           [-1, 96, 56, 56]               0
          Conv2d-296           [-1, 24, 56, 56]          20,736
          Conv2d-297           [-1, 24, 56, 56]           1,152
   DualPathBlock-298           [-1, 48, 56, 56]               0
     BatchNorm2d-299           [-1, 48, 56, 56]              96
            ReLU-300           [-1, 48, 56, 56]               0
          Conv2d-301           [-1, 96, 56, 56]           4,608
     BatchNorm2d-302           [-1, 96, 56, 56]             192
            ReLU-303           [-1, 96, 56, 56]               0
          Conv2d-304           [-1, 24, 56, 56]          20,736
     BatchNorm2d-305           [-1, 48, 56, 56]              96
            ReLU-306           [-1, 48, 56, 56]               0
          Conv2d-307           [-1, 96, 56, 56]           4,608
     BatchNorm2d-308           [-1, 96, 56, 56]             192
            ReLU-309           [-1, 96, 56, 56]               0
          Conv2d-310           [-1, 24, 56, 56]          20,736
          Conv2d-311           [-1, 24, 56, 56]           1,152
   DualPathBlock-312           [-1, 48, 56, 56]               0
     BatchNorm2d-313           [-1, 48, 56, 56]              96
            ReLU-314           [-1, 48, 56, 56]               0
          Conv2d-315           [-1, 96, 56, 56]           4,608
     BatchNorm2d-316           [-1, 96, 56, 56]             192
            ReLU-317           [-1, 96, 56, 56]               0
          Conv2d-318           [-1, 24, 56, 56]          20,736
     BatchNorm2d-319           [-1, 48, 56, 56]              96
            ReLU-320           [-1, 48, 56, 56]               0
          Conv2d-321           [-1, 96, 56, 56]           4,608
     BatchNorm2d-322           [-1, 96, 56, 56]             192
            ReLU-323           [-1, 96, 56, 56]               0
          Conv2d-324           [-1, 24, 56, 56]          20,736
          Conv2d-325           [-1, 24, 56, 56]           1,152
   DualPathBlock-326           [-1, 48, 56, 56]               0
     BatchNorm2d-327           [-1, 48, 56, 56]              96
            ReLU-328           [-1, 48, 56, 56]               0
          Conv2d-329           [-1, 96, 56, 56]           4,608
     BatchNorm2d-330           [-1, 96, 56, 56]             192
            ReLU-331           [-1, 96, 56, 56]               0
          Conv2d-332           [-1, 24, 56, 56]          20,736
     BatchNorm2d-333           [-1, 48, 56, 56]              96
            ReLU-334           [-1, 48, 56, 56]               0
          Conv2d-335           [-1, 96, 56, 56]           4,608
     BatchNorm2d-336           [-1, 96, 56, 56]             192
            ReLU-337           [-1, 96, 56, 56]               0
          Conv2d-338           [-1, 24, 56, 56]          20,736
          Conv2d-339           [-1, 24, 56, 56]           1,152
   DualPathBlock-340           [-1, 48, 56, 56]               0
     BatchNorm2d-341           [-1, 48, 56, 56]              96
            ReLU-342           [-1, 48, 56, 56]               0
          Conv2d-343           [-1, 96, 56, 56]           4,608
     BatchNorm2d-344           [-1, 96, 56, 56]             192
            ReLU-345           [-1, 96, 56, 56]               0
          Conv2d-346           [-1, 24, 56, 56]          20,736
     BatchNorm2d-347           [-1, 48, 56, 56]              96
            ReLU-348           [-1, 48, 56, 56]               0
          Conv2d-349           [-1, 96, 56, 56]           4,608
     BatchNorm2d-350           [-1, 96, 56, 56]             192
            ReLU-351           [-1, 96, 56, 56]               0
          Conv2d-352           [-1, 24, 56, 56]          20,736
          Conv2d-353           [-1, 24, 56, 56]           1,152
   DualPathBlock-354           [-1, 48, 56, 56]               0
     BatchNorm2d-355           [-1, 48, 56, 56]              96
            ReLU-356           [-1, 48, 56, 56]               0
          Conv2d-357           [-1, 96, 56, 56]           4,608
     BatchNorm2d-358           [-1, 96, 56, 56]             192
            ReLU-359           [-1, 96, 56, 56]               0
          Conv2d-360           [-1, 24, 56, 56]          20,736
     BatchNorm2d-361           [-1, 48, 56, 56]              96
            ReLU-362           [-1, 48, 56, 56]               0
          Conv2d-363           [-1, 96, 56, 56]           4,608
     BatchNorm2d-364           [-1, 96, 56, 56]             192
            ReLU-365           [-1, 96, 56, 56]               0
          Conv2d-366           [-1, 24, 56, 56]          20,736
          Conv2d-367           [-1, 24, 56, 56]           1,152
   DualPathBlock-368           [-1, 48, 56, 56]               0
     BatchNorm2d-369           [-1, 48, 56, 56]              96
            ReLU-370           [-1, 48, 56, 56]               0
          Conv2d-371           [-1, 96, 56, 56]           4,608
     BatchNorm2d-372           [-1, 96, 56, 56]             192
            ReLU-373           [-1, 96, 56, 56]               0
          Conv2d-374           [-1, 24, 56, 56]          20,736
     BatchNorm2d-375           [-1, 48, 56, 56]              96
            ReLU-376           [-1, 48, 56, 56]               0
          Conv2d-377           [-1, 96, 56, 56]           4,608
     BatchNorm2d-378           [-1, 96, 56, 56]             192
            ReLU-379           [-1, 96, 56, 56]               0
          Conv2d-380           [-1, 24, 56, 56]          20,736
          Conv2d-381           [-1, 24, 56, 56]           1,152
   DualPathBlock-382           [-1, 48, 56, 56]               0
     BatchNorm2d-383           [-1, 48, 56, 56]              96
            ReLU-384           [-1, 48, 56, 56]               0
          Conv2d-385           [-1, 96, 56, 56]           4,608
     BatchNorm2d-386           [-1, 96, 56, 56]             192
            ReLU-387           [-1, 96, 56, 56]               0
          Conv2d-388           [-1, 24, 56, 56]          20,736
     BatchNorm2d-389           [-1, 48, 56, 56]              96
            ReLU-390           [-1, 48, 56, 56]               0
          Conv2d-391           [-1, 96, 56, 56]           4,608
     BatchNorm2d-392           [-1, 96, 56, 56]             192
            ReLU-393           [-1, 96, 56, 56]               0
          Conv2d-394           [-1, 24, 56, 56]          20,736
          Conv2d-395           [-1, 24, 56, 56]           1,152
   DualPathBlock-396           [-1, 48, 56, 56]               0
     BatchNorm2d-397           [-1, 48, 56, 56]              96
            ReLU-398           [-1, 48, 56, 56]               0
          Conv2d-399           [-1, 96, 56, 56]           4,608
     BatchNorm2d-400           [-1, 96, 56, 56]             192
            ReLU-401           [-1, 96, 56, 56]               0
          Conv2d-402           [-1, 24, 56, 56]          20,736
     BatchNorm2d-403           [-1, 48, 56, 56]              96
            ReLU-404           [-1, 48, 56, 56]               0
          Conv2d-405           [-1, 96, 56, 56]           4,608
     BatchNorm2d-406           [-1, 96, 56, 56]             192
            ReLU-407           [-1, 96, 56, 56]               0
          Conv2d-408           [-1, 24, 56, 56]          20,736
          Conv2d-409           [-1, 24, 56, 56]           1,152
   DualPathBlock-410           [-1, 48, 56, 56]               0
     BatchNorm2d-411           [-1, 48, 56, 56]              96
            ReLU-412           [-1, 48, 56, 56]               0
          Conv2d-413           [-1, 96, 56, 56]           4,608
     BatchNorm2d-414           [-1, 96, 56, 56]             192
            ReLU-415           [-1, 96, 56, 56]               0
          Conv2d-416           [-1, 24, 56, 56]          20,736
     BatchNorm2d-417           [-1, 48, 56, 56]              96
            ReLU-418           [-1, 48, 56, 56]               0
          Conv2d-419           [-1, 96, 56, 56]           4,608
     BatchNorm2d-420           [-1, 96, 56, 56]             192
            ReLU-421           [-1, 96, 56, 56]               0
          Conv2d-422           [-1, 24, 56, 56]          20,736
          Conv2d-423           [-1, 24, 56, 56]           1,152
   DualPathBlock-424           [-1, 48, 56, 56]               0
     BatchNorm2d-425           [-1, 48, 56, 56]              96
            ReLU-426           [-1, 48, 56, 56]               0
          Conv2d-427           [-1, 96, 56, 56]           4,608
     BatchNorm2d-428           [-1, 96, 56, 56]             192
            ReLU-429           [-1, 96, 56, 56]               0
          Conv2d-430           [-1, 24, 56, 56]          20,736
     BatchNorm2d-431           [-1, 48, 56, 56]              96
            ReLU-432           [-1, 48, 56, 56]               0
          Conv2d-433           [-1, 96, 56, 56]           4,608
     BatchNorm2d-434           [-1, 96, 56, 56]             192
            ReLU-435           [-1, 96, 56, 56]               0
          Conv2d-436           [-1, 24, 56, 56]          20,736
          Conv2d-437           [-1, 24, 56, 56]           1,152
   DualPathBlock-438           [-1, 48, 56, 56]               0
     BatchNorm2d-439           [-1, 48, 56, 56]              96
            ReLU-440           [-1, 48, 56, 56]               0
          Conv2d-441           [-1, 96, 56, 56]           4,608
     BatchNorm2d-442           [-1, 96, 56, 56]             192
            ReLU-443           [-1, 96, 56, 56]               0
          Conv2d-444           [-1, 24, 56, 56]          20,736
     BatchNorm2d-445           [-1, 48, 56, 56]              96
            ReLU-446           [-1, 48, 56, 56]               0
          Conv2d-447           [-1, 96, 56, 56]           4,608
     BatchNorm2d-448           [-1, 96, 56, 56]             192
            ReLU-449           [-1, 96, 56, 56]               0
          Conv2d-450           [-1, 24, 56, 56]          20,736
          Conv2d-451           [-1, 24, 56, 56]           1,152
   DualPathBlock-452           [-1, 48, 56, 56]               0
     BatchNorm2d-453           [-1, 48, 56, 56]              96
            ReLU-454           [-1, 48, 56, 56]               0
          Conv2d-455           [-1, 96, 56, 56]           4,608
     BatchNorm2d-456           [-1, 96, 56, 56]             192
            ReLU-457           [-1, 96, 56, 56]               0
          Conv2d-458           [-1, 24, 56, 56]          20,736
     BatchNorm2d-459           [-1, 48, 56, 56]              96
            ReLU-460           [-1, 48, 56, 56]               0
          Conv2d-461           [-1, 96, 56, 56]           4,608
     BatchNorm2d-462           [-1, 96, 56, 56]             192
            ReLU-463           [-1, 96, 56, 56]               0
          Conv2d-464           [-1, 24, 56, 56]          20,736
          Conv2d-465           [-1, 24, 56, 56]           1,152
   DualPathBlock-466           [-1, 48, 56, 56]               0
     BatchNorm2d-467           [-1, 48, 56, 56]              96
            ReLU-468           [-1, 48, 56, 56]               0
          Conv2d-469           [-1, 96, 56, 56]           4,608
     BatchNorm2d-470           [-1, 96, 56, 56]             192
            ReLU-471           [-1, 96, 56, 56]               0
          Conv2d-472           [-1, 24, 56, 56]          20,736
     BatchNorm2d-473           [-1, 48, 56, 56]              96
            ReLU-474           [-1, 48, 56, 56]               0
          Conv2d-475           [-1, 96, 56, 56]           4,608
     BatchNorm2d-476           [-1, 96, 56, 56]             192
            ReLU-477           [-1, 96, 56, 56]               0
          Conv2d-478           [-1, 24, 56, 56]          20,736
          Conv2d-479           [-1, 24, 56, 56]           1,152
   DualPathBlock-480           [-1, 48, 56, 56]               0
     BatchNorm2d-481           [-1, 48, 56, 56]              96
            ReLU-482           [-1, 48, 56, 56]               0
          Conv2d-483           [-1, 96, 56, 56]           4,608
     BatchNorm2d-484           [-1, 96, 56, 56]             192
            ReLU-485           [-1, 96, 56, 56]               0
          Conv2d-486           [-1, 24, 56, 56]          20,736
     BatchNorm2d-487           [-1, 48, 56, 56]              96
            ReLU-488           [-1, 48, 56, 56]               0
          Conv2d-489           [-1, 96, 56, 56]           4,608
     BatchNorm2d-490           [-1, 96, 56, 56]             192
            ReLU-491           [-1, 96, 56, 56]               0
          Conv2d-492           [-1, 24, 56, 56]          20,736
          Conv2d-493           [-1, 24, 56, 56]           1,152
   DualPathBlock-494           [-1, 48, 56, 56]               0
     BatchNorm2d-495           [-1, 48, 56, 56]              96
            ReLU-496           [-1, 48, 56, 56]               0
          Conv2d-497           [-1, 96, 56, 56]           4,608
     BatchNorm2d-498           [-1, 96, 56, 56]             192
            ReLU-499           [-1, 96, 56, 56]               0
          Conv2d-500           [-1, 24, 56, 56]          20,736
     BatchNorm2d-501           [-1, 48, 56, 56]              96
            ReLU-502           [-1, 48, 56, 56]               0
          Conv2d-503           [-1, 96, 56, 56]           4,608
     BatchNorm2d-504           [-1, 96, 56, 56]             192
            ReLU-505           [-1, 96, 56, 56]               0
          Conv2d-506           [-1, 24, 56, 56]          20,736
          Conv2d-507           [-1, 24, 56, 56]           1,152
   DualPathBlock-508           [-1, 48, 56, 56]               0
     BatchNorm2d-509           [-1, 48, 56, 56]              96
            ReLU-510           [-1, 48, 56, 56]               0
          Conv2d-511           [-1, 96, 56, 56]           4,608
     BatchNorm2d-512           [-1, 96, 56, 56]             192
            ReLU-513           [-1, 96, 56, 56]               0
          Conv2d-514           [-1, 24, 56, 56]          20,736
     BatchNorm2d-515           [-1, 48, 56, 56]              96
            ReLU-516           [-1, 48, 56, 56]               0
          Conv2d-517           [-1, 96, 56, 56]           4,608
     BatchNorm2d-518           [-1, 96, 56, 56]             192
            ReLU-519           [-1, 96, 56, 56]               0
          Conv2d-520           [-1, 24, 56, 56]          20,736
          Conv2d-521           [-1, 24, 56, 56]           1,152
   DualPathBlock-522           [-1, 48, 56, 56]               0
     BatchNorm2d-523           [-1, 48, 56, 56]              96
            ReLU-524           [-1, 48, 56, 56]               0
          Conv2d-525           [-1, 96, 56, 56]           4,608
     BatchNorm2d-526           [-1, 96, 56, 56]             192
            ReLU-527           [-1, 96, 56, 56]               0
          Conv2d-528           [-1, 24, 56, 56]          20,736
     BatchNorm2d-529           [-1, 48, 56, 56]              96
            ReLU-530           [-1, 48, 56, 56]               0
          Conv2d-531           [-1, 96, 56, 56]           4,608
     BatchNorm2d-532           [-1, 96, 56, 56]             192
            ReLU-533           [-1, 96, 56, 56]               0
          Conv2d-534           [-1, 24, 56, 56]          20,736
          Conv2d-535           [-1, 24, 56, 56]           1,152
   DualPathBlock-536           [-1, 48, 56, 56]               0
     BatchNorm2d-537           [-1, 48, 56, 56]              96
            ReLU-538           [-1, 48, 56, 56]               0
          Conv2d-539           [-1, 96, 56, 56]           4,608
     BatchNorm2d-540           [-1, 96, 56, 56]             192
            ReLU-541           [-1, 96, 56, 56]               0
          Conv2d-542           [-1, 24, 56, 56]          20,736
     BatchNorm2d-543           [-1, 48, 56, 56]              96
            ReLU-544           [-1, 48, 56, 56]               0
          Conv2d-545           [-1, 96, 56, 56]           4,608
     BatchNorm2d-546           [-1, 96, 56, 56]             192
            ReLU-547           [-1, 96, 56, 56]               0
          Conv2d-548           [-1, 24, 56, 56]          20,736
          Conv2d-549           [-1, 24, 56, 56]           1,152
   DualPathBlock-550           [-1, 48, 56, 56]               0
     BatchNorm2d-551           [-1, 48, 56, 56]              96
            ReLU-552           [-1, 48, 56, 56]               0
          Conv2d-553           [-1, 96, 56, 56]           4,608
     BatchNorm2d-554           [-1, 96, 56, 56]             192
            ReLU-555           [-1, 96, 56, 56]               0
          Conv2d-556           [-1, 24, 56, 56]          20,736
     BatchNorm2d-557           [-1, 48, 56, 56]              96
            ReLU-558           [-1, 48, 56, 56]               0
          Conv2d-559           [-1, 96, 56, 56]           4,608
     BatchNorm2d-560           [-1, 96, 56, 56]             192
            ReLU-561           [-1, 96, 56, 56]               0
          Conv2d-562           [-1, 24, 56, 56]          20,736
          Conv2d-563           [-1, 24, 56, 56]           1,152
   DualPathBlock-564           [-1, 48, 56, 56]               0
     BatchNorm2d-565           [-1, 48, 56, 56]              96
            ReLU-566           [-1, 48, 56, 56]               0
          Conv2d-567           [-1, 96, 56, 56]           4,608
     BatchNorm2d-568           [-1, 96, 56, 56]             192
            ReLU-569           [-1, 96, 56, 56]               0
          Conv2d-570           [-1, 24, 56, 56]          20,736
     BatchNorm2d-571           [-1, 48, 56, 56]              96
            ReLU-572           [-1, 48, 56, 56]               0
          Conv2d-573           [-1, 96, 56, 56]           4,608
     BatchNorm2d-574           [-1, 96, 56, 56]             192
            ReLU-575           [-1, 96, 56, 56]               0
          Conv2d-576           [-1, 24, 56, 56]          20,736
          Conv2d-577           [-1, 24, 56, 56]           1,152
   DualPathBlock-578           [-1, 48, 56, 56]               0
     BatchNorm2d-579           [-1, 48, 56, 56]              96
            ReLU-580           [-1, 48, 56, 56]               0
          Conv2d-581           [-1, 96, 56, 56]           4,608
     BatchNorm2d-582           [-1, 96, 56, 56]             192
            ReLU-583           [-1, 96, 56, 56]               0
          Conv2d-584           [-1, 24, 56, 56]          20,736
     BatchNorm2d-585           [-1, 48, 56, 56]              96
            ReLU-586           [-1, 48, 56, 56]               0
          Conv2d-587           [-1, 96, 56, 56]           4,608
     BatchNorm2d-588           [-1, 96, 56, 56]             192
            ReLU-589           [-1, 96, 56, 56]               0
          Conv2d-590           [-1, 24, 56, 56]          20,736
          Conv2d-591           [-1, 24, 56, 56]           1,152
   DualPathBlock-592           [-1, 48, 56, 56]               0
     BatchNorm2d-593           [-1, 48, 56, 56]              96
            ReLU-594           [-1, 48, 56, 56]               0
          Conv2d-595          [-1, 512, 56, 56]          24,576
     BatchNorm2d-596          [-1, 512, 56, 56]           1,024
            ReLU-597          [-1, 512, 56, 56]               0
          Conv2d-598          [-1, 128, 56, 56]         589,824
     BatchNorm2d-599           [-1, 48, 56, 56]              96
            ReLU-600           [-1, 48, 56, 56]               0
          Conv2d-601          [-1, 512, 56, 56]          24,576
     BatchNorm2d-602          [-1, 512, 56, 56]           1,024
            ReLU-603          [-1, 512, 56, 56]               0
          Conv2d-604          [-1, 128, 56, 56]         589,824
          Conv2d-605          [-1, 128, 56, 56]           6,144
   DualPathBlock-606          [-1, 256, 56, 56]               0
     BatchNorm2d-607          [-1, 256, 56, 56]             512
            ReLU-608          [-1, 256, 56, 56]               0
          Conv2d-609          [-1, 512, 56, 56]         131,072
     BatchNorm2d-610          [-1, 512, 56, 56]           1,024
            ReLU-611          [-1, 512, 56, 56]               0
          Conv2d-612          [-1, 128, 56, 56]         589,824
     BatchNorm2d-613          [-1, 256, 56, 56]             512
            ReLU-614          [-1, 256, 56, 56]               0
          Conv2d-615          [-1, 512, 56, 56]         131,072
     BatchNorm2d-616          [-1, 512, 56, 56]           1,024
            ReLU-617          [-1, 512, 56, 56]               0
          Conv2d-618          [-1, 128, 56, 56]         589,824
          Conv2d-619          [-1, 128, 56, 56]          32,768
   DualPathBlock-620          [-1, 256, 56, 56]               0
     BatchNorm2d-621          [-1, 256, 56, 56]             512
            ReLU-622          [-1, 256, 56, 56]               0
          Conv2d-623          [-1, 512, 56, 56]         131,072
     BatchNorm2d-624          [-1, 512, 56, 56]           1,024
            ReLU-625          [-1, 512, 56, 56]               0
          Conv2d-626          [-1, 128, 56, 56]         589,824
     BatchNorm2d-627          [-1, 256, 56, 56]             512
            ReLU-628          [-1, 256, 56, 56]               0
          Conv2d-629          [-1, 512, 56, 56]         131,072
     BatchNorm2d-630          [-1, 512, 56, 56]           1,024
            ReLU-631          [-1, 512, 56, 56]               0
          Conv2d-632          [-1, 128, 56, 56]         589,824
          Conv2d-633          [-1, 128, 56, 56]          32,768
   DualPathBlock-634          [-1, 256, 56, 56]               0
     BatchNorm2d-635          [-1, 256, 56, 56]             512
            ReLU-636          [-1, 256, 56, 56]               0
          Conv2d-637          [-1, 512, 56, 56]         131,072
     BatchNorm2d-638          [-1, 512, 56, 56]           1,024
            ReLU-639          [-1, 512, 56, 56]               0
          Conv2d-640          [-1, 128, 56, 56]         589,824
     BatchNorm2d-641          [-1, 256, 56, 56]             512
            ReLU-642          [-1, 256, 56, 56]               0
          Conv2d-643          [-1, 512, 56, 56]         131,072
     BatchNorm2d-644          [-1, 512, 56, 56]           1,024
            ReLU-645          [-1, 512, 56, 56]               0
          Conv2d-646          [-1, 128, 56, 56]         589,824
          Conv2d-647          [-1, 128, 56, 56]          32,768
   DualPathBlock-648          [-1, 256, 56, 56]               0
     BatchNorm2d-649          [-1, 256, 56, 56]             512
            ReLU-650          [-1, 256, 56, 56]               0
          Conv2d-651          [-1, 512, 56, 56]         131,072
     BatchNorm2d-652          [-1, 512, 56, 56]           1,024
            ReLU-653          [-1, 512, 56, 56]               0
          Conv2d-654          [-1, 128, 56, 56]         589,824
     BatchNorm2d-655          [-1, 256, 56, 56]             512
            ReLU-656          [-1, 256, 56, 56]               0
          Conv2d-657          [-1, 512, 56, 56]         131,072
     BatchNorm2d-658          [-1, 512, 56, 56]           1,024
            ReLU-659          [-1, 512, 56, 56]               0
          Conv2d-660          [-1, 128, 56, 56]         589,824
          Conv2d-661          [-1, 128, 56, 56]          32,768
   DualPathBlock-662          [-1, 256, 56, 56]               0
     BatchNorm2d-663          [-1, 256, 56, 56]             512
            ReLU-664          [-1, 256, 56, 56]               0
          Conv2d-665          [-1, 512, 56, 56]         131,072
     BatchNorm2d-666          [-1, 512, 56, 56]           1,024
            ReLU-667          [-1, 512, 56, 56]               0
          Conv2d-668          [-1, 128, 56, 56]         589,824
     BatchNorm2d-669          [-1, 256, 56, 56]             512
            ReLU-670          [-1, 256, 56, 56]               0
          Conv2d-671          [-1, 512, 56, 56]         131,072
     BatchNorm2d-672          [-1, 512, 56, 56]           1,024
            ReLU-673          [-1, 512, 56, 56]               0
          Conv2d-674          [-1, 128, 56, 56]         589,824
          Conv2d-675          [-1, 128, 56, 56]          32,768
   DualPathBlock-676          [-1, 256, 56, 56]               0
     BatchNorm2d-677          [-1, 256, 56, 56]             512
            ReLU-678          [-1, 256, 56, 56]               0
          Conv2d-679          [-1, 512, 56, 56]         131,072
     BatchNorm2d-680          [-1, 512, 56, 56]           1,024
            ReLU-681          [-1, 512, 56, 56]               0
          Conv2d-682          [-1, 128, 56, 56]         589,824
     BatchNorm2d-683          [-1, 256, 56, 56]             512
            ReLU-684          [-1, 256, 56, 56]               0
          Conv2d-685          [-1, 512, 56, 56]         131,072
     BatchNorm2d-686          [-1, 512, 56, 56]           1,024
            ReLU-687          [-1, 512, 56, 56]               0
          Conv2d-688          [-1, 128, 56, 56]         589,824
          Conv2d-689          [-1, 128, 56, 56]          32,768
   DualPathBlock-690          [-1, 256, 56, 56]               0
     BatchNorm2d-691          [-1, 256, 56, 56]             512
            ReLU-692          [-1, 256, 56, 56]               0
          Conv2d-693          [-1, 512, 56, 56]         131,072
     BatchNorm2d-694          [-1, 512, 56, 56]           1,024
            ReLU-695          [-1, 512, 56, 56]               0
          Conv2d-696          [-1, 128, 56, 56]         589,824
     BatchNorm2d-697          [-1, 256, 56, 56]             512
            ReLU-698          [-1, 256, 56, 56]               0
          Conv2d-699          [-1, 512, 56, 56]         131,072
     BatchNorm2d-700          [-1, 512, 56, 56]           1,024
            ReLU-701          [-1, 512, 56, 56]               0
          Conv2d-702          [-1, 128, 56, 56]         589,824
          Conv2d-703          [-1, 128, 56, 56]          32,768
   DualPathBlock-704          [-1, 256, 56, 56]               0
     BatchNorm2d-705          [-1, 256, 56, 56]             512
            ReLU-706          [-1, 256, 56, 56]               0
          Conv2d-707          [-1, 512, 56, 56]         131,072
     BatchNorm2d-708          [-1, 512, 56, 56]           1,024
            ReLU-709          [-1, 512, 56, 56]               0
          Conv2d-710          [-1, 128, 56, 56]         589,824
     BatchNorm2d-711          [-1, 256, 56, 56]             512
            ReLU-712          [-1, 256, 56, 56]               0
          Conv2d-713          [-1, 512, 56, 56]         131,072
     BatchNorm2d-714          [-1, 512, 56, 56]           1,024
            ReLU-715          [-1, 512, 56, 56]               0
          Conv2d-716          [-1, 128, 56, 56]         589,824
          Conv2d-717          [-1, 128, 56, 56]          32,768
   DualPathBlock-718          [-1, 256, 56, 56]               0
     BatchNorm2d-719          [-1, 256, 56, 56]             512
            ReLU-720          [-1, 256, 56, 56]               0
          Conv2d-721          [-1, 512, 56, 56]         131,072
     BatchNorm2d-722          [-1, 512, 56, 56]           1,024
            ReLU-723          [-1, 512, 56, 56]               0
          Conv2d-724          [-1, 128, 56, 56]         589,824
     BatchNorm2d-725          [-1, 256, 56, 56]             512
            ReLU-726          [-1, 256, 56, 56]               0
          Conv2d-727          [-1, 512, 56, 56]         131,072
     BatchNorm2d-728          [-1, 512, 56, 56]           1,024
            ReLU-729          [-1, 512, 56, 56]               0
          Conv2d-730          [-1, 128, 56, 56]         589,824
          Conv2d-731          [-1, 128, 56, 56]          32,768
   DualPathBlock-732          [-1, 256, 56, 56]               0
     BatchNorm2d-733          [-1, 256, 56, 56]             512
            ReLU-734          [-1, 256, 56, 56]               0
          Conv2d-735          [-1, 512, 56, 56]         131,072
     BatchNorm2d-736          [-1, 512, 56, 56]           1,024
            ReLU-737          [-1, 512, 56, 56]               0
          Conv2d-738          [-1, 128, 56, 56]         589,824
     BatchNorm2d-739          [-1, 256, 56, 56]             512
            ReLU-740          [-1, 256, 56, 56]               0
          Conv2d-741          [-1, 512, 56, 56]         131,072
     BatchNorm2d-742          [-1, 512, 56, 56]           1,024
            ReLU-743          [-1, 512, 56, 56]               0
          Conv2d-744          [-1, 128, 56, 56]         589,824
          Conv2d-745          [-1, 128, 56, 56]          32,768
   DualPathBlock-746          [-1, 256, 56, 56]               0
     BatchNorm2d-747          [-1, 256, 56, 56]             512
            ReLU-748          [-1, 256, 56, 56]               0
          Conv2d-749          [-1, 512, 56, 56]         131,072
     BatchNorm2d-750          [-1, 512, 56, 56]           1,024
            ReLU-751          [-1, 512, 56, 56]               0
          Conv2d-752          [-1, 128, 56, 56]         589,824
     BatchNorm2d-753          [-1, 256, 56, 56]             512
            ReLU-754          [-1, 256, 56, 56]               0
          Conv2d-755          [-1, 512, 56, 56]         131,072
     BatchNorm2d-756          [-1, 512, 56, 56]           1,024
            ReLU-757          [-1, 512, 56, 56]               0
          Conv2d-758          [-1, 128, 56, 56]         589,824
          Conv2d-759          [-1, 128, 56, 56]          32,768
   DualPathBlock-760          [-1, 256, 56, 56]               0
     BatchNorm2d-761          [-1, 256, 56, 56]             512
            ReLU-762          [-1, 256, 56, 56]               0
          Conv2d-763          [-1, 512, 56, 56]         131,072
     BatchNorm2d-764          [-1, 512, 56, 56]           1,024
            ReLU-765          [-1, 512, 56, 56]               0
          Conv2d-766          [-1, 128, 56, 56]         589,824
     BatchNorm2d-767          [-1, 256, 56, 56]             512
            ReLU-768          [-1, 256, 56, 56]               0
          Conv2d-769          [-1, 512, 56, 56]         131,072
     BatchNorm2d-770          [-1, 512, 56, 56]           1,024
            ReLU-771          [-1, 512, 56, 56]               0
          Conv2d-772          [-1, 128, 56, 56]         589,824
          Conv2d-773          [-1, 128, 56, 56]          32,768
   DualPathBlock-774          [-1, 256, 56, 56]               0
     BatchNorm2d-775          [-1, 256, 56, 56]             512
            ReLU-776          [-1, 256, 56, 56]               0
          Conv2d-777          [-1, 512, 56, 56]         131,072
     BatchNorm2d-778          [-1, 512, 56, 56]           1,024
            ReLU-779          [-1, 512, 56, 56]               0
          Conv2d-780          [-1, 128, 56, 56]         589,824
     BatchNorm2d-781          [-1, 256, 56, 56]             512
            ReLU-782          [-1, 256, 56, 56]               0
          Conv2d-783          [-1, 512, 56, 56]         131,072
     BatchNorm2d-784          [-1, 512, 56, 56]           1,024
            ReLU-785          [-1, 512, 56, 56]               0
          Conv2d-786          [-1, 128, 56, 56]         589,824
          Conv2d-787          [-1, 128, 56, 56]          32,768
   DualPathBlock-788          [-1, 256, 56, 56]               0
     BatchNorm2d-789          [-1, 256, 56, 56]             512
            ReLU-790          [-1, 256, 56, 56]               0
          Conv2d-791          [-1, 512, 56, 56]         131,072
     BatchNorm2d-792          [-1, 512, 56, 56]           1,024
            ReLU-793          [-1, 512, 56, 56]               0
          Conv2d-794          [-1, 128, 56, 56]         589,824
     BatchNorm2d-795          [-1, 256, 56, 56]             512
            ReLU-796          [-1, 256, 56, 56]               0
          Conv2d-797          [-1, 512, 56, 56]         131,072
     BatchNorm2d-798          [-1, 512, 56, 56]           1,024
            ReLU-799          [-1, 512, 56, 56]               0
          Conv2d-800          [-1, 128, 56, 56]         589,824
          Conv2d-801          [-1, 128, 56, 56]          32,768
   DualPathBlock-802          [-1, 256, 56, 56]               0
     BatchNorm2d-803          [-1, 256, 56, 56]             512
            ReLU-804          [-1, 256, 56, 56]               0
          Conv2d-805          [-1, 512, 56, 56]         131,072
     BatchNorm2d-806          [-1, 512, 56, 56]           1,024
            ReLU-807          [-1, 512, 56, 56]               0
          Conv2d-808          [-1, 128, 56, 56]         589,824
     BatchNorm2d-809          [-1, 256, 56, 56]             512
            ReLU-810          [-1, 256, 56, 56]               0
          Conv2d-811          [-1, 512, 56, 56]         131,072
     BatchNorm2d-812          [-1, 512, 56, 56]           1,024
            ReLU-813          [-1, 512, 56, 56]               0
          Conv2d-814          [-1, 128, 56, 56]         589,824
          Conv2d-815          [-1, 128, 56, 56]          32,768
   DualPathBlock-816          [-1, 256, 56, 56]               0
AdaptiveAvgPool2d-817            [-1, 256, 1, 1]               0
          Linear-818                    [-1, 4]           1,028
================================================================
Total params: 25,923,780
Trainable params: 25,923,780
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.57
Forward/backward pass size (MB): 2749.36
Params size (MB): 98.89
Estimated Total Size (MB): 2848.83
----------------------------------------------------------------

5. 设置超参数：定义损失函数，学习率，以及根据学习率定义优化器等

# loss_fn = nn.CrossEntropyLoss() # 创建损失函数

# learn_rate = 1e-3 # 初始学习率
# def adjust_learning_rate(optimizer,epoch,start_lr):
#     # 每两个epoch 衰减到原来的0.98
#     lr = start_lr * (0.92 ** (epoch//2))
#     for param_group in optimizer.param_groups:
#         param_group['lr'] = lr
        
# optimizer = torch.optim.Adam(model.parameters(),lr=learn_rate)

# 调用官方接口示例
loss_fn = nn.CrossEntropyLoss()

learn_rate = 1e-4
lambda1 = lambda epoch:(0.92**(epoch//2))

optimizer = torch.optim.Adam(model.parameters(),lr = learn_rate)
scheduler = torch.optim.lr_scheduler.LambdaLR(optimizer,lr_lambda=lambda1) # 选定调整方法

6. 训练函数

# 训练函数
def train(dataloader,model,loss_fn,optimizer):
    size = len(dataloader.dataset) # 训练集大小
    num_batches = len(dataloader) # 批次数目
    
    train_loss,train_acc = 0,0
    
    for X,y in dataloader:
        X,y = X.to(device),y.to(device)
        
        # 计算预测误差
        pred = model(X)
        loss = loss_fn(pred,y)
        
        # 反向传播
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        
        # 记录acc与loss
        train_acc += (pred.argmax(1)==y).type(torch.float).sum().item()
        train_loss += loss.item()
        
    train_acc /= size
    train_loss /= num_batches
    
    return train_acc,train_loss

7. 测试函数

# 测试函数
def test(dataloader,model,loss_fn):
    size = len(dataloader.dataset)
    num_batches = len(dataloader)
    
    test_acc,test_loss = 0,0
    
    with torch.no_grad():
        for X,y in dataloader:
            X,y = X.to(device),y.to(device)
            
            # 计算loss
            pred = model(X)
            loss = loss_fn(pred,y)
            
            test_acc += (pred.argmax(1)==y).type(torch.float).sum().item()
            test_loss += loss.item()
            
    test_acc /= size
    test_loss /= num_batches
    
    return test_acc,test_loss

8. 正式训练

import copy

epochs = 40

train_acc = []
train_loss = []
test_acc = []
test_loss = []

best_acc = 0.0

# 训练前清除缓存
torch.cuda.empty_cache()


for epoch in range(epochs):
    
    # 更新学习率——使用自定义学习率时使用
    # adjust_learning_rate(optimizer,epoch,learn_rate)
    
    model.train()
    epoch_train_acc,epoch_train_loss = train(train_dl,model,loss_fn,optimizer)
    scheduler.step() # 更新学习率——调用官方动态学习率时使用
    
    model.eval()
    epoch_test_acc,epoch_test_loss = test(test_dl,model,loss_fn)
    
    # 保存最佳模型到 best_model
    if epoch_test_acc > best_acc:
        best_acc = epoch_test_acc
        best_model = copy.deepcopy(model)
    
    train_acc.append(epoch_train_acc)
    train_loss.append(epoch_train_loss)
    test_acc.append(epoch_test_acc)
    test_loss.append(epoch_test_loss)
    
    # 获取当前学习率
    lr = optimizer.state_dict()['param_groups'][0]['lr']
    
    template = ('Epoch:{:2d},Train_acc:{:.1f}%,Train_loss:{:.3f},Test_acc:{:.1f}%,Test_loss:{:.3f},Lr:{:.2E}')
    print(template.format(epoch+1,epoch_train_acc*100,epoch_train_loss,epoch_test_acc*100,epoch_test_loss,lr))

print('Done')

Epoch: 1,Train_acc:29.2%,Train_loss:1.440,Test_acc:30.1%,Test_loss:1.402,Lr:1.00E-04
Epoch: 2,Train_acc:32.5%,Train_loss:1.387,Test_acc:23.0%,Test_loss:1.387,Lr:9.20E-05
Epoch: 3,Train_acc:32.7%,Train_loss:1.377,Test_acc:33.6%,Test_loss:1.458,Lr:9.20E-05
Epoch: 4,Train_acc:35.0%,Train_loss:1.357,Test_acc:35.4%,Test_loss:1.473,Lr:8.46E-05
Epoch: 5,Train_acc:33.4%,Train_loss:1.357,Test_acc:40.7%,Test_loss:1.333,Lr:8.46E-05
Epoch: 6,Train_acc:36.1%,Train_loss:1.369,Test_acc:37.2%,Test_loss:1.350,Lr:7.79E-05
Epoch: 7,Train_acc:33.4%,Train_loss:1.368,Test_acc:32.7%,Test_loss:1.566,Lr:7.79E-05
Epoch: 8,Train_acc:33.4%,Train_loss:1.358,Test_acc:36.3%,Test_loss:1.423,Lr:7.16E-05
Epoch: 9,Train_acc:35.8%,Train_loss:1.350,Test_acc:36.3%,Test_loss:1.302,Lr:7.16E-05
Epoch:10,Train_acc:36.5%,Train_loss:1.341,Test_acc:35.4%,Test_loss:1.643,Lr:6.59E-05
Epoch:11,Train_acc:36.5%,Train_loss:1.348,Test_acc:35.4%,Test_loss:1.342,Lr:6.59E-05
Epoch:12,Train_acc:33.4%,Train_loss:1.337,Test_acc:34.5%,Test_loss:1.334,Lr:6.06E-05
Epoch:13,Train_acc:37.6%,Train_loss:1.338,Test_acc:36.3%,Test_loss:1.954,Lr:6.06E-05
Epoch:14,Train_acc:35.0%,Train_loss:1.353,Test_acc:38.1%,Test_loss:1.400,Lr:5.58E-05
Epoch:15,Train_acc:32.3%,Train_loss:1.340,Test_acc:36.3%,Test_loss:1.380,Lr:5.58E-05
Epoch:16,Train_acc:35.4%,Train_loss:1.350,Test_acc:44.2%,Test_loss:1.350,Lr:5.13E-05
Epoch:17,Train_acc:36.7%,Train_loss:1.339,Test_acc:36.3%,Test_loss:1.430,Lr:5.13E-05
Epoch:18,Train_acc:36.1%,Train_loss:1.337,Test_acc:37.2%,Test_loss:1.350,Lr:4.72E-05
Epoch:19,Train_acc:39.2%,Train_loss:1.334,Test_acc:33.6%,Test_loss:1.323,Lr:4.72E-05
Epoch:20,Train_acc:36.9%,Train_loss:1.316,Test_acc:32.7%,Test_loss:1.491,Lr:4.34E-05
Epoch:21,Train_acc:36.7%,Train_loss:1.337,Test_acc:32.7%,Test_loss:1.323,Lr:4.34E-05
Epoch:22,Train_acc:33.8%,Train_loss:1.353,Test_acc:37.2%,Test_loss:1.307,Lr:4.00E-05
Epoch:23,Train_acc:36.3%,Train_loss:1.335,Test_acc:40.7%,Test_loss:1.276,Lr:4.00E-05
Epoch:24,Train_acc:36.9%,Train_loss:1.324,Test_acc:41.6%,Test_loss:1.294,Lr:3.68E-05
Epoch:25,Train_acc:36.9%,Train_loss:1.314,Test_acc:38.9%,Test_loss:1.245,Lr:3.68E-05
Epoch:26,Train_acc:37.6%,Train_loss:1.329,Test_acc:41.6%,Test_loss:1.273,Lr:3.38E-05
Epoch:27,Train_acc:34.7%,Train_loss:1.334,Test_acc:40.7%,Test_loss:1.278,Lr:3.38E-05
Epoch:28,Train_acc:38.9%,Train_loss:1.326,Test_acc:39.8%,Test_loss:1.278,Lr:3.11E-05
Epoch:29,Train_acc:36.9%,Train_loss:1.313,Test_acc:46.0%,Test_loss:1.254,Lr:3.11E-05
Epoch:30,Train_acc:36.5%,Train_loss:1.298,Test_acc:38.9%,Test_loss:1.228,Lr:2.86E-05
Epoch:31,Train_acc:37.8%,Train_loss:1.301,Test_acc:45.1%,Test_loss:1.223,Lr:2.86E-05
Epoch:32,Train_acc:36.7%,Train_loss:1.328,Test_acc:44.2%,Test_loss:1.275,Lr:2.63E-05
Epoch:33,Train_acc:36.1%,Train_loss:1.321,Test_acc:42.5%,Test_loss:1.260,Lr:2.63E-05
Epoch:34,Train_acc:36.3%,Train_loss:1.339,Test_acc:36.3%,Test_loss:1.264,Lr:2.42E-05
Epoch:35,Train_acc:37.2%,Train_loss:1.323,Test_acc:37.2%,Test_loss:1.245,Lr:2.42E-05
Epoch:36,Train_acc:33.8%,Train_loss:1.348,Test_acc:42.5%,Test_loss:1.256,Lr:2.23E-05
Epoch:37,Train_acc:37.4%,Train_loss:1.322,Test_acc:38.9%,Test_loss:1.230,Lr:2.23E-05
Epoch:38,Train_acc:38.7%,Train_loss:1.315,Test_acc:46.9%,Test_loss:1.261,Lr:2.05E-05
Epoch:39,Train_acc:38.7%,Train_loss:1.307,Test_acc:37.2%,Test_loss:1.243,Lr:2.05E-05
Epoch:40,Train_acc:38.1%,Train_loss:1.302,Test_acc:37.2%,Test_loss:1.230,Lr:1.89E-05
Done

9. 结果可视化

epochs_range = range(epochs)

plt.figure(figsize = (12,3))

plt.subplot(1,2,1)
plt.plot(epochs_range,train_acc,label = 'Training Accuracy')
plt.plot(epochs_range,test_acc,label = 'Test Accuracy')
plt.legend(loc = 'lower right')
plt.title('Training and Validation Accuracy')

plt.subplot(1,2,2)
plt.plot(epochs_range,train_loss,label = 'Test Accuracy')
plt.plot(epochs_range,test_loss,label = 'Test Loss')
plt.legend(loc = 'lower right')
plt.title('Training and validation Loss')
plt.show()

findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei

​

10. 模型的保存

# 自定义模型保存
# 状态字典保存
torch.save(model.state_dict(),'./模型参数/J4_DPN_model_state_dict.pth') # 仅保存状态字典

# 定义模型用来加载参数
best_model = DPN(len(classNames)).to(device)

# best_model = DPN(num_classes=len(classNames), growth_rate=16, block_config=(3, 6, 12, 8),
#             C_res=(16, 32, 24, 64), C_dense=(16, 32, 24, 64), bn_size=4, drop_rate=0).to(device)

best_model.load_state_dict(torch.load('./模型参数/J4_DPN_model_state_dict.pth')) # 加载状态字典到模型

<All keys matched successfully>

11. 使用训练好的模型进行预测

# 指定路径图片预测
from PIL import Image
import torchvision.transforms as transforms

classes = list(total_data.class_to_idx) # classes = list(total_data.class_to_idx)

def predict_one_image(image_path,model,transform,classes):
    
    test_img = Image.open(image_path).convert('RGB')
    # plt.imshow(test_img) # 展示待预测的图片
    
    test_img = transform(test_img)
    img = test_img.to(device).unsqueeze(0)
    
    model.eval()
    output = model(img)
    print(output) # 观察模型预测结果的输出数据
    
    _,pred = torch.max(output,1)
    pred_class = classes[pred]
    print(f'预测结果是:{pred_class}')

# 预测训练集中的某张照片
predict_one_image(image_path='./data/bird_photos/Bananaquit/007.jpg',
                 model = model,
                 transform = test_transforms,
                 classes = classes
                 )

tensor([[ 0.9736, -0.8309,  0.9055, -0.5009]], device='cuda:0',
       grad_fn=<AddmmBackward0>)
预测结果是:Bananaquit

classes

['Bananaquit', 'Black Skimmer', 'Black Throated Bushtiti', 'Cockatoo']