🤵 Author ：Horizon Max
✨ 编程技巧篇：各种操作小结
🎇 机器视觉篇：会变魔术 OpenCV
💥 深度学习篇：简单入门 PyTorch
🏆 神经网络篇：经典网络模型
💻 算法篇：再忙也别忘了 LeetCode
[ 图像分类 ] 经典网络模型5——DenseNet 详解与复现
🚀 Dense Convolutional Network
Dense Convolutional Network 就是熟知的 DenseNet（密集卷积网络），获得了 CVPR 2017年 的 Oral ；
对于每一层，前面所有层的特征映射作为输入，并将特征映射作为后续所有层的输入 ；
它缓解了 消失梯度 问题，加强了 特征传播 ，鼓励了 特征重用 ，并大大减少了 参数量 ；
🔗 论文地址：Densely Connected Convolutional Networks
🚀 DenseNet 详解
🎨 DenseNet 网络结构
卷积神经网络(CNN) 已经成为视觉的主要机器学习方法 ；
随着CNN 的不断发展，网络模型越来越深入，一个新的研究问题出现了：
是关于 输入或梯度的信息 通过许多层后，当它到达网络的末端（或开始）时，它可能会 消失 和 "冲刷" ；
🚩 核心思想
基于此，作者提炼了一个简单的 连接模式 ：
为确保网络中各层之间的信息流动最大化，将所有层(具有匹配的特征图大小)直接连接起来 ；
每一层从所有前面的层获得额外的输入，并将自己的特征映射传递给所有后面的层 ；
在传统的 L层网络 中引入了 L 连接，而在 DenseNet 中引入了 L(L+1) / 2 连接 ！
以下是一个生长速率 k = 4 的 5层 dense block ，每一层都以前面所有的特征图作为输入：
🎨 DenseNet 网络特点
这种连接模式与直觉相反的效果是：它需要的 参数量 比传统卷积网络 更少 ！
除了更好的 参数效率 外，DenseNet的一大优点是改进了整个 网络的信息流和梯度，这使得它们 易于训练 ！
密集连接 具有 正则化效应 ，它减少了训练集规模较小的任务的 过拟合 现象 ；
DenseNet并没有从极 深（如 ResNet）或极 宽（如 GoogLeNet）的架构中获取表征力，而是通过 特征重用 来挖掘网络的潜力，产生 易于训练和高参数效率 的压缩模型 ；
卷积网络传递每一层实现的非线性变换一般都定义一个 批量归一化(BN)、修正线性单元(ReLU)、池化(Pool) 或 卷积(Conv) 等操作的复合函数 ；
在 DenseNet 中为三个连续操作的复合函数：批归一化(BN)，随后是 修正线性单元(ReLU) 和 3 × 3卷积(Conv) ；
🚀 DenseNet 复现
# Here is the code ：
import torch
import torch.nn as nn
import torch.nn.functional as F
from collections import OrderedDict
from torchinfo import summary
class Bottleneck(nn.Module):
def __init__(self, in_features, growth_rate, bn_size, drop_rate):
super(Bottleneck, self).__init__()
out_features = bn_size * growth_rate
self.norm1: nn.BatchNorm2d
self.add_module('norm1', nn.BatchNorm2d(in_features))
self.relu1: nn.ReLU
self.add_module('relu1', nn.ReLU(inplace=True))
self.conv1: nn.Conv2d
self.add_module('conv1', nn.Conv2d(in_features, out_features,
kernel_size=1, stride=1, bias=False))
self.norm2: nn.BatchNorm2d
self.add_module('norm2', nn.BatchNorm2d(bn_size * growth_rate))
self.relu2: nn.ReLU
self.add_module('relu2', nn.ReLU(inplace=True))
self.conv2: nn.Conv2d
self.add_module('conv2', nn.Conv2d(out_features, growth_rate,
kernel_size=3, stride=1, padding=1,
bias=False))
self.drop_rate = float(drop_rate)
def forward(self, x):
new_features = self.conv1(self.relu1(self.norm1(x)))
new_features = self.conv2(self.relu2(self.norm2(new_features)))
if self.drop_rate > 0:
new_features = F.dropout(new_features, p=self.drop_rate, training=self.training)
new_features = torch.cat([x, new_features], 1)
return new_features
class Dense_Block(nn.Sequential): # Dense_Block
def __init__(self, in_features, growth_rate, num_block, bn_size, drop_rate):
super(Dense_Block, self).__init__()
for i in range(num_block):
layer = Bottleneck(in_features + i * growth_rate,
growth_rate, bn_size, drop_rate)
self.add_module('dense_layer%d' % (i + 1), layer)
class Transition_Layer(nn.Sequential): # Transition_Layer
def __init__(self, in_features, out_features):
super(Transition_Layer, self).__init__()
self.add_module('norm', nn.BatchNorm2d(in_features))
self.add_module('relu', nn.ReLU(inplace=True))
self.add_module('conv', nn.Conv2d(in_features, out_features,
kernel_size=1, stride=1, bias=False))
self.add_module('pool', nn.AvgPool2d(kernel_size=2, stride=2))
class DenseNet(nn.Module):
def __init__(self, block_config, growth_rate, num_init_features, bn_size=4, drop_rate=0, num_classes=1000):
super(DenseNet, self).__init__()
# First convolution
self.features = nn.Sequential(OrderedDict([
('conv0', nn.Conv2d(3, num_init_features, kernel_size=7, stride=2, padding=3, bias=False)),
('norm0', nn.BatchNorm2d(num_init_features)),
('relu0', nn.ReLU(inplace=True)),
('pool0', nn.MaxPool2d(kernel_size=3, stride=2, padding=1)),
]))
# Each denseblock
num_features = num_init_features
for i, num_layers in enumerate(block_config):
block = Dense_Block(in_features=num_features, growth_rate=growth_rate, num_block=num_layers,
bn_size=bn_size, drop_rate=drop_rate)
self.features.add_module('dense_block%d' % (i + 1), block)
num_features = num_features + num_layers * growth_rate
if i != len(block_config) - 1:
trans = Transition_Layer(in_features=num_features,
out_features=num_features // 2)
self.features.add_module('transition%d' % (i + 1), trans)
num_features = num_features // 2
# Final batch norm
self.features.add_module('norm5', nn.BatchNorm2d(num_features))
# Classification layer
self.avg = nn.AvgPool2d(kernel_size=7, stride=1)
self.classifier = nn.Linear(num_features, num_classes)
def forward(self, x):
x = self.features(x)
x = F.relu(x, inplace=True)
x = self.avg(x)
x = torch.flatten(x, 1)
out = self.classifier(x)
return out
def DenseNet121(**kwargs):
return DenseNet(block_config=[6, 12, 24, 16], growth_rate=32, num_init_features=64, **kwargs)
def DenseNet161(**kwargs):
return DenseNet(block_config=[6, 12, 36, 24], growth_rate=48, num_init_features=96, **kwargs)
def DenseNet169(**kwargs):
return DenseNet(block_config=[6, 12, 32, 32], growth_rate=32, num_init_features=64, **kwargs)
def DenseNet201(**kwargs):
return DenseNet(block_config=[6, 12, 48, 32], growth_rate=32, num_init_features=64, **kwargs)
def DenseNet264(**kwargs):
return DenseNet(block_config=[6, 12, 64, 48], growth_rate=32, num_init_features=64, **kwargs)
def test():
net = DenseNet121()
y = net(torch.randn(1, 3, 224, 224))
print(y.size())
summary(net, (1, 3, 224, 224), depth=4)
if __name__ == '__main__':
test()
输出结果：
torch.Size([1, 1000])
==========================================================================================
Layer (type:depth-idx) Output Shape Param #
==========================================================================================
DenseNet -- --
├─Sequential: 1-1 [1, 1024, 7, 7] --
│ └─Conv2d: 2-1 [1, 64, 112, 112] 9,408
│ └─BatchNorm2d: 2-2 [1, 64, 112, 112] 128
│ └─ReLU: 2-3 [1, 64, 112, 112] --
│ └─MaxPool2d: 2-4 [1, 64, 56, 56] --
│ └─Dense_Block: 2-5 [1, 256, 56, 56] --
│ │ └─Bottleneck: 3-1 [1, 96, 56, 56] --
│ │ │ └─BatchNorm2d: 4-1 [1, 64, 56, 56] 128
│ │ │ └─ReLU: 4-2 [1, 64, 56, 56] --
│ │ │ └─Conv2d: 4-3 [1, 128, 56, 56] 8,192
│ │ │ └─BatchNorm2d: 4-4 [1, 128, 56, 56] 256
│ │ │ └─ReLU: 4-5 [1, 128, 56, 56] --
│ │ │ └─Conv2d: 4-6 [1, 32, 56, 56] 36,864
│ │ └─Bottleneck: 3-2 [1, 128, 56, 56] --
│ │ │ └─BatchNorm2d: 4-7 [1, 96, 56, 56] 192
│ │ │ └─ReLU: 4-8 [1, 96, 56, 56] --
│ │ │ └─Conv2d: 4-9 [1, 128, 56, 56] 12,288
│ │ │ └─BatchNorm2d: 4-10 [1, 128, 56, 56] 256
│ │ │ └─ReLU: 4-11 [1, 128, 56, 56] --
│ │ │ └─Conv2d: 4-12 [1, 32, 56, 56] 36,864
│ │ └─Bottleneck: 3-3 [1, 160, 56, 56] --
│ │ │ └─BatchNorm2d: 4-13 [1, 128, 56, 56] 256
│ │ │ └─ReLU: 4-14 [1, 128, 56, 56] --
│ │ │ └─Conv2d: 4-15 [1, 128, 56, 56] 16,384
│ │ │ └─BatchNorm2d: 4-16 [1, 128, 56, 56] 256
│ │ │ └─ReLU: 4-17 [1, 128, 56, 56] --
│ │ │ └─Conv2d: 4-18 [1, 32, 56, 56] 36,864
│ │ └─Bottleneck: 3-4 [1, 192, 56, 56] --
│ │ │ └─BatchNorm2d: 4-19 [1, 160, 56, 56] 320
│ │ │ └─ReLU: 4-20 [1, 160, 56, 56] --
│ │ │ └─Conv2d: 4-21 [1, 128, 56, 56] 20,480
│ │ │ └─BatchNorm2d: 4-22 [1, 128, 56, 56] 256
│ │ │ └─ReLU: 4-23 [1, 128, 56, 56] --
│ │ │ └─Conv2d: 4-24 [1, 32, 56, 56] 36,864
│ │ └─Bottleneck: 3-5 [1, 224, 56, 56] --
│ │ │ └─BatchNorm2d: 4-25 [1, 192, 56, 56] 384
│ │ │ └─ReLU: 4-26 [1, 192, 56, 56] --
│ │ │ └─Conv2d: 4-27 [1, 128, 56, 56] 24,576
│ │ │ └─BatchNorm2d: 4-28 [1, 128, 56, 56] 256
│ │ │ └─ReLU: 4-29 [1, 128, 56, 56] --
│ │ │ └─Conv2d: 4-30 [1, 32, 56, 56] 36,864
│ │ └─Bottleneck: 3-6 [1, 256, 56, 56] --
│ │ │ └─BatchNorm2d: 4-31 [1, 224, 56, 56] 448
│ │ │ └─ReLU: 4-32 [1, 224, 56, 56] --
│ │ │ └─Conv2d: 4-33 [1, 128, 56, 56] 28,672
│ │ │ └─BatchNorm2d: 4-34 [1, 128, 56, 56] 256
│ │ │ └─ReLU: 4-35 [1, 128, 56, 56] --
│ │ │ └─Conv2d: 4-36 [1, 32, 56, 56] 36,864
│ └─Transition_Layer: 2-6 [1, 128, 28, 28] --
│ │ └─BatchNorm2d: 3-7 [1, 256, 56, 56] 512
│ │ └─ReLU: 3-8 [1, 256, 56, 56] --
│ │ └─Conv2d: 3-9 [1, 128, 56, 56] 32,768
│ │ └─AvgPool2d: 3-10 [1, 128, 28, 28] --
│ └─Dense_Block: 2-7 [1, 512, 28, 28] --
│ │ └─Bottleneck: 3-11 [1, 160, 28, 28] --
│ │ │ └─BatchNorm2d: 4-37 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-38 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-39 [1, 128, 28, 28] 16,384
│ │ │ └─BatchNorm2d: 4-40 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-41 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-42 [1, 32, 28, 28] 36,864
│ │ └─Bottleneck: 3-12 [1, 192, 28, 28] --
│ │ │ └─BatchNorm2d: 4-43 [1, 160, 28, 28] 320
│ │ │ └─ReLU: 4-44 [1, 160, 28, 28] --
│ │ │ └─Conv2d: 4-45 [1, 128, 28, 28] 20,480
│ │ │ └─BatchNorm2d: 4-46 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-47 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-48 [1, 32, 28, 28] 36,864
│ │ └─Bottleneck: 3-13 [1, 224, 28, 28] --
│ │ │ └─BatchNorm2d: 4-49 [1, 192, 28, 28] 384
│ │ │ └─ReLU: 4-50 [1, 192, 28, 28] --
│ │ │ └─Conv2d: 4-51 [1, 128, 28, 28] 24,576
│ │ │ └─BatchNorm2d: 4-52 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-53 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-54 [1, 32, 28, 28] 36,864
│ │ └─Bottleneck: 3-14 [1, 256, 28, 28] --
│ │ │ └─BatchNorm2d: 4-55 [1, 224, 28, 28] 448
│ │ │ └─ReLU: 4-56 [1, 224, 28, 28] --
│ │ │ └─Conv2d: 4-57 [1, 128, 28, 28] 28,672
│ │ │ └─BatchNorm2d: 4-58 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-59 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-60 [1, 32, 28, 28] 36,864
│ │ └─Bottleneck: 3-15 [1, 288, 28, 28] --
│ │ │ └─BatchNorm2d: 4-61 [1, 256, 28, 28] 512
│ │ │ └─ReLU: 4-62 [1, 256, 28, 28] --
│ │ │ └─Conv2d: 4-63 [1, 128, 28, 28] 32,768
│ │ │ └─BatchNorm2d: 4-64 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-65 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-66 [1, 32, 28, 28] 36,864
│ │ └─Bottleneck: 3-16 [1, 320, 28, 28] --
│ │ │ └─BatchNorm2d: 4-67 [1, 288, 28, 28] 576
│ │ │ └─ReLU: 4-68 [1, 288, 28, 28] --
│ │ │ └─Conv2d: 4-69 [1, 128, 28, 28] 36,864
│ │ │ └─BatchNorm2d: 4-70 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-71 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-72 [1, 32, 28, 28] 36,864
│ │ └─Bottleneck: 3-17 [1, 352, 28, 28] --
│ │ │ └─BatchNorm2d: 4-73 [1, 320, 28, 28] 640
│ │ │ └─ReLU: 4-74 [1, 320, 28, 28] --
│ │ │ └─Conv2d: 4-75 [1, 128, 28, 28] 40,960
│ │ │ └─BatchNorm2d: 4-76 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-77 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-78 [1, 32, 28, 28] 36,864
│ │ └─Bottleneck: 3-18 [1, 384, 28, 28] --
│ │ │ └─BatchNorm2d: 4-79 [1, 352, 28, 28] 704
│ │ │ └─ReLU: 4-80 [1, 352, 28, 28] --
│ │ │ └─Conv2d: 4-81 [1, 128, 28, 28] 45,056
│ │ │ └─BatchNorm2d: 4-82 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-83 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-84 [1, 32, 28, 28] 36,864
│ │ └─Bottleneck: 3-19 [1, 416, 28, 28] --
│ │ │ └─BatchNorm2d: 4-85 [1, 384, 28, 28] 768
│ │ │ └─ReLU: 4-86 [1, 384, 28, 28] --
│ │ │ └─Conv2d: 4-87 [1, 128, 28, 28] 49,152
│ │ │ └─BatchNorm2d: 4-88 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-89 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-90 [1, 32, 28, 28] 36,864
│ │ └─Bottleneck: 3-20 [1, 448, 28, 28] --
│ │ │ └─BatchNorm2d: 4-91 [1, 416, 28, 28] 832
│ │ │ └─ReLU: 4-92 [1, 416, 28, 28] --
│ │ │ └─Conv2d: 4-93 [1, 128, 28, 28] 53,248
│ │ │ └─BatchNorm2d: 4-94 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-95 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-96 [1, 32, 28, 28] 36,864
│ │ └─Bottleneck: 3-21 [1, 480, 28, 28] --
│ │ │ └─BatchNorm2d: 4-97 [1, 448, 28, 28] 896
│ │ │ └─ReLU: 4-98 [1, 448, 28, 28] --
│ │ │ └─Conv2d: 4-99 [1, 128, 28, 28] 57,344
│ │ │ └─BatchNorm2d: 4-100 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-101 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-102 [1, 32, 28, 28] 36,864
│ │ └─Bottleneck: 3-22 [1, 512, 28, 28] --
│ │ │ └─BatchNorm2d: 4-103 [1, 480, 28, 28] 960
│ │ │ └─ReLU: 4-104 [1, 480, 28, 28] --
│ │ │ └─Conv2d: 4-105 [1, 128, 28, 28] 61,440
│ │ │ └─BatchNorm2d: 4-106 [1, 128, 28, 28] 256
│ │ │ └─ReLU: 4-107 [1, 128, 28, 28] --
│ │ │ └─Conv2d: 4-108 [1, 32, 28, 28] 36,864
│ └─Transition_Layer: 2-8 [1, 256, 14, 14] --
│ │ └─BatchNorm2d: 3-23 [1, 512, 28, 28] 1,024
│ │ └─ReLU: 3-24 [1, 512, 28, 28] --
│ │ └─Conv2d: 3-25 [1, 256, 28, 28] 131,072
│ │ └─AvgPool2d: 3-26 [1, 256, 14, 14] --
│ └─Dense_Block: 2-9 [1, 1024, 14, 14] --
│ │ └─Bottleneck: 3-27 [1, 288, 14, 14] --
│ │ │ └─BatchNorm2d: 4-109 [1, 256, 14, 14] 512
│ │ │ └─ReLU: 4-110 [1, 256, 14, 14] --
│ │ │ └─Conv2d: 4-111 [1, 128, 14, 14] 32,768
│ │ │ └─BatchNorm2d: 4-112 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-113 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-114 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-28 [1, 320, 14, 14] --
│ │ │ └─BatchNorm2d: 4-115 [1, 288, 14, 14] 576
│ │ │ └─ReLU: 4-116 [1, 288, 14, 14] --
│ │ │ └─Conv2d: 4-117 [1, 128, 14, 14] 36,864
│ │ │ └─BatchNorm2d: 4-118 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-119 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-120 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-29 [1, 352, 14, 14] --
│ │ │ └─BatchNorm2d: 4-121 [1, 320, 14, 14] 640
│ │ │ └─ReLU: 4-122 [1, 320, 14, 14] --
│ │ │ └─Conv2d: 4-123 [1, 128, 14, 14] 40,960
│ │ │ └─BatchNorm2d: 4-124 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-125 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-126 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-30 [1, 384, 14, 14] --
│ │ │ └─BatchNorm2d: 4-127 [1, 352, 14, 14] 704
│ │ │ └─ReLU: 4-128 [1, 352, 14, 14] --
│ │ │ └─Conv2d: 4-129 [1, 128, 14, 14] 45,056
│ │ │ └─BatchNorm2d: 4-130 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-131 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-132 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-31 [1, 416, 14, 14] --
│ │ │ └─BatchNorm2d: 4-133 [1, 384, 14, 14] 768
│ │ │ └─ReLU: 4-134 [1, 384, 14, 14] --
│ │ │ └─Conv2d: 4-135 [1, 128, 14, 14] 49,152
│ │ │ └─BatchNorm2d: 4-136 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-137 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-138 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-32 [1, 448, 14, 14] --
│ │ │ └─BatchNorm2d: 4-139 [1, 416, 14, 14] 832
│ │ │ └─ReLU: 4-140 [1, 416, 14, 14] --
│ │ │ └─Conv2d: 4-141 [1, 128, 14, 14] 53,248
│ │ │ └─BatchNorm2d: 4-142 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-143 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-144 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-33 [1, 480, 14, 14] --
│ │ │ └─BatchNorm2d: 4-145 [1, 448, 14, 14] 896
│ │ │ └─ReLU: 4-146 [1, 448, 14, 14] --
│ │ │ └─Conv2d: 4-147 [1, 128, 14, 14] 57,344
│ │ │ └─BatchNorm2d: 4-148 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-149 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-150 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-34 [1, 512, 14, 14] --
│ │ │ └─BatchNorm2d: 4-151 [1, 480, 14, 14] 960
│ │ │ └─ReLU: 4-152 [1, 480, 14, 14] --
│ │ │ └─Conv2d: 4-153 [1, 128, 14, 14] 61,440
│ │ │ └─BatchNorm2d: 4-154 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-155 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-156 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-35 [1, 544, 14, 14] --
│ │ │ └─BatchNorm2d: 4-157 [1, 512, 14, 14] 1,024
│ │ │ └─ReLU: 4-158 [1, 512, 14, 14] --
│ │ │ └─Conv2d: 4-159 [1, 128, 14, 14] 65,536
│ │ │ └─BatchNorm2d: 4-160 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-161 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-162 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-36 [1, 576, 14, 14] --
│ │ │ └─BatchNorm2d: 4-163 [1, 544, 14, 14] 1,088
│ │ │ └─ReLU: 4-164 [1, 544, 14, 14] --
│ │ │ └─Conv2d: 4-165 [1, 128, 14, 14] 69,632
│ │ │ └─BatchNorm2d: 4-166 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-167 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-168 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-37 [1, 608, 14, 14] --
│ │ │ └─BatchNorm2d: 4-169 [1, 576, 14, 14] 1,152
│ │ │ └─ReLU: 4-170 [1, 576, 14, 14] --
│ │ │ └─Conv2d: 4-171 [1, 128, 14, 14] 73,728
│ │ │ └─BatchNorm2d: 4-172 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-173 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-174 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-38 [1, 640, 14, 14] --
│ │ │ └─BatchNorm2d: 4-175 [1, 608, 14, 14] 1,216
│ │ │ └─ReLU: 4-176 [1, 608, 14, 14] --
│ │ │ └─Conv2d: 4-177 [1, 128, 14, 14] 77,824
│ │ │ └─BatchNorm2d: 4-178 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-179 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-180 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-39 [1, 672, 14, 14] --
│ │ │ └─BatchNorm2d: 4-181 [1, 640, 14, 14] 1,280
│ │ │ └─ReLU: 4-182 [1, 640, 14, 14] --
│ │ │ └─Conv2d: 4-183 [1, 128, 14, 14] 81,920
│ │ │ └─BatchNorm2d: 4-184 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-185 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-186 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-40 [1, 704, 14, 14] --
│ │ │ └─BatchNorm2d: 4-187 [1, 672, 14, 14] 1,344
│ │ │ └─ReLU: 4-188 [1, 672, 14, 14] --
│ │ │ └─Conv2d: 4-189 [1, 128, 14, 14] 86,016
│ │ │ └─BatchNorm2d: 4-190 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-191 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-192 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-41 [1, 736, 14, 14] --
│ │ │ └─BatchNorm2d: 4-193 [1, 704, 14, 14] 1,408
│ │ │ └─ReLU: 4-194 [1, 704, 14, 14] --
│ │ │ └─Conv2d: 4-195 [1, 128, 14, 14] 90,112
│ │ │ └─BatchNorm2d: 4-196 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-197 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-198 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-42 [1, 768, 14, 14] --
│ │ │ └─BatchNorm2d: 4-199 [1, 736, 14, 14] 1,472
│ │ │ └─ReLU: 4-200 [1, 736, 14, 14] --
│ │ │ └─Conv2d: 4-201 [1, 128, 14, 14] 94,208
│ │ │ └─BatchNorm2d: 4-202 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-203 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-204 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-43 [1, 800, 14, 14] --
│ │ │ └─BatchNorm2d: 4-205 [1, 768, 14, 14] 1,536
│ │ │ └─ReLU: 4-206 [1, 768, 14, 14] --
│ │ │ └─Conv2d: 4-207 [1, 128, 14, 14] 98,304
│ │ │ └─BatchNorm2d: 4-208 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-209 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-210 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-44 [1, 832, 14, 14] --
│ │ │ └─BatchNorm2d: 4-211 [1, 800, 14, 14] 1,600
│ │ │ └─ReLU: 4-212 [1, 800, 14, 14] --
│ │ │ └─Conv2d: 4-213 [1, 128, 14, 14] 102,400
│ │ │ └─BatchNorm2d: 4-214 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-215 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-216 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-45 [1, 864, 14, 14] --
│ │ │ └─BatchNorm2d: 4-217 [1, 832, 14, 14] 1,664
│ │ │ └─ReLU: 4-218 [1, 832, 14, 14] --
│ │ │ └─Conv2d: 4-219 [1, 128, 14, 14] 106,496
│ │ │ └─BatchNorm2d: 4-220 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-221 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-222 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-46 [1, 896, 14, 14] --
│ │ │ └─BatchNorm2d: 4-223 [1, 864, 14, 14] 1,728
│ │ │ └─ReLU: 4-224 [1, 864, 14, 14] --
│ │ │ └─Conv2d: 4-225 [1, 128, 14, 14] 110,592
│ │ │ └─BatchNorm2d: 4-226 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-227 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-228 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-47 [1, 928, 14, 14] --
│ │ │ └─BatchNorm2d: 4-229 [1, 896, 14, 14] 1,792
│ │ │ └─ReLU: 4-230 [1, 896, 14, 14] --
│ │ │ └─Conv2d: 4-231 [1, 128, 14, 14] 114,688
│ │ │ └─BatchNorm2d: 4-232 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-233 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-234 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-48 [1, 960, 14, 14] --
│ │ │ └─BatchNorm2d: 4-235 [1, 928, 14, 14] 1,856
│ │ │ └─ReLU: 4-236 [1, 928, 14, 14] --
│ │ │ └─Conv2d: 4-237 [1, 128, 14, 14] 118,784
│ │ │ └─BatchNorm2d: 4-238 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-239 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-240 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-49 [1, 992, 14, 14] --
│ │ │ └─BatchNorm2d: 4-241 [1, 960, 14, 14] 1,920
│ │ │ └─ReLU: 4-242 [1, 960, 14, 14] --
│ │ │ └─Conv2d: 4-243 [1, 128, 14, 14] 122,880
│ │ │ └─BatchNorm2d: 4-244 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-245 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-246 [1, 32, 14, 14] 36,864
│ │ └─Bottleneck: 3-50 [1, 1024, 14, 14] --
│ │ │ └─BatchNorm2d: 4-247 [1, 992, 14, 14] 1,984
│ │ │ └─ReLU: 4-248 [1, 992, 14, 14] --
│ │ │ └─Conv2d: 4-249 [1, 128, 14, 14] 126,976
│ │ │ └─BatchNorm2d: 4-250 [1, 128, 14, 14] 256
│ │ │ └─ReLU: 4-251 [1, 128, 14, 14] --
│ │ │ └─Conv2d: 4-252 [1, 32, 14, 14] 36,864
│ └─Transition_Layer: 2-10 [1, 512, 7, 7] --
│ │ └─BatchNorm2d: 3-51 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-52 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-53 [1, 512, 14, 14] 524,288
│ │ └─AvgPool2d: 3-54 [1, 512, 7, 7] --
│ └─Dense_Block: 2-11 [1, 1024, 7, 7] --
│ │ └─Bottleneck: 3-55 [1, 544, 7, 7] --
│ │ │ └─BatchNorm2d: 4-253 [1, 512, 7, 7] 1,024
│ │ │ └─ReLU: 4-254 [1, 512, 7, 7] --
│ │ │ └─Conv2d: 4-255 [1, 128, 7, 7] 65,536
│ │ │ └─BatchNorm2d: 4-256 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-257 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-258 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-56 [1, 576, 7, 7] --
│ │ │ └─BatchNorm2d: 4-259 [1, 544, 7, 7] 1,088
│ │ │ └─ReLU: 4-260 [1, 544, 7, 7] --
│ │ │ └─Conv2d: 4-261 [1, 128, 7, 7] 69,632
│ │ │ └─BatchNorm2d: 4-262 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-263 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-264 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-57 [1, 608, 7, 7] --
│ │ │ └─BatchNorm2d: 4-265 [1, 576, 7, 7] 1,152
│ │ │ └─ReLU: 4-266 [1, 576, 7, 7] --
│ │ │ └─Conv2d: 4-267 [1, 128, 7, 7] 73,728
│ │ │ └─BatchNorm2d: 4-268 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-269 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-270 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-58 [1, 640, 7, 7] --
│ │ │ └─BatchNorm2d: 4-271 [1, 608, 7, 7] 1,216
│ │ │ └─ReLU: 4-272 [1, 608, 7, 7] --
│ │ │ └─Conv2d: 4-273 [1, 128, 7, 7] 77,824
│ │ │ └─BatchNorm2d: 4-274 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-275 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-276 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-59 [1, 672, 7, 7] --
│ │ │ └─BatchNorm2d: 4-277 [1, 640, 7, 7] 1,280
│ │ │ └─ReLU: 4-278 [1, 640, 7, 7] --
│ │ │ └─Conv2d: 4-279 [1, 128, 7, 7] 81,920
│ │ │ └─BatchNorm2d: 4-280 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-281 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-282 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-60 [1, 704, 7, 7] --
│ │ │ └─BatchNorm2d: 4-283 [1, 672, 7, 7] 1,344
│ │ │ └─ReLU: 4-284 [1, 672, 7, 7] --
│ │ │ └─Conv2d: 4-285 [1, 128, 7, 7] 86,016
│ │ │ └─BatchNorm2d: 4-286 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-287 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-288 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-61 [1, 736, 7, 7] --
│ │ │ └─BatchNorm2d: 4-289 [1, 704, 7, 7] 1,408
│ │ │ └─ReLU: 4-290 [1, 704, 7, 7] --
│ │ │ └─Conv2d: 4-291 [1, 128, 7, 7] 90,112
│ │ │ └─BatchNorm2d: 4-292 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-293 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-294 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-62 [1, 768, 7, 7] --
│ │ │ └─BatchNorm2d: 4-295 [1, 736, 7, 7] 1,472
│ │ │ └─ReLU: 4-296 [1, 736, 7, 7] --
│ │ │ └─Conv2d: 4-297 [1, 128, 7, 7] 94,208
│ │ │ └─BatchNorm2d: 4-298 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-299 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-300 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-63 [1, 800, 7, 7] --
│ │ │ └─BatchNorm2d: 4-301 [1, 768, 7, 7] 1,536
│ │ │ └─ReLU: 4-302 [1, 768, 7, 7] --
│ │ │ └─Conv2d: 4-303 [1, 128, 7, 7] 98,304
│ │ │ └─BatchNorm2d: 4-304 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-305 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-306 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-64 [1, 832, 7, 7] --
│ │ │ └─BatchNorm2d: 4-307 [1, 800, 7, 7] 1,600
│ │ │ └─ReLU: 4-308 [1, 800, 7, 7] --
│ │ │ └─Conv2d: 4-309 [1, 128, 7, 7] 102,400
│ │ │ └─BatchNorm2d: 4-310 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-311 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-312 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-65 [1, 864, 7, 7] --
│ │ │ └─BatchNorm2d: 4-313 [1, 832, 7, 7] 1,664
│ │ │ └─ReLU: 4-314 [1, 832, 7, 7] --
│ │ │ └─Conv2d: 4-315 [1, 128, 7, 7] 106,496
│ │ │ └─BatchNorm2d: 4-316 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-317 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-318 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-66 [1, 896, 7, 7] --
│ │ │ └─BatchNorm2d: 4-319 [1, 864, 7, 7] 1,728
│ │ │ └─ReLU: 4-320 [1, 864, 7, 7] --
│ │ │ └─Conv2d: 4-321 [1, 128, 7, 7] 110,592
│ │ │ └─BatchNorm2d: 4-322 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-323 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-324 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-67 [1, 928, 7, 7] --
│ │ │ └─BatchNorm2d: 4-325 [1, 896, 7, 7] 1,792
│ │ │ └─ReLU: 4-326 [1, 896, 7, 7] --
│ │ │ └─Conv2d: 4-327 [1, 128, 7, 7] 114,688
│ │ │ └─BatchNorm2d: 4-328 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-329 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-330 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-68 [1, 960, 7, 7] --
│ │ │ └─BatchNorm2d: 4-331 [1, 928, 7, 7] 1,856
│ │ │ └─ReLU: 4-332 [1, 928, 7, 7] --
│ │ │ └─Conv2d: 4-333 [1, 128, 7, 7] 118,784
│ │ │ └─BatchNorm2d: 4-334 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-335 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-336 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-69 [1, 992, 7, 7] --
│ │ │ └─BatchNorm2d: 4-337 [1, 960, 7, 7] 1,920
│ │ │ └─ReLU: 4-338 [1, 960, 7, 7] --
│ │ │ └─Conv2d: 4-339 [1, 128, 7, 7] 122,880
│ │ │ └─BatchNorm2d: 4-340 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-341 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-342 [1, 32, 7, 7] 36,864
│ │ └─Bottleneck: 3-70 [1, 1024, 7, 7] --
│ │ │ └─BatchNorm2d: 4-343 [1, 992, 7, 7] 1,984
│ │ │ └─ReLU: 4-344 [1, 992, 7, 7] --
│ │ │ └─Conv2d: 4-345 [1, 128, 7, 7] 126,976
│ │ │ └─BatchNorm2d: 4-346 [1, 128, 7, 7] 256
│ │ │ └─ReLU: 4-347 [1, 128, 7, 7] --
│ │ │ └─Conv2d: 4-348 [1, 32, 7, 7] 36,864
│ └─BatchNorm2d: 2-12 [1, 1024, 7, 7] 2,048
├─AvgPool2d: 1-2 [1, 1024, 1, 1] --
├─Linear: 1-3 [1, 1000] 1,025,000
==========================================================================================
Total params: 7,978,856
Trainable params: 7,978,856
Non-trainable params: 0
Total mult-adds (G): 2.83
==========================================================================================
Input size (MB): 0.60
Forward/backward pass size (MB): 180.54
Params size (MB): 31.92
Estimated Total Size (MB): 213.06
==========================================================================================
3213
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
