网络模型打印
最简单的就是 print(model)
import torchvision.models as models
network = models.resnet18()
print(network)
当然,一般需求都比这个多,效果我放在文末尾了。
继续往下看👇
模型参数量
参考github
使用torchinfo, 安装使用 pip install torchinfo
from torchinfo import summary
import torchvision.models as models
network = models.resnet18()
summary(network, (1, 3, 224, 224))
这个模块比较简单,可以很方便的使用,除了模型加载一共两行。
我们简单分析一下后面这些有用的输出,
Total params: 60,192,808
Trainable params: 60,192,808
Non-trainable params: 0
Total mult-adds (G): 11.51
Total params - 模型参数量,上面可以翻译为60.19 M
Trainable params - 模型可训练参数量,假如freeze某些层就会出现下面样子
Total params: 11,689,512
Trainable params: 11,680,104
Non-trainable params: 9,408
Total mult-adds (G): 1.81
这是我把 Conv2d: 1-1 进行参数冻结显示的
继续,
Total mult-adds (G) 这是 MACs 总乘加计数,单位是G(10^9)可以直接用,肯定有人会问,那个FLOPs呢,这个FLOPS我看了很多实现,很难实现一致性,所以建议采用MACs就可以,主要是要对比效果,如果非要转换,大约是 1 MACs 包含 2 FLOPs,这个说法仅供参考。
下面没有冻结哦
Input size (MB): 0.60
Forward/backward pass size (MB): 360.87
Params size (MB): 240.77
Estimated Total Size (MB): 602.25
Input size 输入尺寸,这里是说3x224x224的图片占用的内存大小
Forward/backward pass size (MB) 前向进行推理所占用的显存大小
Params size (MB) 保存模型参数所占用的显存大小
将上面三项加起来就是Estimated Total Size (MB),若显卡显存是24GB,即24576 MB, 则可以计算batch_size大小为 24576/602.25 约等于 40,这个只是大概估计,可能具体情况需要±10?
实现效果1
==========================================================================================
Layer (type:depth-idx) Output Shape Param #
==========================================================================================
ResNet [1, 1000] --
├─Conv2d: 1-1 [1, 64, 112, 112] 9,408
├─BatchNorm2d: 1-2 [1, 64, 112, 112] 128
├─ReLU: 1-3 [1, 64, 112, 112] --
├─MaxPool2d: 1-4 [1, 64, 56, 56] --
├─Sequential: 1-5 [1, 256, 56, 56] --
│ └─Bottleneck: 2-1 [1, 256, 56, 56] --
│ │ └─Conv2d: 3-1 [1, 64, 56, 56] 4,096
│ │ └─BatchNorm2d: 3-2 [1, 64, 56, 56] 128
│ │ └─ReLU: 3-3 [1, 64, 56, 56] --
│ │ └─Conv2d: 3-4 [1, 64, 56, 56] 36,864
│ │ └─BatchNorm2d: 3-5 [1, 64, 56, 56] 128
│ │ └─ReLU: 3-6 [1, 64, 56, 56] --
│ │ └─Conv2d: 3-7 [1, 256, 56, 56] 16,384
│ │ └─BatchNorm2d: 3-8 [1, 256, 56, 56] 512
│ │ └─Sequential: 3-9 [1, 256, 56, 56] 16,896
│ │ └─ReLU: 3-10 [1, 256, 56, 56] --
│ └─Bottleneck: 2-2 [1, 256, 56, 56] --
│ │ └─Conv2d: 3-11 [1, 64, 56, 56] 16,384
│ │ └─BatchNorm2d: 3-12 [1, 64, 56, 56] 128
│ │ └─ReLU: 3-13 [1, 64, 56, 56] --
│ │ └─Conv2d: 3-14 [1, 64, 56, 56] 36,864
│ │ └─BatchNorm2d: 3-15 [1, 64, 56, 56] 128
│ │ └─ReLU: 3-16 [1, 64, 56, 56] --
│ │ └─Conv2d: 3-17 [1, 256, 56, 56] 16,384
│ │ └─BatchNorm2d: 3-18 [1, 256, 56, 56] 512
│ │ └─ReLU: 3-19 [1, 256, 56, 56] --
│ └─Bottleneck: 2-3 [1, 256, 56, 56] --
│ │ └─Conv2d: 3-20 [1, 64, 56, 56] 16,384
│ │ └─BatchNorm2d: 3-21 [1, 64, 56, 56] 128
│ │ └─ReLU: 3-22 [1, 64, 56, 56] --
│ │ └─Conv2d: 3-23 [1, 64, 56, 56] 36,864
│ │ └─BatchNorm2d: 3-24 [1, 64, 56, 56] 128
│ │ └─ReLU: 3-25 [1, 64, 56, 56] --
│ │ └─Conv2d: 3-26 [1, 256, 56, 56] 16,384
│ │ └─BatchNorm2d: 3-27 [1, 256, 56, 56] 512
│ │ └─ReLU: 3-28 [1, 256, 56, 56] --
├─Sequential: 1-6 [1, 512, 28, 28] --
│ └─Bottleneck: 2-4 [1, 512, 28, 28] --
│ │ └─Conv2d: 3-29 [1, 128, 56, 56] 32,768
│ │ └─BatchNorm2d: 3-30 [1, 128, 56, 56] 256
│ │ └─ReLU: 3-31 [1, 128, 56, 56] --
│ │ └─Conv2d: 3-32 [1, 128, 28, 28] 147,456
│ │ └─BatchNorm2d: 3-33 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-34 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-35 [1, 512, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-36 [1, 512, 28, 28] 1,024
│ │ └─Sequential: 3-37 [1, 512, 28, 28] 132,096
│ │ └─ReLU: 3-38 [1, 512, 28, 28] --
│ └─Bottleneck: 2-5 [1, 512, 28, 28] --
│ │ └─Conv2d: 3-39 [1, 128, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-40 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-41 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-42 [1, 128, 28, 28] 147,456
│ │ └─BatchNorm2d: 3-43 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-44 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-45 [1, 512, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-46 [1, 512, 28, 28] 1,024
│ │ └─ReLU: 3-47 [1, 512, 28, 28] --
│ └─Bottleneck: 2-6 [1, 512, 28, 28] --
│ │ └─Conv2d: 3-48 [1, 128, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-49 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-50 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-51 [1, 128, 28, 28] 147,456
│ │ └─BatchNorm2d: 3-52 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-53 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-54 [1, 512, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-55 [1, 512, 28, 28] 1,024
│ │ └─ReLU: 3-56 [1, 512, 28, 28] --
│ └─Bottleneck: 2-7 [1, 512, 28, 28] --
│ │ └─Conv2d: 3-57 [1, 128, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-58 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-59 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-60 [1, 128, 28, 28] 147,456
│ │ └─BatchNorm2d: 3-61 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-62 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-63 [1, 512, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-64 [1, 512, 28, 28] 1,024
│ │ └─ReLU: 3-65 [1, 512, 28, 28] --
│ └─Bottleneck: 2-8 [1, 512, 28, 28] --
│ │ └─Conv2d: 3-66 [1, 128, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-67 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-68 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-69 [1, 128, 28, 28] 147,456
│ │ └─BatchNorm2d: 3-70 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-71 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-72 [1, 512, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-73 [1, 512, 28, 28] 1,024
│ │ └─ReLU: 3-74 [1, 512, 28, 28] --
│ └─Bottleneck: 2-9 [1, 512, 28, 28] --
│ │ └─Conv2d: 3-75 [1, 128, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-76 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-77 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-78 [1, 128, 28, 28] 147,456
│ │ └─BatchNorm2d: 3-79 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-80 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-81 [1, 512, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-82 [1, 512, 28, 28] 1,024
│ │ └─ReLU: 3-83 [1, 512, 28, 28] --
│ └─Bottleneck: 2-10 [1, 512, 28, 28] --
│ │ └─Conv2d: 3-84 [1, 128, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-85 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-86 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-87 [1, 128, 28, 28] 147,456
│ │ └─BatchNorm2d: 3-88 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-89 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-90 [1, 512, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-91 [1, 512, 28, 28] 1,024
│ │ └─ReLU: 3-92 [1, 512, 28, 28] --
│ └─Bottleneck: 2-11 [1, 512, 28, 28] --
│ │ └─Conv2d: 3-93 [1, 128, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-94 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-95 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-96 [1, 128, 28, 28] 147,456
│ │ └─BatchNorm2d: 3-97 [1, 128, 28, 28] 256
│ │ └─ReLU: 3-98 [1, 128, 28, 28] --
│ │ └─Conv2d: 3-99 [1, 512, 28, 28] 65,536
│ │ └─BatchNorm2d: 3-100 [1, 512, 28, 28] 1,024
│ │ └─ReLU: 3-101 [1, 512, 28, 28] --
├─Sequential: 1-7 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-12 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-102 [1, 256, 28, 28] 131,072
│ │ └─BatchNorm2d: 3-103 [1, 256, 28, 28] 512
│ │ └─ReLU: 3-104 [1, 256, 28, 28] --
│ │ └─Conv2d: 3-105 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-106 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-107 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-108 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-109 [1, 1024, 14, 14] 2,048
│ │ └─Sequential: 3-110 [1, 1024, 14, 14] 526,336
│ │ └─ReLU: 3-111 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-13 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-112 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-113 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-114 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-115 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-116 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-117 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-118 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-119 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-120 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-14 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-121 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-122 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-123 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-124 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-125 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-126 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-127 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-128 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-129 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-15 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-130 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-131 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-132 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-133 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-134 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-135 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-136 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-137 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-138 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-16 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-139 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-140 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-141 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-142 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-143 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-144 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-145 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-146 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-147 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-17 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-148 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-149 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-150 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-151 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-152 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-153 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-154 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-155 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-156 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-18 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-157 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-158 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-159 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-160 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-161 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-162 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-163 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-164 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-165 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-19 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-166 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-167 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-168 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-169 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-170 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-171 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-172 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-173 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-174 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-20 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-175 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-176 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-177 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-178 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-179 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-180 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-181 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-182 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-183 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-21 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-184 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-185 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-186 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-187 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-188 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-189 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-190 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-191 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-192 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-22 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-193 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-194 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-195 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-196 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-197 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-198 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-199 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-200 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-201 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-23 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-202 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-203 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-204 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-205 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-206 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-207 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-208 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-209 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-210 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-24 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-211 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-212 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-213 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-214 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-215 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-216 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-217 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-218 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-219 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-25 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-220 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-221 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-222 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-223 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-224 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-225 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-226 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-227 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-228 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-26 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-229 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-230 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-231 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-232 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-233 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-234 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-235 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-236 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-237 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-27 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-238 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-239 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-240 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-241 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-242 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-243 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-244 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-245 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-246 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-28 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-247 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-248 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-249 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-250 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-251 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-252 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-253 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-254 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-255 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-29 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-256 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-257 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-258 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-259 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-260 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-261 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-262 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-263 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-264 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-30 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-265 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-266 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-267 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-268 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-269 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-270 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-271 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-272 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-273 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-31 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-274 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-275 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-276 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-277 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-278 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-279 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-280 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-281 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-282 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-32 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-283 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-284 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-285 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-286 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-287 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-288 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-289 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-290 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-291 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-33 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-292 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-293 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-294 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-295 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-296 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-297 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-298 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-299 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-300 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-34 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-301 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-302 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-303 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-304 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-305 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-306 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-307 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-308 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-309 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-35 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-310 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-311 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-312 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-313 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-314 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-315 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-316 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-317 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-318 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-36 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-319 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-320 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-321 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-322 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-323 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-324 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-325 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-326 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-327 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-37 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-328 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-329 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-330 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-331 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-332 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-333 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-334 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-335 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-336 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-38 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-337 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-338 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-339 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-340 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-341 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-342 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-343 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-344 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-345 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-39 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-346 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-347 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-348 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-349 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-350 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-351 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-352 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-353 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-354 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-40 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-355 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-356 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-357 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-358 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-359 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-360 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-361 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-362 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-363 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-41 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-364 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-365 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-366 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-367 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-368 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-369 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-370 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-371 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-372 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-42 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-373 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-374 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-375 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-376 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-377 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-378 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-379 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-380 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-381 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-43 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-382 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-383 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-384 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-385 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-386 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-387 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-388 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-389 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-390 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-44 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-391 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-392 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-393 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-394 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-395 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-396 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-397 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-398 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-399 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-45 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-400 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-401 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-402 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-403 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-404 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-405 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-406 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-407 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-408 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-46 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-409 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-410 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-411 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-412 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-413 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-414 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-415 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-416 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-417 [1, 1024, 14, 14] --
│ └─Bottleneck: 2-47 [1, 1024, 14, 14] --
│ │ └─Conv2d: 3-418 [1, 256, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-419 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-420 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-421 [1, 256, 14, 14] 589,824
│ │ └─BatchNorm2d: 3-422 [1, 256, 14, 14] 512
│ │ └─ReLU: 3-423 [1, 256, 14, 14] --
│ │ └─Conv2d: 3-424 [1, 1024, 14, 14] 262,144
│ │ └─BatchNorm2d: 3-425 [1, 1024, 14, 14] 2,048
│ │ └─ReLU: 3-426 [1, 1024, 14, 14] --
├─Sequential: 1-8 [1, 2048, 7, 7] --
│ └─Bottleneck: 2-48 [1, 2048, 7, 7] --
│ │ └─Conv2d: 3-427 [1, 512, 14, 14] 524,288
│ │ └─BatchNorm2d: 3-428 [1, 512, 14, 14] 1,024
│ │ └─ReLU: 3-429 [1, 512, 14, 14] --
│ │ └─Conv2d: 3-430 [1, 512, 7, 7] 2,359,296
│ │ └─BatchNorm2d: 3-431 [1, 512, 7, 7] 1,024
│ │ └─ReLU: 3-432 [1, 512, 7, 7] --
│ │ └─Conv2d: 3-433 [1, 2048, 7, 7] 1,048,576
│ │ └─BatchNorm2d: 3-434 [1, 2048, 7, 7] 4,096
│ │ └─Sequential: 3-435 [1, 2048, 7, 7] 2,101,248
│ │ └─ReLU: 3-436 [1, 2048, 7, 7] --
│ └─Bottleneck: 2-49 [1, 2048, 7, 7] --
│ │ └─Conv2d: 3-437 [1, 512, 7, 7] 1,048,576
│ │ └─BatchNorm2d: 3-438 [1, 512, 7, 7] 1,024
│ │ └─ReLU: 3-439 [1, 512, 7, 7] --
│ │ └─Conv2d: 3-440 [1, 512, 7, 7] 2,359,296
│ │ └─BatchNorm2d: 3-441 [1, 512, 7, 7] 1,024
│ │ └─ReLU: 3-442 [1, 512, 7, 7] --
│ │ └─Conv2d: 3-443 [1, 2048, 7, 7] 1,048,576
│ │ └─BatchNorm2d: 3-444 [1, 2048, 7, 7] 4,096
│ │ └─ReLU: 3-445 [1, 2048, 7, 7] --
│ └─Bottleneck: 2-50 [1, 2048, 7, 7] --
│ │ └─Conv2d: 3-446 [1, 512, 7, 7] 1,048,576
│ │ └─BatchNorm2d: 3-447 [1, 512, 7, 7] 1,024
│ │ └─ReLU: 3-448 [1, 512, 7, 7] --
│ │ └─Conv2d: 3-449 [1, 512, 7, 7] 2,359,296
│ │ └─BatchNorm2d: 3-450 [1, 512, 7, 7] 1,024
│ │ └─ReLU: 3-451 [1, 512, 7, 7] --
│ │ └─Conv2d: 3-452 [1, 2048, 7, 7] 1,048,576
│ │ └─BatchNorm2d: 3-453 [1, 2048, 7, 7] 4,096
│ │ └─ReLU: 3-454 [1, 2048, 7, 7] --
├─AdaptiveAvgPool2d: 1-9 [1, 2048, 1, 1] --
├─Linear: 1-10 [1, 1000] 2,049,000
==========================================================================================
Total params: 60,192,808
Trainable params: 60,192,808
Non-trainable params: 0
Total mult-adds (G): 11.51
==========================================================================================
Input size (MB): 0.60
Forward/backward pass size (MB): 360.87
Params size (MB): 240.77
Estimated Total Size (MB): 602.25
==========================================================================================
实现效果2
ResNet(
(conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(layer1): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(1): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(layer2): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(layer3): Sequential(
(0): BasicBlock(
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(layer4): Sequential(
(0): BasicBlock(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
(fc): Linear(in_features=512, out_features=1000, bias=True)
)