最近看论文发现有些作者会把模型的浮点运算数给展现出来,自己也学习一下,记录下来方便以后查阅。
PyTorch 打印模型的FLOPs(torchstat)
区分FLOPs和FLOPS
FLOPS
:注意全大写,是floating point operations per second的缩写,意指每秒浮点运算次数,理解为计算速度。是一个衡量硬件性能的指标。
FLOPs
:注意s小写,是floating point operations的缩写(s表复数),意指浮点运算数,理解为计算量。可以用来衡量算法/模型的复杂度。
注意在深度学习中,我们用的是FLOPs,也就是说计算量,即用来衡量算法/模型的复杂度。
MACs 和 MAdds
(Multiply–Accumulate Operations) 即乘加累积操作数,常常与FLOPs概念混淆,实际上1MACs包含一个乘法操作与一个加法操作,大约包含2FLOPs。通常MACs与FLOPs存在一个2倍的关系。MACs和MAdds说的是一个东西。
MAC
(memory access cost)即内存使用量,用来评价模型在运行时的内存占用情况。1x1卷积的FLOPs为 2 * H * W * Cin * Cout。对应的MAC为H * W * (Cin + Cout) + Cin * Cout(这里假定内存足够)。
安装 torchstat
pip install torchstat
import torch
import torch.nn as nn
from torchstat import stat
class Corr_CNN(nn.Module):
def __init__(self, Filters, channels, dropoutRate_1, dropoutRate_2, n_classes):
super(Corr_CNN, self).__init__()
self.conv_1 = nn.Conv2d(
in_channels=1,
out_channels=Filters,
kernel_size=(1, channels),
bias=False
)
self.activate_1 = nn.ReLU()
self.bn_1 = nn.BatchNorm2d(num_features=Filters)
self.dropout_1 = nn.Dropout(p=dropoutRate_1)
self.conv_2 = nn.Conv2d(
in_channels=Filters,
out_channels=Filters,
kernel_size=(channels, 1),
bias=False
)
self.activate_2 = nn.ReLU()
self.bn_2 = nn.BatchNorm2d(num_features=Filters)
self.dropout_2 = nn.Dropout(p=dropoutRate_2)
self.fc = nn.Linear(
in_features=Filters,
out_features=n_classes,
)
self.softmax = nn.Softmax(dim=-1)
def forward(self, x):
# input shape (batch_size, C, C)
if len(x.shape) is not 4:
x = torch.unsqueeze(x, 1)
# input shape (batch_size, 1, C, C)
x = self.conv_1(x)
x = self.activate_1(x)
x = self.bn_1(x)
x = self.dropout_1(x)
x = self.conv_2(x)
x = self.activate_2(x)
x = self.bn_2(x)
x = self.dropout_2(x)
x = x.view(x.size()