深度学习模型运行的浮点次数FLOPs和训练参数程序获取方法

# 浮点运行次数
# FLOPS：注意全大写，是floating point operations per second的缩写，意指每秒浮点运算次数，理解为计算速度。是一个衡量硬件性能的指标。
# FLOPs：注意s小写，是floating point operations的缩写（s表复数），意指浮点运算数，理解为计算量。可以用来衡量算法/模型的复杂度。
# In TF 2.x you have to use tf.compat.v1.RunMetadata instead of tf.RunMetadata
# To work your code in TF 2.1.0, i have made all necessary changes that are compliant to TF 2.x

# print(tf.__version__)
import tensorflow as tf
# 必须要下面这行代码
tf.compat.v1.disable_eager_execution()
print(tf.__version__)

# 我自己使用的函数
def get_flops_params():
    sess = tf.compat.v1.Session()
    graph = sess.graph
    flops = tf.compat.v1.profiler.profile(graph, options=tf.compat.v1.profiler.ProfileOptionBuilder.float_operation())
    params = tf.compat.v1.profiler.profile(graph, options=tf.compat.v1.profiler.ProfileOptionBuilder.trainable_variables_parameter())
    print('FLOPs: {};    Trainable params: {}'.format(flops.total_float_ops, params.total_parameters))


# 网上推荐的
# sess = tf.compat.v1.Session()
# graph = sess.graph
# stats_graph(graph)
def stats_graph(graph):
    flops = tf.compat.v1.profiler.profile(graph, options=tf.compat.v1.profiler.ProfileOptionBuilder.float_operation())
    # print('FLOPs: {}'.format(flops.total_float_ops))
    params = tf.compat.v1.profiler.profile(graph, options=tf.compat.v1.profiler.ProfileOptionBuilder.trainable_variables_parameter())
    # print('Trainable params: {}'.format(params.total_parameters))
    print('FLOPs: {};    Trainable params: {}'.format(flops.total_float_ops, params.total_parameters))


def get_flops(model):
    run_meta = tf.compat.v1.RunMetadata()
    opts = tf.compat.v1.profiler.ProfileOptionBuilder.float_operation()
    # We use the Keras session graph in the call to the profiler.
    flops = tf.compat.v1.profiler.profile(graph=tf.compat.v1.keras.backend.get_session().graph, run_meta=run_meta, cmd='op', options=opts)
    return flops.total_float_ops  # Prints the "flops" of the model.

# 必须使用tensorflow中的keras才能够获取到FLOPs, 模型中的各个函数都必须使用tensorflow.keras中的函数，和keras混用会报错
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense
from tensorflow.keras.models import Sequential

model = Sequential()
model.add(Conv2D(filters=64, kernel_size=(3, 3), input_shape=(28, 28, 1), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(units=100, activation='relu'))
model.add(Dense(units=10, activation='softmax'))
# 获取模型每一层的参数详情
model.summary()
# 获取模型浮点运算总次数和模型的总参数
get_flops_params()

• model.summary() 运行结果

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 26, 26, 64)        640       
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 13, 13, 64)        0         
_________________________________________________________________
flatten (Flatten)            (None, 10816)             0         
_________________________________________________________________
dense (Dense)                (None, 100)               1081700   
_________________________________________________________________
dense_1 (Dense)              (None, 10)                1010      
=================================================================
Total params: 1,083,350
Trainable params: 1,083,350
Non-trainable params: 0

• get_flops_params()运行结果

==================Model Analysis Report======================
Incomplete shape.
Incomplete shape.

Doc:
scope: The nodes in the model graph are organized by their names, which is hierarchical like filesystem.
param: Number of parameters (in the Variable).

Profile:
node name | # parameters
_TFProfRoot (--/1.08m params)
  conv2d (--/640 params)
    conv2d/bias (64, 64/64 params)
    conv2d/kernel (3x3x1x64, 576/576 params)
  dense (--/1.08m params)
    dense/bias (100, 100/100 params)
    dense/kernel (10816x100, 1.08m/1.08m params)
  dense_1 (--/1.01k params)
    dense_1/bias (10, 10/10 params)
    dense_1/kernel (100x10, 1.00k/1.00k params)

======================End of Report==========================
FLOPs: 2166355;    Trainable params: 1083350