tensorflow 2.5中，采用keras，predict很慢，300倍减少predict运行时间的优化方法

版本如下

import tensorflow as tf
import numpy as np
import time

from tensorflow import keras

print(tf.__version__)

2.5.0

下面设置循环次数，构建模型。

loop_times = 100
input_data = np.zeros((2,8,8))
model  = keras.Sequential([
    keras.layers.Flatten(input_shape=(8,8)),
    keras.layers.Dense(10)
])
model.compile(optimizer='adam',loss='softmax')

测试代码1，需要3.59s

clock = time.time()

for i in range(loop_times):
    res = model.predict(x=input_data)

print(time.time() - clock,'s')
print(res)

输出：

3.599362850189209 s
[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]

优化代码1，需要0.8s

clock = time.time()

for i in range(loop_times):
    res = model.predict(x=tf.data.Dataset.from_tensors(input_data))
    
print(time.time() - clock,'s')
print(res)

输出：

0.8024599552154541 s
[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]

优化代码2，需要0.07s

这里注意输出为tensor

clock = time.time()
for i in range(loop_times):
    res_m = model(input_data,training = False)
    res = np.array(res_m)
    
print(time.time() - clock,'s')
print(res_m)
print(res)

输出：

0.07204914093017578 s
tf.Tensor(
[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]], shape=(2, 10), dtype=float32)

[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]