Recently, I struggled trying to export a model built with Keras and TensorFlow 2.x in the proper format to make inference with OpenCV’s DNN module. Here’s how I got those desired .pb and .pbtxt optimized for inference graph files!
最近，我努力尝试以正确的格式导出使用Keras和TensorFlow 2.x构建的模型以与OpenCVDNN模块进行推理。 这是我如何为推理图文件优化所需的.pb和.pbtxt的方法！
介绍 (Introduction)
When we finish the process of training our model and want it running as fast as possible on different programming languages using OpenCV cross-platform library or serve it on the web or mobile, we must export our model’s graph in the most efficient format possible, this translates into two stages: freezing and optimizing.
当我们完成训练模型的过程并希望它使用OpenCV跨平台库在不同的编程语言上尽可能快地运行或者在网络或移动设备上提供服务时，我们必须以最有效的格式导出模型图，这转换为两个阶段：冻结和优化。
The process of freezing a TensorFlow model consist of converting its variables into constants that are stored straight in its graph. On the other hand, the process of optimizing consist of deleting nodes that are only necessary on training stage (such as Dropout layers) or inefficient operations.
冻结TensorFlow模型的过程包括将其变量转换为直接存储在其图中的常量。 另一方面，优化过程包括删除仅在训练阶段(例如，辍学层)或效率低下的操作所必需的节点。
This was easy in TensorFlow 1.x because back in that version existed helper functions that do that work for us. Nevertheless, although TensorFlow 2.x still supporting graph freezing through a newer API, I didn’t find any utility to optimize the frozen graph for inference. So, in essence, we will:
在TensorFlow 1.x中这很容易，因为该版本中存在帮助我们完成此工作的辅助函数。 尽管如此，尽管TensorFlow 2.x仍通过更新的API支持图形冻结，但我没有找到任何用于优化冻结图形进行推理的实用程序。 因此，从本质上讲，我们将：
- Freeze the Keras model using TF 2.x: 使用TF 2.x冻结Keras模型：
SavedModel ⇒ GraphDef
保存的模型⇒GraphDef
2. Optimize the frozen graph using TF 1.x:
2.使用TF 1.x优化冻结图：
GraphDef ⇒ GraphDef
GraphDef⇒GraphDef
3. Convert the optimized frozen graph back to SavedModel format using TF 1.x (although it can be also done with TF 2.x):
3.使用TF 1.x将优化的冻结图转换回SavedModel格式(尽管也可以使用TF 2.x来完成)：
GraphDef ⇒ SavedModel
GraphDef⇒保存的模型
0.假设 (0. Assumptions)
For this tutorial, I’ll suppose that you have:
对于本教程，我假设您具有：
- TensorFlow 2.0 installed. 已安装TensorFlow 2.0。
- Anaconda installed. 安装了Anaconda。
Saved your model in a .h5 or SavedModel format and is already loaded, or is available as an object.
以.h5或S avedModel格式保存您的模型，并且已经加载，或者可以作为对象使用。
1.使用TensorFlow 2.x从TF.Keras模型中获取冻结的图 (1. Get the frozen graph out of the TF.Keras model with TensorFlow 2.x)
First, we do the imports.
首先，我们进行进口。
import tensorflow as tf
from tensorflow import keras
from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2
import numpy as npThen, specify the location where you want to save your frozen graph files.
然后，指定要保存冻结图形文件的位置。
#path of the directory where you want to save your model
frozen_out_path = ''# name of the .pb file
frozen_graph_filename = “frozen_graph”model = # Your modelConvert the Keras model to ConcreteFunction format, which is more general:
将Keras模型转换为ConcreteFunction格式，这是更通用的格式：
# Convert Keras model to ConcreteFunction
full_model = tf.function(lambda x: model(x))
full_model = full_model.get_concrete_function(
tf.TensorSpec(model.inputs[0].shape, model.inputs[0].dtype))Once we have our model in the format of ConcreteFunction, we convert its variables to constants.
一旦我们采用了ConcreteFunction格式的模型，就可以将其变量转换为常量。
# Get frozen graph def
frozen_func = convert_variables_to_constants_v2(full_model)
frozen_func.graph.as_graph_def()We are almost done. If you want to inspect the layers operations inside your frozen graph definition and see the name of its input and output tensors (important for the next stage), use this code:
我们快完成了。 如果要检查冻结图定义内的图层操作并查看其输入和输出张量的名称( 对于下一阶段很重要 )，请使用以下代码：
layers = [op.name for op in frozen_func.graph.get_operations()]
print("-" * 60)
print("Frozen model layers: ")
for layer in layers:
print(layer)print("-" * 60)
print("Frozen model inputs: ")
print(frozen_func.inputs)
print("Frozen model outputs: ")
print(frozen_func.outputs)Then, serialize the frozen graph and its text representation to disk.
然后，将冻结的图及其文本表示序列化到磁盘。
tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
logdir=frozen_out_path,
name=f"{frozen_graph_filename}.pb",
as_text=False)tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
logdir=frozen_out_path,
name=f"{frozen_graph_filename}.pbtxt",
as_text=True)完整代码冻结您的Keras模型并保存： (Full code to freeze your Keras model and save it:)
import tensorflow as tf
from tensorflow import keras
from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2
import numpy as np#path of the directory where you want to save your model
frozen_out_path = ''# name of the .pb file
frozen_graph_filename = “frozen_graph”model = # Your model# Convert Keras model to ConcreteFunction
full_model = tf.function(lambda x: model(x))
full_model = full_model.get_concrete_function(
tf.TensorSpec(model.inputs[0].shape, model.inputs[0].dtype))# Get frozen ConcreteFunction
frozen_func = convert_variables_to_constants_v2(full_model)
frozen_func.graph.as_graph_def()layers = [op.name for op in frozen_func.graph.get_operations()]
print("-" * 60)
print("Frozen model layers: ")
for layer in layers:
print(layer)print("-" * 60)
print("Frozen model inputs: ")
print(frozen_func.inputs)
print("Frozen model outputs: ")
print(frozen_func.outputs)# Save frozen graph to disk
tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
logdir=frozen_out_path,
name=f"{frozen_graph_filename}.pb",
as_text=False)# Save its text representation
tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
logdir=frozen_out_path,
name=f"{frozen_graph_filename}.pbtxt",
as_text=True)2.优化冻结图以进行更快的推理 (2. Optimizing the frozen graph for faster inference)
In this stage, we’ll use a helper function in order to optimize the graph for inference available in TensorFlow 1.x, hence, we need to create a virtual environment that contains that version of TensorFlow (I recommend 1.5) using Anaconda (which is quite straightforward).
在此阶段，我们将使用一个辅助函数来优化图形以进行TensorFlow 1.x中的推理，因此，我们需要使用Anaconda(其中建议使用包含TensorFlow版本(我建议1.5))创建一个虚拟环境非常简单)。
Note: Don’t use pip and conda to install packages at the same time in one environment. Check Using Pip in a Conda Environment for more info.
注意 ：不要在一个环境中同时使用pip和conda来安装软件包。 有关更多信息，请检查在Conda环境中使用Pip 。
conda create -n tf15 python=3.7
conda activate tf15
conda install tensorflow=1.5.0Then, in the same virtual environment we use the optimize_for_inference function that takes the following arguments:
然后，在相同的虚拟环境中，我们使用带有以下参数的optimize_for_inference函数：
- input: str, path to the .pb file graph to optimize (the one that we generated earlier) 输入：str，要优化的.pb文件图的路径(我们之前生成的路径)
- output: str, path of the resulting ‘.pb’ file that contains the optimized graph 输出：str，包含优化图形的结果“ .pb”文件的路径
- frozen_graph: bool, whether the input graph is frozen (in our case it does) Frozen_graph：bool，输入图是否冻结(在我们的例子中是冻结的)
- input_names: str, name of the input tensor input_names：str，输入张量的名称
- output_names: str, name of the output tensor output_names：str，输出张量的名称
So, an example of its usage is:
因此，其用法示例如下：
python -m tensorflow.python.tools.optimize_for_inference --input ./model_20K_96_soft_f1/frozen_model/frozen_graph.pb --output ./model_20K_96_soft_f1/optimized/optmized_graph.pb --frozen_graph=True --input_names=x --output_names=IdentityOnce you do that, hopefully you would get your frozen graph optimized written in disk with the .pb format. Next, we will write that graph as text in order to get that ‘.pbtxt’ file using the same virtual environment and a short Python script. You can do the same with a code similar to the presented one in the first step using TensorFlow 2.0; but since we are already on TF 1.5, we’ll do that with that version.
完成此操作后，希望可以将冻结图优化后以.pb格式写入磁盘。 接下来，我们将以图形形式编写该图形，以便使用相同的虚拟环境和简短的Python脚本获取该.pbtxt文件。 您可以使用TensorFlow 2.0，使用类似于第一步中所示的代码进行相同的操作； 但是由于我们已经在TF 1.5上使用，所以我们将使用该版本。
# Needs tensorflow 1.5
import tensorflow as tfoptimized_graph_path = "./model_20K_96_soft_f1/optimized/optmized_graph.pb"output_pbtxt = "./model_20K_96_soft_f1/optimized/optmized_graph.pbtxt"# Read the graph.
with tf.gfile.FastGFile(optimized_graph_path, "rb") as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())# Remove Const nodes.
for i in reversed(range(len(graph_def.node))):
if graph_def.node[i].op == 'Const':
del graph_def.node[i]
for attr in ['T', 'data_format', 'Tshape', 'N', 'Tidx', 'Tdim',
'use_cudnn_on_gpu', 'Index', 'Tperm', 'is_training',
'Tpaddings']:
if attr in graph_def.node[i].attr:
del graph_def.node[i].attr[attr]# Save as text.
tf.train.write_graph(graph_def, "", output_pbtxt, as_text=True)示例：使用Java中的 OpenCVDNN模块加载优化的推理模型(是，Java) (Example: loading an optimized for inference model using OpenCV’s DNN module in Java (yes, Java))
Now you can enjoy the benefits of having frozen and optimized the graph of your model. This will increase significantly its prediction speed and will allow you to use that general serialized model within another libraries in other languages (such as OpenCV in Java or C++, etc…), giving you the power of serving your model and use it with another technologies.
现在，您可以享受冻结和优化模型图的好处。 这将大大提高其预测速度，并使您可以在其他语言的其他库(例如Java或C ++中的OpenCV等)中使用该通用序列化模型，从而为您提供服务模型并与其他技术结合使用的功能。
import org.opencv.dnn.Dnn;
import org.opencv.dnn.Net;public class Main { public static final String MODEL_PATH = ""; // SavedModel format, i.e., the '.pb' file
public static final String WEIGHTS_PATH = ""; // .pbtxt file public static void main(String args[]) {
Net model = Dnn.readNet(MODEL_PATH,WEIGHTS_PATH);
}
}结论 (Conclusions)
Now you are able to serve and deploy your tf.Keras model with the newer and awesome version of TensorFlow, increasing your model’s inference performance, and coming up with better IA solutions.
现在，您可以使用TensorFlow的新版本和强大版本来服务和部署tf.Keras模型，提高模型的推理性能，并提出更好的IA解决方案。
扫一扫
2万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
