导出模型:
将tensorflow模型导出为.pb文件
tf.keras.models.save_model(model,"./models/pb_/")
将.pb文件转换为.onnx文件(命令行操作)
python -m tf2onnx.convert --saved-model ./pb_ --opset 18 --output model.onnx
python下查看onnx结构:
import onnx
def print_graph_nodes(model_path):
# 加载 ONNX 模型
model = onnx.load(model_path)
# 遍历所有图节点并打印节点信息
for node in model.graph.node:
node_type = node.op_type
node_name = node.name
print(f'Node Type: {node_type}, Node Name: {node_name}')
if __name__ == '__main__':
onnx_model_file = './models/model.onnx'
print_graph_nodes(onnx_model_file)
测试onnx模型:
import onnxruntime
# import numpy as np
# input1 = np.random.random((1, 7, 7, 3)).astype('float32')
ort_sess = onnxruntime.InferenceSession("./models/model.onnx")
ort_inputs = {ort_sess.get_inputs()[0].name: input1 }
ort_outs = ort_sess.run(None, ort_inputs)
RKNN环境搭建:ubuntu 22.04.4,Python3.10,RKNN-Toolkit2-2.0.0,
在~/.config/pip/pip.conf文件中设置镜像以下载tensorflow-2.8.0:
index-url = https://pypi.tuna.tsinghua.edu.cn/simple
依次执行
sudo apt-get install python3 python3-dev python3-pip
sudo apt-get install libxslt1-dev zlib1g zlib1g-dev libglib2.0-0 libsm6 libgl1-mesa-glx libprotobuf-dev gcc
pip3 install -r doc/requirements_cp310.txt
ip3 install packages/rknn_toolkit2-2.0.0b0+9bab5682-cp310-cp310-linux_x86_64.whl
使用rknn库转换模型(Linux系统)
实际部署后,将动态输入修改为静态,由于python API没有动态接口的讲解和示例,实际运行也不成功。
同时去掉量化:
rknn.config(dynamic_input = [[[1,600,1]]],
target_platform = 'rk3568')
ret = rknn.load_onnx(model = './model.onnx')
ret = rknn.build(do_quantization=True, dataset='./dataset.txt')
改为:
rknn.config(target_platform = 'rk3568')
ret = rknn.load_onnx(model = './model.onnx', inputs = ['input'], input_size_list = [[1,600,1]])
ret = rknn.build(do_quantization=False)
原代码:
import numpy as np
from rknn.api import RKNN
rknn = RKNN(verbose=True)
rknn.config(dynamic_input = [[[1,600,1]]],
target_platform = 'rk3568')
if __name__ == '__main__':
# Load model
print('--> Loading model')
#ret = rknn.load_tflite(model = './quantized_saved_model.tflite')
ret = rknn.load_onnx(model = './model.onnx')
if ret != 0:
print('Load model failed!')
exit(ret)
print('done')
# Build model
print('--> Building model')
ret = rknn.build(do_quantization=True, dataset='./dataset.txt')
if ret != 0:
print('Build model failed!')
exit(ret)
print('done')
# Export rknn model
print('--> Export rknn model')
ret = rknn.export_rknn('./mobilenet_v1.rknn')
if ret != 0:
print('Export rknn model failed!')
exit(ret)
print('done')
rknn.release()
待解决问题:模型中使用了talking head attention,模型使用tf.sinsum构建。tf.sinsum会导致构建RKNN模型失败,报错提示使用了动态图。
p_attn_talk = tf.convert_to_tensor(tf.einsum('ij,ajbk->aibk', self._post_softmax_weight, p_attn))
解决方案:
修改为:
_pre_softmax_weight = np.random.randint(0, 10, size=(8, 8))
scores = np.random.randint(0, 10, size=(1, 8, 300, 300))
scores_rhp = tf.reshape(scores,(tf.shape(scores)[0],tf.shape(scores)[1],tf.shape(scores)[2]*tf.shape(scores)[3]))
print(scores_rhp.shape)
scores_talk = tf.matmul(_pre_softmax_weight, scores_rhp)
print(scores_talk.shape)
scores_talk = tf.reshape(scores_talk,(tf.shape(scores)[0],tf.shape(scores)[1],tf.shape(scores)[2],tf.shape(scores)[3]))
print(scores_talk.shape)
scores_talk_1 = tf.convert_to_tensor(tf.einsum('ij,ajbk->aibk', _pre_softmax_weight, scores))
print("\n结果是否相等:", np.allclose(scores_talk.numpy(), scores_talk_1.numpy()))
修改后的模型可以继续使用原始权重数据,效果基本不变。