tensorflow1.12 to onnx 然后再转tensorRT(这一步没有做)

如果需要打印中间层，请参考：

ONNX前向inference调试

ONNX前向inference调试_SilentOB的博客-CSDN博客

ONNX动态输入尺寸的问题【多输出/多输入】【pytorch/onnx/onnxruntime】

ONNX动态输入尺寸的问题【多输出/多输入】【pytorch/onnx/onnxruntime】_Tsuinosora -CSDN博客

1. 立交桥跳水冠军：Pytorch转ONNX-理论篇
2. 立交桥跳水冠军：Pytorch转ONNX-实战篇1（tracing机制）
3. 立交桥跳水冠军：Pytorch转ONNX-实战篇2（实战踩坑总结）
4. BBuf：ONNX学习笔记
5. AIPredict：使用ONNX部署深度学习和传统机器学习模型
6. BBuf：ONNX再探
7. BBuf：onnx2pytorch和onnx-simplifer新版介绍
8. 大缺弦：onnx simplifier 和 optimizer
9. 模型部署翻车记：pytorch转onnx踩坑实录
10. 糖心他爸：基于onnx的网络裁剪
11. Jack Chen：pytorch->onnx踩坑总结(1)
12. pytorch->onnx踩坑总结(1) - 知乎

主要参考：

tensorflow/tensorflow/tools/graph_transforms at r1.12 · tensorflow/tensorflow · GitHub

keras-tensorflow cuda-cudnn版本对应关系 pytorch-torchision版本对应关系 pytorch下载地址_baidu_40840693的博客-CSDN博客

环境cuda9.0 
tensorflow-1.2  to  onnx
python -mpip install -U tf2onnx -i https://pypi.tuna.tsinghua.edu.cn/simple
python -mpip install onnxruntime -i https://pypi.tuna.tsinghua.edu.cn/simple
python -mpip install onnx -i https://pypi.tuna.tsinghua.edu.cn/simple

下载bazel
https://github.com/bazelbuild/bazel/releases
bazel_4.2.1-linux-x86_64.deb
下载tensorflow1.2的源代码
https://github.com/tensorflow/tensorflow/tree/r1.12/tensorflow/tools/graph_transforms
参考：
https://github.com/tensorflow/tensorflow/tree/r1.12/tensorflow/tools/graph_transforms/README.md
搜索关键字 summarize_graph
bazel build tensorflow/tools/graph_transforms:summarize_graph
bazel-bin/tensorflow/tools/graph_transforms/summarize_graph --in_graph=tensorflow_inception_graph.pb

出错了好像bazel版本过高
(py3.6_tf1.12_cuda9.0) ninghua@apple:SuperPoint/tensorflow-r1.12$ bazel build tensorflow/tools/graph_transforms:summarize_graph
Starting local Bazel server and connecting to it...
ERROR: /home/ninghua/SuperPoint/tensorflow-r1.12/WORKSPACE:3:1: name 'http_archive' is not defined
ERROR: error loading package '': Encountered error while reading extension file 'closure/defs.bzl': no such package '@io_bazel_rules_closure//closure': error loading package 'external': Could not load //external package
INFO: Elapsed time: 1.164s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)

我看到网上有人用bazel-0.5.2 编译tensorflow1.2，我也不确定，找了好几个博客
https://stackoverflow.com/questions/54500193/build-tensorflow-from-source-error-encountered-error-while-reading-extension-fi
https://github.com/tensorflow/tensorflow/issues/24385
https://stackoverflow.com/questions/53707068/error-while-building-tensorflow-from-source/53708415#53708415
卸载之前的
sudo apt-get remove bazel
后来发现官方给出了编译版本
https://www.tensorflow.org/install/source
版本    Python 版本    编译器    构建工具    cuDNN    CUDA
tensorflow-2.5.0    3.6-3.9    GCC 7.3.1    Bazel 3.7.2    8.1    11.2
tensorflow-2.4.0    3.6-3.8    GCC 7.3.1    Bazel 3.1.0    8.0    11.0
tensorflow-2.3.0    3.5-3.8    GCC 7.3.1    Bazel 3.1.0    7.6    10.1
tensorflow-2.2.0    3.5-3.8    GCC 7.3.1    Bazel 2.0.0    7.6    10.1
tensorflow-2.1.0    2.7、3.5-3.7    GCC 7.3.1    Bazel 0.27.1    7.6    10.1
tensorflow-2.0.0    2.7、3.3-3.7    GCC 7.3.1    Bazel 0.26.1    7.4    10.0
tensorflow_gpu-1.15.0    2.7、3.3-3.7    GCC 7.3.1    Bazel 0.26.1    7.4    10.0
tensorflow_gpu-1.14.0    2.7、3.3-3.7    GCC 4.8    Bazel 0.24.1    7.4    10.0
tensorflow_gpu-1.13.1    2.7、3.3-3.7    GCC 4.8    Bazel 0.19.2    7.4    10.0
tensorflow_gpu-1.12.0    2.7、3.3-3.6    GCC 4.8    Bazel 0.15.0    7    9

下载了Bazel 0.15.0
bazel_0.15.0-linux-x86_64.deb
sudo dpkg -i bazel_0.15.0-linux-x86_64.deb
安装出错，没有jdk-8依赖，我怀疑是我的ubuntu20.04导致的，查找资料，发现

后来查看需要通过apt安装依赖
https://gist.github.com/diegopacheco/10f4d7be75574e53e91c49f19bf2613f
https://docs.bazel.build/versions/3.7.0/install-ubuntu.html
通过如下命令安装：
sudo apt install curl gnupg
curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg
sudo mv bazel.gpg /etc/apt/trusted.gpg.d/
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
sudo apt update

查看都有哪些版本
sudo apt-cache madison
sudo apt install bazel-1.0.0
sudo ln -s /usr/bin/bazel-1.0.0 /usr/bin/bazel

尝试后还是不行

我再试试
下载：
bazel-0.15.0-installer-linux-x86_64.sh
他比bazel_0.15.0-linux-x86_64.deb更大，可能会解决依赖问题
参考https://hackmd.io/@kcchien/BJzHPQdSN?type=view

cd ~/Downloads
# 安裝前置必要的packages
sudo apt-get install git pkg-config zip g++ zlib1g-dev unzip python
# 加上执行权限
chmod +x bazel-0.15.0-installer-linux-x86_64.sh
# 执行bazel安裝
./bazel-0.15.0-installer-linux-x86_64.sh --user
东西被安装到了
sudo ln -s /home/ninghua/.bazel/bin/bazel /usr/bin/bazel

运行
bazel build tensorflow/tools/graph_transforms:summarize_graph
编译过程跑起来了，但是我发现一件事
怎么是把tensorflow完整的在编译。。。。。。。
bazel-bin/tensorflow/tools/graph_transforms/summarize_graph --in_graph=tensorflow_inception_graph.pb

发现不好使，因为我的pb有点问题，参考：
https://stackoverflow.com/questions/51834382/bazel-error-parsing-tf-estimator-model

直接使用命令：
https://github.com/onnx/tensorflow-onnx
python -m tf2onnx.convert --saved-model SuperPoint/output/saved_models/sp_v6 --output model.onnx
遇到了一个算子不可以使用
NonMaxSuppressionV3  ScatterNd
因为这个命令默认--opset 9
通过查找资料，发现
https://github.com/onnx/tensorflow-onnx/blob/master/support_status.md
该算子必须10-14
NonMaxSuppressionV3    10 ~ 14
ScatterNd    11 ~ 14
python -m tf2onnx.convert --saved-model SuperPoint/output/saved_models/sp_v6 --output model.onnx --opset 11

onnx参考：
ONNX学习笔记
https://zhuanlan.zhihu.com/p/346511883
https://github.com/onnx/onnx-docker/blob/master/onnx-ecosystem/inference_demos/resnet50_modelzoo_onnxruntime_inference.ipynb
https://github.com/onnx/tutorials/blob/master/tutorials/TensorflowToOnnx-1.ipynb

通过软件Netron-5.2.1.AppImage 打开onnx
知道输入和输出

举个例子，比如：
https://github.com/rpautrat/SuperPoint/blob/master/pretrained_models/sp_v6.tgz
python -m tf2onnx.convert --saved-model /home/ninghua/深度学习/特征点检测/SuperPoint/output/saved_models/sp_v6 --output model.onnx --opset 11
可以看到
输入
name: image
type: float32[?, ?, ?, 1]
输出
name: descriptors
type: float32[1,?,?,256]
name: prob
type: float32[1,?,?]
name: logits
type: float32[1,?,?,65]
name: pred
type: int32[1,?,?]
name: descriptors_raw
type: float32[1,?,?,256]
name: prob_nms
type: float32[1,?,?]

但是我实际使用的时候，发现
但是网络输出的顺序和这个顺序好像不一致，所以我们必须有一个方法去弄明白输出的时候各自对应关系

参考：

ONNX and Azure Machine Learning: Create and accelerate ML models

ONNX models: Optimize inference - Azure Machine Learning | Microsoft Docs

speeding-up-deep-learning-inference-using-tensorflow-onnx-and-tensorrt

Speeding Up Deep Learning Inference Using TensorFlow, ONNX, and NVIDIA TensorRT | NVIDIA Developer Blog

(OPTIONAL) EXPORTING A MODEL FROM PYTORCH TO ONNX AND RUNNING IT USING ONNX RUNTIME

(optional) Exporting a Model from PyTorch to ONNX and Running it using ONNX Runtime — PyTorch Tutorials 1.9.0+cu102 documentation

对于代码：

https://github.com/rpautrat/SuperPoint

完成tensorflow1.2 到 onnx的转换

几个重要库的版本

numpy                 1.19.5
onnx                  1.10.1
onnxruntime           1.8.1
tensorboard           1.12.2
tensorflow-gpu        1.12.3
tf2onnx               1.9.2

import os
import sys
import onnx
from onnx import numpy_helper
import onnxruntime
import numpy as np

from PIL import Image
from PIL import Image, ImageDraw, ImageFont
import matplotlib.pyplot as plt
import matplotlib.patches as patches
import time
import cv2

def preprocess_image(img_file, img_size):
    img = cv2.imread(img_file, cv2.IMREAD_COLOR)
    img = cv2.resize(img, img_size)
    img_orig = img.copy()

    img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    img = np.expand_dims(img, 2)
    img = img.astype(np.float32)
    img_preprocessed = img / 255.

    return img_preprocessed, img_orig

def extract_SIFT_keypoints_and_descriptors(img):
    gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    sift = cv2.xfeatures2d.SIFT_create()
    kp, desc = sift.detectAndCompute(np.squeeze(gray_img), None)

    return kp, desc


def extract_superpoint_keypoints_and_descriptors(keypoint_map, descriptor_map,
                                                 keep_k_points=1000):

    def select_k_best(points, k):
        """ Select the k most probable points (and strip their proba).
        points has shape (num_points, 3) where the last coordinate is the proba. """
        sorted_prob = points[points[:, 2].argsort(), :2]
        start = min(k, points.shape[0])
        return sorted_prob[-start:, :]

    # Extract keypoints
    keypoints = np.where(keypoint_map > 0)
    prob = keypoint_map[keypoints[0], keypoints[1]]
    keypoints = np.stack([keypoints[0], keypoints[1], prob], axis=-1)

    keypoints = select_k_best(keypoints, keep_k_points)
    keypoints = keypoints.astype(int)

    # Get descriptors for keypoints
    desc = descriptor_map[keypoints[:, 0], keypoints[:, 1]]

    # Convert from just pts to cv2.KeyPoints
    keypoints = [cv2.KeyPoint(p[1], p[0], 1) for p in keypoints]

    return keypoints, desc


def match_descriptors(kp1, desc1, kp2, desc2):
    # Match the keypoints with the warped_keypoints with nearest neighbor search
    bf = cv2.BFMatcher(cv2.NORM_L2, crossCheck=True)
    matches = bf.match(desc1, desc2)
    matches_idx = np.array([m.queryIdx for m in matches])
    m_kp1 = [kp1[idx] for idx in matches_idx]
    matches_idx = np.array([m.trainIdx for m in matches])
    m_kp2 = [kp2[idx] for idx in matches_idx]

    return m_kp1, m_kp2, matches


def compute_homography(matched_kp1, matched_kp2):
    matched_pts1 = cv2.KeyPoint_convert(matched_kp1)
    matched_pts2 = cv2.KeyPoint_convert(matched_kp2)

    # Estimate the homography between the matches using RANSAC
    H, inliers = cv2.findHomography(matched_pts1[:, [1, 0]],
                                    matched_pts2[:, [1, 0]],
                                    cv2.RANSAC)
    inliers = inliers.flatten()
    return H, inliers

session = onnxruntime.InferenceSession('/home/ninghua/深度学习/特征点检测/SuperPoint/output/saved_models/sp_v6/model.onnx')

session.get_modelmeta()
input_information = session.get_inputs()
outputs_information = session.get_outputs()
for tensor_ind, tensor_val in enumerate(input_information):
    print("input_name index{} name:{}  type:{}".format(tensor_ind, tensor_val.name, tensor_val.type))
for tensor_ind, tensor_val in enumerate(outputs_information):
    print("output_name index{} name:{}  type:{}".format(tensor_ind, tensor_val.name, tensor_val.type))

'''
通过软件Netron-5.2.1.AppImage 打开onnx
知道输入和输出
输入
name: image
type: float32[?, ?, ?, 1]
输出
name: descriptors
type: float32[1,?,?,256]
name: prob
type: float32[1,?,?]
name: logits
type: float32[1,?,?,65]
name: pred
type: int32[1,?,?]
name: descriptors_raw
type: float32[1,?,?,256]
name: prob_nms
type: float32[1,?,?]

但是网络输出的顺序和这个顺序好像不一致
网络如下：
(1, 60, 80, 256)
(1, 480, 640)
(1, 60, 80, 256)
(1, 480, 640)
(1, 480, 640)
(1, 480, 640, 256)
'''
img_size = (640, 480)
keep_k_best = 1000

img1_file = '/home/ninghua/深度学习/特征点检测/SuperPoint/dataset/graf1.png'
img1, img1_orig = preprocess_image(img1_file, img_size)
input_img1 = np.expand_dims(img1, 0).astype('float32')
res = session.run(None, {"image": input_img1})
'''
(1, 60, 80, 256)
(1, 480, 640)
(1, 60, 80, 256)
(1, 480, 640)
(1, 480, 640)
(1, 480, 640, 256)
'''
output_prob_nms_tensor = res[4]
output_desc_tensors = res[5]
keypoint_map1 = np.squeeze(output_prob_nms_tensor)
descriptor_map1 = np.squeeze(output_desc_tensors)
kp1, desc1 = extract_superpoint_keypoints_and_descriptors(
        keypoint_map1, descriptor_map1, keep_k_best)


img2_file = '/home/ninghua/深度学习/特征点检测/SuperPoint/dataset/graf3.png'
img2, img2_orig = preprocess_image(img2_file, img_size)
input_img2 = np.expand_dims(img2, 0).astype('float32')
res = session.run(None, {"image": input_img2})
output_prob_nms_tensor = res[4]
output_desc_tensors = res[5]
keypoint_map2 = np.squeeze(output_prob_nms_tensor)
descriptor_map2 = np.squeeze(output_desc_tensors)
kp2, desc2 = extract_superpoint_keypoints_and_descriptors(
        keypoint_map2, descriptor_map2, keep_k_best)


# Match and get rid of outliers
m_kp1, m_kp2, matches = match_descriptors(kp1, desc1, kp2, desc2)
H, inliers = compute_homography(m_kp1, m_kp2)

# Draw SuperPoint matches
matches = np.array(matches)[inliers.astype(bool)].tolist()
matched_img = cv2.drawMatches(img1_orig, kp1, img2_orig, kp2, matches,
                                None, matchColor=(0, 255, 0),
                                singlePointColor=(0, 0, 255))

cv2.imshow("SuperPoint matches", matched_img)

# Compare SIFT matches
sift_kp1, sift_desc1 = extract_SIFT_keypoints_and_descriptors(img1_orig)
sift_kp2, sift_desc2 = extract_SIFT_keypoints_and_descriptors(img2_orig)
sift_m_kp1, sift_m_kp2, sift_matches = match_descriptors(
        sift_kp1, sift_desc1, sift_kp2, sift_desc2)
sift_H, sift_inliers = compute_homography(sift_m_kp1, sift_m_kp2)

# Draw SIFT matches
sift_matches = np.array(sift_matches)[sift_inliers.astype(bool)].tolist()
sift_matched_img = cv2.drawMatches(img1_orig, sift_kp1, img2_orig,
                                    sift_kp2, sift_matches, None,
                                    matchColor=(0, 255, 0),
                                    singlePointColor=(0, 0, 255))
cv2.imshow("SIFT matches", sift_matched_img)

cv2.waitKey(0)