onnxruntime C++推理

元气少女缘结神

已于 2024-01-16 22:43:16 修改

阅读量914

点赞数 1

分类专栏： AI Basises 文章标签： c++

于 2023-09-19 13:51:45 首次发布

本文链接：https://blog.csdn.net/wd1603926823/article/details/132270747

版权

AI Basises 专栏收录该内容

18 篇文章 0 订阅

订阅专栏

Opencv C++ onnx模型推理

最先想用的就是opencv的dnn模块对onnx模型推理，其实之前也用过dnn模块不过以前推理的是tensorflow模型，当时是成功的。我以为这次onnx模型推理也没问题，结果：

cv::dnn::Net thenet=cv::dnn::readNetFromONNX("model.onnx");
		//为什么图像blob行列错误，为什么是-1,-1呢 ？？
		Mat blob=cv::dnn::blobFromImage(inputimg,1.0/255);
		thenet.setInput(blob);
		Mat output=thenet.forward();

加载模型是ok的，结果blobFromImage返回的blob行列是-1、-1？？？为什么blob是3维或4维？！

onnxruntime C++ ubuntu源码编译

准备编译出动态库，然后调用推理。先看ONNX Runtime | Home官网，纠结了一下还是先搞cpu版本。

适合的onnxruntime版本

根据NVIDIA - CUDA | onnxruntime 知道CUDA版本与配套的onnxruntime版本，我根据自己的CUDA版本需要下载 onnxruntime: Github onnxruntime镜像 - Gitee.com 这个版本，网好的也可以在https://github.com/microsoft/onnxruntime/releases?page=2 这里下载。下载完毕，解压：然后运行

./build.sh --config RelWithDebInfo --build_shared_lib --parallel --skip_submodule_sync --skip_tests

numpy版本报错

报错日志如下，原因就是网不好，需要的版本无法下载下来。

2023-08-09 14:49:01,895 build [DEBUG] - Defaulting to running update, build [and test for native builds].
2023-08-09 14:49:01,896 build [INFO] - Build started
2023-08-09 14:49:01,896 util.run [INFO] - Running subprocess in '/home/jumper/workspace/onnxruntime_1.6.0'
['/usr/bin/python3', '-m', 'pip', 'install', '--trusted-host', 'files.pythonhosted.org', 'setuptools', 'wheel', 'pytest', 'numpy>=1.16.6', 'sympy>=1.1', 'packaging', 'cerberus']
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Requirement already satisfied: setuptools in /usr/local/lib/python3.5/dist-packages (41.0.1)
Requirement already satisfied: wheel in /root/.local/lib/python3.5/site-packages (0.33.4)
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)",)': /simple/pytest/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)",)': /simple/pytest/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)",)': /simple/pytest/
Collecting pytest
  Using cached pytest-6.1.2-py3-none-any.whl (272 kB)
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)",)': /simple/numpy/
Collecting numpy>=1.16.6
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)",)': /packages/b5/36/88723426b4ff576809fec7d73594fe17a35c27f8d01f93637637a29ae25b/numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl
  WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f8f783b9668>: Failed to establish a new connection: [Errno 101] 网络不可达',)': /packages/b5/36/88723426b4ff576809fec7d73594fe17a35c27f8d01f93637637a29ae25b/numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl
  WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)",)': /packages/b5/36/88723426b4ff576809fec7d73594fe17a35c27f8d01f93637637a29ae25b/numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl
  WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f8f783b9ac8>: Failed to establish a new connection: [Errno 101] 网络不可达',)': /packages/b5/36/88723426b4ff576809fec7d73594fe17a35c27f8d01f93637637a29ae25b/numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl
  WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f8f783b9c50>: Failed to establish a new connection: [Errno 101] 网络不可达',)': /packages/b5/36/88723426b4ff576809fec7d73594fe17a35c27f8d01f93637637a29ae25b/numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl
  Downloading numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl (19.9 MB)
     |▎                               | 204 kB 27 kB/s eta 0:12:01ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 171, in _merge_into_criterion
    crit = self.state.criteria[name]
KeyError: 'numpy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 438, in _error_catcher
    yield
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 519, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/cachecontrol/filewrapper.py", line 62, in read
    data = self.__fp.read(amt)
  File "/usr/lib/python3.5/http/client.py", line 462, in read
    n = self.readinto(b)
  File "/usr/lib/python3.5/http/client.py", line 502, in readinto
    n = self.fp.readinto(b)
  File "/usr/lib/python3.5/socket.py", line 575, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.5/ssl.py", line 929, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.5/ssl.py", line 791, in read
    return self._sslobj.read(len, buffer)
  File "/usr/lib/python3.5/ssl.py", line 575, in read
    v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/cli/base_command.py", line 223, in _main
    status = self.run(options, args)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/cli/req_command.py", line 180, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/commands/install.py", line 321, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 122, in resolve
    requirements, max_rounds=try_to_avoid_resolution_too_deep,
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 453, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 318, in resolve
    name, crit = self._merge_into_criterion(r, parent=None)
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _merge_into_criterion
    crit = Criterion.from_requirement(self._p, requirement, parent)
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 82, in from_requirement
    if not cands:
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/structs.py", line 124, in __bool__
    return bool(self._sequence)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 99, in __bool__
    return any(self)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 244, in iter_index_candidates
    version=ican.version,
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 169, in _make_candidate_from_link
    name=name, version=version,
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 302, in __init__
    version=version,
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 144, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 222, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
    self._ireq, parallel_builds=True,
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 480, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 505, in _prepare_linked_requirement
    self.download_dir, hashes,
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 257, in unpack_url
    hashes=hashes,
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 130, in get_http_url
    from_path, content_type = download(link, temp_dir.path)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/network/download.py", line 163, in __call__
    for chunk in chunks:
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/cli/progress_bars.py", line 168, in iter
    for x in it:
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/network/utils.py", line 88, in response_chunks
    decode_content=False,
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 576, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 541, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 443, in _error_catcher
    raise ReadTimeoutError(self._pool, None, "Read timed out.")
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
Traceback (most recent call last):
  File "/home/jumper/workspace/onnxruntime_1.6.0/tools/ci_build/build.py", line 2088, in <module>
    sys.exit(main())
  File "/home/jumper/workspace/onnxruntime_1.6.0/tools/ci_build/build.py", line 2003, in main
    install_python_deps()
  File "/home/jumper/workspace/onnxruntime_1.6.0/tools/ci_build/build.py", line 585, in install_python_deps
    'files.pythonhosted.org'] + dep_packages)
  File "/home/jumper/workspace/onnxruntime_1.6.0/tools/ci_build/build.py", line 523, in run_subprocess
    return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
  File "/home/jumper/workspace/onnxruntime_1.6.0/tools/python/util/run.py", line 42, in run
    env=env, shell=shell)
  File "/usr/lib/python3.5/subprocess.py", line 708, in run
    output=stdout, stderr=stderr)

解决办法：另开窗口下载指定版本

pip install -U numpy==1.16.6 -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install -U sympy==1.6.2 -i https://pypi.tuna.tsinghua.edu.cn/simple

pyinstaller报错

重新运行后，出现pyinstaller报错，日志如下：

Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /usr/local/lib/python3.5/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpkg3cye79
       cwd: /tmp/pip-install-__e1hgq0/cerberus_f4b541de90114b768be7d7f6ac4d2370
  Complete output (10 lines):
  Traceback (most recent call last):
    File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
      main()
    File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pep517/_in_process.py", line 108, in get_requires_for_build_wheel
      backend = _build_backend()
    File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pep517/_in_process.py", line 99, in _build_backend
      obj = getattr(obj, path_part)
  AttributeError: module 'setuptools.build_meta' has no attribute '__legacy__'

解决办法如下：

pip install pyinstaller

第三方库缺少报错

重新运行后，又报错说缺少googletest、flatbuffers、nsync、date、protobuf、re2等这些第三方库，日志如下：

CMake Error at CMakeLists.txt:478 (add_subdirectory):
  The source directory

    /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/googletest

  does not contain a CMakeLists.txt file.


CMake Error at CMakeLists.txt:479 (set_target_properties):
  set_target_properties Can not find target to add properties to: gmock


CMake Error at CMakeLists.txt:480 (set_target_properties):
  set_target_properties Can not find target to add properties to: gmock_main


CMake Error at CMakeLists.txt:481 (set_target_properties):
  set_target_properties Can not find target to add properties to: gtest


CMake Error at CMakeLists.txt:482 (set_target_properties):
  set_target_properties Can not find target to add properties to: gtest_main


CMake Error at CMakeLists.txt:483 (add_library):
  add_library cannot create ALIAS target "GTest::gmock" because target
  "gmock" does not already exist.


CMake Error at CMakeLists.txt:484 (add_library):
  add_library cannot create ALIAS target "GTest::gmock_main" because target
  "gmock_main" does not already exist.


CMake Error at CMakeLists.txt:485 (add_library):
  add_library cannot create ALIAS target "GTest::gtest" because target
  "gtest" does not already exist.


CMake Error at CMakeLists.txt:486 (add_library):
  add_library cannot create ALIAS target "GTest::gtest_main" because target
  "gtest_main" does not already exist.


CMake Error at CMakeLists.txt:526 (add_subdirectory):
  The source directory

    /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/nsync

  does not contain a CMakeLists.txt file.


Use protobuf from submodule
CMake Error at CMakeLists.txt:565 (add_subdirectory):
  add_subdirectory given source
  "/home/jumper/workspace/onnxruntime_1.6.0/cmake/external/protobuf/cmake"
  which is not an existing directory.


CMake Error at CMakeLists.txt:577 (add_library):
  add_library cannot create ALIAS target "protobuf::libprotobuf" because
  target "libprotobuf-lite" does not already exist.


CMake Error at CMakeLists.txt:626 (add_subdirectory):
  The source directory

    /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/date

  does not contain a CMakeLists.txt file.


CMake Error at CMakeLists.txt:642 (add_subdirectory):
  The source directory

    /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2

  does not contain a CMakeLists.txt file.


CMake Error at CMakeLists.txt:643 (set_target_properties):
  set_target_properties Can not find target to add properties to: re2


CMake Error at CMakeLists.txt:644 (add_library):
  add_library cannot create ALIAS target "re2::re2" because target "re2" does
  not already exist.


CMake Error at CMakeLists.txt:837 (add_subdirectory):
  The source directory

    /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/onnx

  does not contain a CMakeLists.txt file.


CMake Error at CMakeLists.txt:842 (target_compile_definitions):
  Cannot specify compile definitions for target "onnx" which is not built by
  this project.


CMake Error at CMakeLists.txt:844 (target_compile_definitions):
  Cannot specify compile definitions for target "onnx" which is not built by
  this project.


CMake Error at CMakeLists.txt:846 (set_target_properties):
  set_target_properties Can not find target to add properties to: onnx


CMake Error at CMakeLists.txt:847 (set_target_properties):
  set_target_properties Can not find target to add properties to: onnx_proto


CMake Error at CMakeLists.txt:885 (add_subdirectory):
  The source directory

    /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/flatbuffers

  does not contain a CMakeLists.txt file.


CMake Warning at flake8.cmake:19 (message):
  Could not find 'flake8' to check python scripts.  Please install flake8
  using pip.
Call Stack (most recent call first):
  CMakeLists.txt:1501 (include)

解决办法：可以看到onnxruntime: Forked from OnnxRuntime - Gitee.com 这里每个第三方库点开都是空的，所以需要自己另行下载解压后拷贝到对应位置。以googletest为例：

https://github.com/google/googletest

下载完，然后运行：

cd googletest         
mkdir build           
cd build
cmake .. 
make
make install

可选是否运行test : make test && make test install

再如缺少 nsync 就是在GitHub - google/nsync: nsync is a C library that exports various synchronization primitives, such as mutexes 这里下载，然后也是如googletest一样编译。

对于protobuf：

在这里下载解压并拷贝到onnxruntime对应位置 https://github.com/protocolbuffers/protobuf/tree/498de9f761bef56a032815ee44b6e6dbe0892cc4
./autogen.sh
./configure
make
make install
ldconfig

所有第三方库安装好后，重新运行。

date_inference报错

日志如下，

2023-08-09 20:21:03,982 build [DEBUG] - Defaulting to running update, build [and test for native builds].
2023-08-09 20:21:03,983 build [INFO] - Build started
2023-08-09 20:21:03,983 util.run [INFO] - Running subprocess in '/home/jumper/workspace/onnxruntime_1.6.0'
['/usr/bin/python3', '-m', 'pip', 'install', '--trusted-host', 'files.pythonhosted.org', 'setuptools', 'wheel', 'pytest', 'numpy>=1.16.6', 'sympy>=1.1', 'packaging', 'cerberus']
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Requirement already satisfied: setuptools in /usr/local/lib/python3.5/dist-packages (41.0.1)
Requirement already satisfied: wheel in /root/.local/lib/python3.5/site-packages (0.33.4)
Requirement already satisfied: pytest in /usr/local/lib/python3.5/dist-packages (6.1.2)
Requirement already satisfied: numpy>=1.16.6 in /usr/local/lib/python3.5/dist-packages (1.16.6)
Requirement already satisfied: sympy>=1.1 in /usr/local/lib/python3.5/dist-packages (1.6.2)
Requirement already satisfied: packaging in /usr/local/lib/python3.5/dist-packages (20.9)
Requirement already satisfied: cerberus in /usr/local/lib/python3.5/dist-packages (1.3.3)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.5/dist-packages (from sympy>=1.1) (1.3.0)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/lib/python3/dist-packages (from packaging) (2.0.3)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.5/dist-packages (from pytest) (22.1.0)
Requirement already satisfied: importlib-metadata>=0.12 in /usr/local/lib/python3.5/dist-packages (from pytest) (2.1.3)
Requirement already satisfied: pluggy<1.0,>=0.12 in /usr/local/lib/python3.5/dist-packages (from pytest) (0.13.1)
Requirement already satisfied: iniconfig in /usr/local/lib/python3.5/dist-packages (from pytest) (1.1.1)
Requirement already satisfied: py>=1.8.2 in /usr/local/lib/python3.5/dist-packages (from pytest) (1.11.0)
Requirement already satisfied: toml in /usr/local/lib/python3.5/dist-packages (from pytest) (0.10.2)
Requirement already satisfied: pathlib2>=2.2.0 in /usr/local/lib/python3.5/dist-packages (from pytest) (2.3.7.post1)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.5/dist-packages (from importlib-metadata>=0.12->pytest) (1.2.0)
Requirement already satisfied: six in /root/.local/lib/python3.5/site-packages (from pathlib2>=2.2.0->pytest) (1.12.0)
2023-08-09 20:21:04,630 util.run [DEBUG] - Subprocess completed. Return code: 0
2023-08-09 20:21:04,630 build [INFO] - Generating CMake build tree
2023-08-09 20:21:04,631 util.run [INFO] - Running subprocess in '/home/jumper/workspace/onnxruntime_1.6.0/build/Linux/RelWithDebInfo'
['/usr/local/bin/cmake', '/home/jumper/workspace/onnxruntime_1.6.0/cmake', '-Donnxruntime_RUN_ONNX_TESTS=OFF', '-Donnxruntime_BUILD_WINML_TESTS=ON', '-Donnxruntime_GENERATE_TEST_REPORTS=ON', '-Donnxruntime_DEV_MODE=ON', '-DPYTHON_EXECUTABLE=/usr/bin/python3', '-Donnxruntime_USE_CUDA=OFF', '-Donnxruntime_CUDNN_HOME=', '-Donnxruntime_USE_FEATURIZERS=OFF', '-Donnxruntime_CUDA_HOME=', '-Donnxruntime_USE_JEMALLOC=OFF', '-Donnxruntime_USE_MIMALLOC_STL_ALLOCATOR=OFF', '-Donnxruntime_USE_MIMALLOC_ARENA_ALLOCATOR=OFF', '-Donnxruntime_ENABLE_PYTHON=OFF', '-Donnxruntime_BUILD_CSHARP=OFF', '-Donnxruntime_BUILD_JAVA=OFF', '-Donnxruntime_BUILD_NODEJS=OFF', '-Donnxruntime_BUILD_SHARED_LIB=ON', '-Donnxruntime_USE_EIGEN_FOR_BLAS=ON', '-Donnxruntime_USE_OPENBLAS=OFF', '-Donnxruntime_USE_DNNL=OFF', '-Donnxruntime_DNNL_GPU_RUNTIME=', '-Donnxruntime_DNNL_OPENCL_ROOT=', '-Donnxruntime_USE_MKLML=OFF', '-Donnxruntime_USE_NNAPI_BUILTIN=OFF', '-Donnxruntime_USE_RKNPU=OFF', '-Donnxruntime_USE_OPENMP=OFF', '-Donnxruntime_USE_TVM=OFF', '-Donnxruntime_USE_LLVM=OFF', '-Donnxruntime_ENABLE_MICROSOFT_INTERNAL=OFF', '-Donnxruntime_USE_VITISAI=OFF', '-Donnxruntime_USE_NUPHAR=OFF', '-Donnxruntime_USE_TENSORRT=OFF', '-Donnxruntime_TENSORRT_HOME=', '-Donnxruntime_USE_MIGRAPHX=OFF', '-Donnxruntime_MIGRAPHX_HOME=', '-Donnxruntime_CROSS_COMPILING=OFF', '-Donnxruntime_DISABLE_CONTRIB_OPS=OFF', '-Donnxruntime_DISABLE_ML_OPS=OFF', '-Donnxruntime_DISABLE_RTTI=OFF', '-Donnxruntime_DISABLE_EXCEPTIONS=OFF', '-Donnxruntime_DISABLE_ORT_FORMAT_LOAD=OFF', '-Donnxruntime_MINIMAL_BUILD=OFF', '-Donnxruntime_EXTENDED_MINIMAL_BUILD=OFF', '-Donnxruntime_REDUCED_OPS_BUILD=OFF', '-Donnxruntime_MSVC_STATIC_RUNTIME=OFF', '-Donnxruntime_ENABLE_LANGUAGE_INTEROP_OPS=OFF', '-Donnxruntime_USE_DML=OFF', '-Donnxruntime_USE_WINML=OFF', '-Donnxruntime_USE_TELEMETRY=OFF', '-Donnxruntime_ENABLE_LTO=OFF', '-Donnxruntime_USE_ACL=OFF', '-Donnxruntime_USE_ACL_1902=OFF', '-Donnxruntime_USE_ACL_1905=OFF', '-Donnxruntime_USE_ACL_1908=OFF', '-Donnxruntime_USE_ACL_2002=OFF', '-Donnxruntime_USE_ARMNN=OFF', '-Donnxruntime_ARMNN_RELU_USE_CPU=ON', '-Donnxruntime_ARMNN_BN_USE_CPU=ON', '-Donnxruntime_ENABLE_NVTX_PROFILE=OFF', '-Donnxruntime_ENABLE_TRAINING=OFF', '-Donnxruntime_USE_HOROVOD=OFF', '-Donnxruntime_USE_NCCL=ON', '-Donnxruntime_BUILD_BENCHMARKS=OFF', '-Donnxruntime_USE_ROCM=OFF', '-Donnxruntime_ROCM_HOME=', '-Donnxruntime_PYBIND_EXPORT_OPSCHEMA=OFF', '-Donnxruntime_ENABLE_MEMLEAK_CHECKER=OFF', '-DCMAKE_BUILD_TYPE=RelWithDebInfo']
Use gtest from submodule
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.5.2", minimum required is "3.4") 
Use protobuf from submodule
-- 
-- 3.11.3.0
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found PythonInterp: /usr/bin/python3 (found version "3.5.2") 
Generated: /home/jumper/workspace/onnxruntime_1.6.0/build/Linux/RelWithDebInfo/external/onnx/onnx/onnx-ml.proto
Generated: /home/jumper/workspace/onnxruntime_1.6.0/build/Linux/RelWithDebInfo/external/onnx/onnx/onnx-operators-ml.proto
Generated: /home/jumper/workspace/onnxruntime_1.6.0/build/Linux/RelWithDebInfo/external/onnx/onnx/onnx-data.proto
-- 
-- ******** Summary ********
--   CMake version                     : 3.18.4
--   CMake command                     : /usr/local/bin/cmake
--   System                            : Linux
--   C++ compiler                      : /usr/bin/c++
--   C++ compiler version              : 6.5.0
--   CXX flags                         :  -Wnon-virtual-dtor
--   Build type                        : RelWithDebInfo
--   Compile definitions               : ENABLE_ORT_FORMAT_LOAD;EIGEN_MPL2_ONLY;USE_EIGEN_FOR_BLAS;__STDC_FORMAT_MACROS
--   CMAKE_PREFIX_PATH                 : 
--   CMAKE_INSTALL_PREFIX              : /usr/local
--   CMAKE_MODULE_PATH                 : /home/jumper/workspace/onnxruntime_1.6.0/cmake/external
-- 
--   ONNX version                      : 1.15.0
--   ONNX NAMESPACE                    : onnx
--   ONNX_USE_LITE_PROTO               : ON
--   USE_PROTOBUF_SHARED_LIBS          : OFF
--   Protobuf_USE_STATIC_LIBS          : ON
--   ONNX_DISABLE_EXCEPTIONS           : OFF
--   ONNX_DISABLE_STATIC_REGISTRATION  : OFF
--   ONNX_WERROR                       : OFF
--   ONNX_BUILD_TESTS                  : OFF
--   ONNX_BUILD_BENCHMARKS             : OFF
-- 
--   Protobuf compiler                 : 
--   Protobuf includes                 : 
--   Protobuf libraries                : 
--   BUILD_ONNX_PYTHON                 : OFF
-- Proceeding with version: 23.5.26.0
-- CMAKE_CXX_FLAGS: 
-- Configuring done
CMake Error at CMakeLists.txt:673 (target_compile_definitions):
  Error evaluating generator expression:

    $<TARGET_PROPERTY:date_interface,INTERFACE_COMPILE_DEFINITIONS>

  Target "date_interface" not found.
Call Stack (most recent call first):
  onnxruntime_unittests.cmake:64 (onnxruntime_add_include_to_target)
  onnxruntime_unittests.cmake:888 (AddTest)
  CMakeLists.txt:1475 (include)


CMake Error at CMakeLists.txt:672 (target_include_directories):
  Error evaluating generator expression:

    $<TARGET_PROPERTY:date_interface,INTERFACE_INCLUDE_DIRECTORIES>

  Target "date_interface" not found.
Call Stack (most recent call first):
  onnxruntime_unittests.cmake:64 (onnxruntime_add_include_to_target)
  onnxruntime_unittests.cmake:888 (AddTest)
  CMakeLists.txt:1475 (include)


CMake Error at CMakeLists.txt:673 (target_compile_definitions):
  Error evaluating generator expression:

    $<TARGET_PROPERTY:date_interface,INTERFACE_COMPILE_DEFINITIONS>

  Target "date_interface" not found.
Call Stack (most recent call first):
  onnxruntime_unittests.cmake:64 (onnxruntime_add_include_to_target)
  onnxruntime_unittests.cmake:888 (AddTest)
  CMakeLists.txt:1475 (include)


CMake Error at CMakeLists.txt:672 (target_include_directories):
  Error evaluating generator expression:

    $<TARGET_PROPERTY:date_interface,INTERFACE_INCLUDE_DIRECTORIES>

  Target "date_interface" not found.
Call Stack (most recent call first):
  onnxruntime_unittests.cmake:64 (onnxruntime_add_include_to_target)
  onnxruntime_unittests.cmake:888 (AddTest)
  CMakeLists.txt:1475 (include)

解决办法：按compile errors · Issue #4379 · microsoft/onnxruntime · GitHub 这样解决。即将onnxruntime_common.cmake and onnxruntime_unittests.cmake 中的"date_interface" 改成 "date"即可。

GCC警告

上面将所有的第三方库都安装好了，但重新运行时报错如下：

[  6%] Built target onnxruntime_generate_def
Scanning dependencies of target gtest
[ 12%] Built target onnxruntime_mlas
[ 13%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/compile.cc.o
[ 13%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/dfa.cc.o
[ 13%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/bitmap256.cc.o
[ 13%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/filtered_re2.cc.o
[ 13%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/bitstate.cc.o
[ 22%] Built target libprotobuf
[ 22%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/mimics_pcre.cc.o
[ 22%] Building CXX object external/googletest/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
In file included from /usr/local/include/absl/base/config.h:86:0,
                 from /usr/local/include/absl/base/attributes.h:37,
                 from /usr/local/include/absl/base/macros.h:34,
                 from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/compile.cc:15:
/usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
 #error "This package requires GCC 7 or higher."
  ^~~~~
In file included from /usr/local/include/absl/base/config.h:86:0,
                 from /usr/local/include/absl/base/internal/invoke.h:40,
                 from /usr/local/include/absl/base/call_once.h:34,
                 from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/dfa.cc:36:
/usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
 #error "This package requires GCC 7 or higher."
  ^~~~~
In file included from /usr/local/include/absl/base/config.h:86:0,
                 from /usr/local/include/absl/base/attributes.h:37,
                 from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/util/logging.h:16,
                 from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/bitstate.cc:26:
/usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
 #error "This package requires GCC 7 or higher."
  ^~~~~
In file included from /usr/local/include/absl/base/config.h:86:0,
                 from /usr/local/include/absl/base/attributes.h:37,
                 from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/util/logging.h:16,
                 from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/mimics_pcre.cc:25:
/usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
 #error "This package requires GCC 7 or higher."
  ^~~~~
In file included from /usr/local/include/absl/base/config.h:86:0,
                 from /usr/local/include/absl/base/attributes.h:37,
                 from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/util/logging.h:16,
                 from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/bitmap256.h:14,
                 from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/bitmap256.cc:5:
/usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
 #error "This package requires GCC 7 or higher."
  ^~~~~
In file included from /usr/local/include/absl/base/config.h:86:0,
                 from /usr/local/include/absl/base/attributes.h:37,
                 from /usr/local/include/absl/strings/string_view.h:39,
                 from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/filtered_re2.h:28,
                 from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/filtered_re2.cc:5:
/usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
 #error "This package requires GCC 7 or higher."
  ^~~~~
external/re2/CMakeFiles/re2.dir/build.make:81: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/bitmap256.cc.o' failed
make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/bitmap256.cc.o] Error 1
make[2]: *** 正在等待未完成的任务....
Scanning dependencies of target libprotoc
external/re2/CMakeFiles/re2.dir/build.make:146: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/mimics_pcre.cc.o' failed
make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/mimics_pcre.cc.o] Error 1
[ 22%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/code_generator.cc.o
[ 22%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/command_line_interface.cc.o
external/re2/CMakeFiles/re2.dir/build.make:94: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/bitstate.cc.o' failed
make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/bitstate.cc.o] Error 1
[ 22%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_enum.cc.o
external/re2/CMakeFiles/re2.dir/build.make:133: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/filtered_re2.cc.o' failed
make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/filtered_re2.cc.o] Error 1
[ 22%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_enum_field.cc.o
external/re2/CMakeFiles/re2.dir/build.make:107: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/compile.cc.o' failed
make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/compile.cc.o] Error 1
[ 22%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_extension.cc.o
[ 22%] Built target pep8_check
[ 23%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_field.cc.o
external/re2/CMakeFiles/re2.dir/build.make:120: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/dfa.cc.o' failed
make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/dfa.cc.o] Error 1
CMakeFiles/Makefile2:2137: recipe for target 'external/re2/CMakeFiles/re2.dir/all' failed
make[1]: *** [external/re2/CMakeFiles/re2.dir/all] Error 2
make[1]: *** 正在等待未完成的任务....
[ 23%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_file.cc.o
[ 23%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_generator.cc.o
[ 23%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_helpers.cc.o
[ 23%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_map_field.cc.o

但实际我的gcc版本是9.1，是满足要求的，为什么会报这个错呢？我的电脑里也没有多个版本的gcc，只有一个9.1啊！？

使用预编译的onnxruntime-cpu

暂时只能使用预编译的onnxruntime了，我这次下载1.12.0 Releases · microsoft/onnxruntime · GitHub版本onnxruntime-linux-x64-1.12.0.tgz，然后将include和lib拷贝到系统对应位置，即：

cp -rfv onnxruntime-linux-x64-1.6.0/include/* /usr/local/include/ && \    cp -rfv onnxruntime-linux-x64-1.6.0/lib/* /usr/local/lib/ && \    ldconfig

mnist实例

按照https://github.com/microsoft/onnxruntime-inference-examples/tree/main/c_cxx/MNIST 这个

#include <onnxruntime_cxx_api.h>

#include <opencv2/opencv.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc.hpp>
using namespace cv;
using namespace std;

//这是分类推理
template <typename T>
static void softmax(T& input) {
  float rowmax = *std::max_element(input.begin(), input.end());
  std::vector<float> y(input.size());
  float sum = 0.0f;
  for (size_t i = 0; i != input.size(); ++i) {
    sum += y[i] = std::exp(input[i] - rowmax);
  }
  for (size_t i = 0; i != input.size(); ++i) {
    input[i] = y[i] / sum;
  }
}

int main()//
{
	Ort::Env env;
//	Ort::Session session(env, L"mnist.onnx", Ort::SessionOptions(nullptr));
	const char *modelpath="model/mnist.onnx";
//	Ort::Session session_{env, modelpath, Ort::SessionOptions{nullptr}};
	Ort::SessionOptions sessopt;
	sessopt.SetIntraOpNumThreads(1);
	sessopt.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_ENABLE_EXTENDED);
	Ort::Session sess_(env,modelpath,sessopt);

    const int modelsize=28;
    Ort::Value input_tensor_{nullptr};
    std::array<int64_t, 4> input_shape_{1, 1, modelsize, modelsize};

    const int labelsnum=10;
    Ort::Value output_tensor_{nullptr};
    std::array<int64_t, 2> output_shape_{1, labelsnum};

    std::array<float, modelsize * modelsize> input_image_{};
    std::array<float, labelsnum> results_{};
    int64_t result_{0};

    auto memory_info = Ort::MemoryInfo::CreateCpu(OrtDeviceAllocator, OrtMemTypeCPU);
	input_tensor_ = Ort::Value::CreateTensor<float>(memory_info, input_image_.data(), input_image_.size(),
													input_shape_.data(), input_shape_.size());
	output_tensor_ = Ort::Value::CreateTensor<float>(memory_info, results_.data(), results_.size(),
													 output_shape_.data(), output_shape_.size());

	// We need to convert the true-color data in the DIB into the model's floating point format
	float* inputdata = input_image_.data();
	//std::fill(input_image_.begin(), input_image_.end(), 0.f);

	const char* input_names[] = {"Input3"};
	const char* output_names[] = {"Plus214_Output_0"};

	char srcimgs[300];
	for(int imgid=0;imgid!=6;imgid++)
	{
		sprintf(srcimgs,"images/%d.jpg",imgid);
		Mat inputimg=imread(srcimgs,0);
		if(inputimg.empty())
			continue;

		cout<<"image "<<imgid<<":"<<endl;

		inputimg.convertTo(inputimg,CV_32FC1);
		for (unsigned y = 0; y < modelsize; y++)
		{
			for (unsigned x = 0; x < modelsize; x++)
			{
				inputdata[y*modelsize+x] = inputimg.at<float>(y,x);
			}
		}

		Ort::RunOptions run_options;
		sess_.Run(run_options, input_names, &input_tensor_, 1, output_names, &output_tensor_, 1);
		//get result ...
		softmax(results_);//results_和容器一样使用，得到每个labels的结果，以及最终的分类结果
		std::ptrdiff_t result_2 = std::distance(results_.begin(), std::max_element(results_.begin(), results_.end()));
		cout<<"      result_2:"<<result_2<<endl;
		//or get result ...
//		for (size_t i = 0; i != results_.size(); ++i) {
//		    float score = results_[i] ;
//		    printf("    label %d , score :%f \n",i,score);
//		}
	}

	return 0;
}

可以看到结果是正确的：

电池材料检测

使用电池分解材料模型和图片，如下：

#include <onnxruntime_cxx_api.h>

#include <opencv2/opencv.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc.hpp>
using namespace cv;
using namespace std;


int main()//
{
	Ort::Env env;
	const char *modelpath="model/segment.onnx";
	Ort::SessionOptions sessopt;
	sessopt.SetIntraOpNumThreads(1);
	sessopt.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_ENABLE_EXTENDED);
	Ort::Session sess_(env,modelpath,sessopt);

    const int modelsize=512;
    Ort::Value input_tensor_{nullptr};
    std::array<int64_t, 4> input_shape_{1, 1, modelsize, modelsize};
 
    Ort::Value output_tensor_{nullptr};
    std::array<int64_t, 4> output_shape_{1, 1, modelsize, modelsize};

    std::array<float, modelsize * modelsize> input_image_{};
    std::array<float, modelsize * modelsize> results_{};

    auto memory_info = Ort::MemoryInfo::CreateCpu(OrtDeviceAllocator, OrtMemTypeCPU);
	input_tensor_ = Ort::Value::CreateTensor<float>(memory_info, input_image_.data(), input_image_.size(),
													input_shape_.data(), input_shape_.size());
	output_tensor_ = Ort::Value::CreateTensor<float>(memory_info, results_.data(), results_.size(),
													 output_shape_.data(), output_shape_.size());

	// We need to convert the true-color data in the DIB into the model's floating point format
	float* inputdata = input_image_.data();

	const char* input_names[] = {"input.1"};
	const char* output_names[] = {"376"};

	char srcimgs[300];
	for(int imgid=1;imgid!=2;imgid++)
	{
		sprintf(srcimgs,"images2/contrast_%d.png",imgid);
		Mat inputimg3=imread(srcimgs);
		if(inputimg3.empty())
			continue;

		Mat inputimg;
		cvtColor(inputimg3,inputimg,COLOR_BGR2GRAY);
		cout<<"image "<<imgid<<":"<<endl;


		inputimg.convertTo(inputimg,CV_32FC1);
		inputimg/=255;
		cv::resize(inputimg,inputimg,Size(modelsize,modelsize));
		for (unsigned y = 0; y < modelsize; y++)
		{
			for (unsigned x = 0; x < modelsize; x++)
			{
				inputdata[y*modelsize+x] = inputimg.at<float>(y,x);
			}
		}

		Ort::RunOptions run_options;
		sess_.Run(run_options, input_names, &input_tensor_, 1, output_names, &output_tensor_, 1);
		
		//or get result ...
		float* outputdata = results_.data();
		Mat outimg(inputimg.size(),CV_32FC1);
		for (unsigned y = 0; y < modelsize; y++)
		{
			for (unsigned x = 0; x < modelsize; x++)
			{
				outimg.at<float>(y,x)=outputdata[y*modelsize+x];
			}
		}
		cv::resize(outimg,outimg,inputimg3.size());
		imwrite("result.jpg",outimg*255);

	}

	return 0;
}

推理结果正确：

如上，左边是原图，右边是推理结果，CPU版本的预编译库可正常推理。耗时尚未测试。

经过Performance | onnxruntime优化后，发现推理耗时少了很多，从而整体耗时少了很多，左边是优化前，右图是优化后，少了整整5秒，目前这还只是初步优化：

另外看到这已经是利用上了所有核都已经到了100%。下一步我不知道怎么让多张图片并行推理？如果我那样做是不是就是将核都利用过去了，而推理时就没有核可利用了？

###################################################################

采图装置是0.1mm比色皿，来自下图：

元气少女缘结神

关注

1
点赞
踩
4

收藏

觉得还不错? 一键收藏
打赏
1
评论
onnxruntime C++推理

最先想用的就是opencv的dnn模块对onnx模型推理，其实之前也用过dnn模块不过以前推理的是tensorflow模型，当时是成功的。加载模型是ok的，结果blobFromImage返回的blob行列是-1、-1？？？为什么blob是3维或4维？！
复制链接

扫一扫