使用 onnxruntime-gpu(简称ORT)1.7 推断 onnx模型时出现如下警告,
2021-12-01 15:50:30.792327215 [W:onnxruntime:Default, fallback_cpu_capability.cc:135 GetCpuPreferredNodes] Force fallback to CPU execution for node: Gather_191
2021-12-01 15:50:30.792374122 [W:onnxruntime:Default, fallback_cpu_capability.cc:135 GetCpuPreferredNodes] Force fallback to CPU execution for node: Unsqueeze_363
2021-12-01 15:50:30.792386144 [W:onnxruntime:Default, fallback_cpu_capability.cc:135 GetCpuPreferredNodes] Force fallback to CPU execution for node: Gather_188
2021-12-01 15:50:30.792413728 [W:onnxruntime:Default, fallback_cpu_capability.cc:135 GetCpuPreferredNodes] Force fallback to CPU execution for node: Unsqueeze_362
2021-12-01 15:50:30.792442104 [W:onnxruntime:Default, fallback_cpu_capability.cc:135 GetCpuPreferredNodes] Force fallback to CPU execution for node: Gather_182
2021-12-01 15:50:30.792453046 [W:onnxruntime:Default, fallback_cpu_capability.cc:135 GetCpuPreferredNodes] Force fallback to CPU execution for node: Unsqueeze_361
ORT 选择 cpu 去做,是由于这个操作 相对于 cuda 在 CPU 上做效率更高。
升级 ORT 到 1.8,出现了莫名其妙的错误:
[E:onnxruntime:, sequential_executor.cc:339 Execute] Non-zero status code returned while running Conv node. Name:'Conv_2' Status Message: /onnxruntime_src/onnxruntime/core/framework/bfc_arena.cc:330 void* onnxruntime::BFCArena::AllocateRawInternal(size_t, bool) Failed to allocate memory for requested buffer of size 409856
升级到 1.9,就没问题了。