MediaPipe框架- GPU支持

GPU Support

GPU支持

OpenGL ES Support

OpenGL ES支持

MediaPipe supports OpenGL ES up to version 3.2 on Android/Linux and up to ES 3.0 on iOS. In addition, MediaPipe also supports Metal on iOS.

MediaPipe在Android/Linux上支持OpenGL ES 3.2版本，在iOS上支持ES 3.0版本。此外，MediaPipe还在iOS上支持Metal。

OpenGL ES 3.1 or greater is required (on Android/Linux systems) for running machine learning inference calculators and graphs.

运行机器学习推理计算器和图需要OpenGL ES 3.1或更高版本（在Android/Linux系统上）。

Disable OpenGL ES Support

禁用OpenGL ES支持

By default, building MediaPipe (with no special bazel flags) attempts to compile and link against OpenGL ES (and for iOS also Metal) libraries.

默认情况下，构建MediaPipe（没有特殊的bazel标志）会尝试针对OpenGL ES（以及iOS和Metal）库进行编译和链接。

On platforms where OpenGL ES is not available (see also OpenGL ES Setup on Linux Desktop), you should disable OpenGL ES support with:

​在OpenGL ES不可用的平台上（另请参阅Linux Desktop上的OpenGL ES设置），应使用以下选项禁用OpenGL ES支持：

$ bazel build --define MEDIAPIPE_DISABLE_GPU=1 <my-target>

Note: On Android and iOS, OpenGL ES is required by MediaPipe framework and the support should never be disabled.

注意：在Android和iOS上，OpenGL ES是MediaPipe框架所必需的，并且不应禁用其支持。

OpenGL ES Setup on Linux Desktop

OpenGL ES在Linux桌面上的安装

On Linux desktop with video cards that support OpenGL ES 3.1+, MediaPipe can run GPU compute and rendering and perform TFLite inference on GPU.

在支持OpenGL ES 3.1+视频卡的Linux桌面上，MediaPipe可以运行GPU计算和渲染，并在GPU上执行TFLite推理。

To check if your Linux desktop GPU can run MediaPipe with OpenGL ES:

要检查Linux桌面GPU是否可以使用OpenGL ES运行MediaPipe，请执行以下操作：

$ sudo apt-get install mesa-common-dev libegl1-mesa-dev libgles2-mesa-dev
$ sudo apt-get install mesa-utils
$ glxinfo | grep -i opengl

For example, it may print:

例如，它可以打印：

$ glxinfo | grep -i opengl
...
OpenGL ES profile version string: OpenGL ES 3.2 NVIDIA 430.50
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions:

If you have connected to your computer through SSH and find when you probe for GPU information you see the output:

如果您已通过SSH连接到计算机，并在探测GPU信息时发现输出：

glxinfo | grep -i opengl
Error: unable to open display

Try re-establishing your SSH connection with the -X option and try again. For example:

请尝试使用-X选项重新建立SSH连接，然后重试。例如

ssh -X <user>@<host>

Notice the ES 3.20 text above.

请注意上面的ES 3.20文本。

You need to see ES 3.1 or greater printed in order to perform TFLite inference on GPU in MediaPipe. With this setup, build with:

为了在MediaPipe中的GPU上执行TFLite推理，您需要看到打印的ES 3.1或更高版本。使用此设置，使用以下内容构建：

$ bazel build --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 <my-target>

If only ES 3.0 or below is supported, you can still build MediaPipe targets that don't require TFLite inference on GPU with:

如果仅支持ES 3.0或更低版本，仍然可以使用以下功能在GPU上构建不需要TFLite推理的MediaPipe目标：

$ bazel build --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 --copt -DMEDIAPIPE_DISABLE_GL_COMPUTE <my-target>

Note: MEDIAPIPE_DISABLE_GL_COMPUTE is already defined automatically on all Apple systems (Apple doesn't support OpenGL ES 3.1+).

注意：MEDIAPIPE_DISABLE_GL_COMPUTE已经在所有苹果系统上自动定义（苹果不支持OpenGL ES 3.1+）。

TensorFlow CUDA Support and Setup on Linux Desktop

TensorFlow CUDA在Linux桌面上的支持和设置

MediaPipe framework doesn't require CUDA for GPU compute and rendering. However, MediaPipe can work with TensorFlow to perform GPU inference on video cards that support CUDA.

MediaPipe框架不需要CUDA进行GPU计算和渲染。但是，MediaPipe可以与TensorFlow一起在支持CUDA的视频卡上执行GPU推理。

To enable TensorFlow GPU inference with MediaPipe, the first step is to follow the TensorFlow GPU documentation to install the required NVIDIA software on your Linux desktop.

​要使用MediaPipe启用TensorFlow GPU推理，第一步是按照TensorFlow的GPU文档在Linux桌面上安装所需的NVIDIA软件。

After installation, update $PATH and $LD_LIBRARY_PATH and run ldconfig with:

安装后，更新$PATH和$LD_LIBRARY_PATH，并使用以下命令运行ldconfig：

$ export PATH=/usr/local/cuda-10.1/bin${PATH:+:${PATH}}
$ export LD_LIBRARY_PATH=/usr/local/cuda/extras/CUPTI/lib64,/usr/local/cuda-10.1/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
$ sudo ldconfig

It's recommended to verify the installation of CUPTI, CUDA, CuDNN, and NVCC:

建议验证CUPTI、CUDA、CuDNN和NVCC的安装情况：

$ ls /usr/local/cuda/extras/CUPTI
/lib64
libcupti.so       libcupti.so.10.1.208  libnvperf_host.so        libnvperf_target.so
libcupti.so.10.1  libcupti_static.a     libnvperf_host_static.a

$ ls /usr/local/cuda-10.1
LICENSE  bin  extras   lib64      libnvvp           nvml  samples  src      tools
README   doc  include  libnsight  nsightee_plugins  nvvm  share    targets  version.txt

$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243

$ ls /usr/lib/x86_64-linux-gnu/ | grep libcudnn.so
libcudnn.so
libcudnn.so.7
libcudnn.so.7.6.4

Setting $TF_CUDA_PATHS is the way to declare where the CUDA library is. Note that the following code snippet also adds /usr/lib/x86_64-linux-gnu and /usr/include into $TF_CUDA_PATHS for cudablas and libcudnn.

设置$TF_CUDA_PATHS是声明CUDA库所在位置的方法。请注意，以下代码片段还将/usr/lib/x86_64-linux-gnu和/usr/include添加到用于cudabas和libcudnn的$TF_CUDA_PATHS中。

$ export TF_CUDA_PATHS=/usr/local/cuda-10.1,/usr/lib/x86_64-linux-gnu,/usr/include

To make MediaPipe get TensorFlow's CUDA settings, find TensorFlow's .bazelrc and copy the build:using_cuda and build:cuda section into MediaPipe's .bazelrc file. For example, as of April 23, 2020, TensorFlow's CUDA setting is the following:

​要使MediaPipe获得TensorFlow的CUDA设置，请找到TensorFlow中的.bazelrc，并将build:using_cuda和build:cuda部分复制到MediaPipe的.bazellc文件中。例如，截至2020年4月23日，TensorFlow的CUDA设置如下：

# This config refers to building with CUDA available. It does not necessarily
# mean that we build CUDA op kernels.
build:using_cuda --define=using_cuda=true
build:using_cuda --action_env TF_NEED_CUDA=1
build:using_cuda --crosstool_top=@local_config_cuda//crosstool:toolchain

# This config refers to building CUDA op kernels with nvcc.
build:cuda --config=using_cuda
build:cuda --define=using_cuda_nvcc=true

Finally, build MediaPipe with TensorFlow GPU with two more flags --config=cuda and --spawn_strategy=local. For example:

最后，使用TensorFlow GPU再使用两个标志构建MediaPipe --config=cuda和--spawn_strategy=local。例如

$ bazel build -c opt --config=cuda --spawn_strategy=local \
    --define no_aws_support=true --copt -DMESA_EGL_NO_X11_HEADERS \
    mediapipe/examples/desktop/object_detection:object_detection_tensorflow

While the binary is running, it prints out the GPU device info:

当二进制文件运行时，它会打印出GPU设备信息：

I external/org_tensorflow/tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
I external/org_tensorflow/tensorflow/core/common_runtime/gpu/gpu_device.cc:1544] Found device 0 with properties: pciBusID: 0000:00:04.0 name: Tesla T4 computeCapability: 7.5 coreClock: 1.59GHz coreCount: 40 deviceMemorySize: 14.75GiB deviceMemoryBandwidth: 298.08GiB/s
I external/org_tensorflow/tensorflow/core/common_runtime/gpu/gpu_device.cc:1686] Adding visible gpu devices: 0

You can monitor the GPU usage to verify whether the GPU is used for model inference.

可以监视GPU的使用情况，以验证GPU是否用于模型推理。

$ nvidia-smi --query-gpu=utilization.gpu --format=csv --loop=1

0 %
0 %
4 %
5 %
83 %
21 %
22 %
27 %
29 %
100 %
0 %
0%