环境
ubuntu18.04
cuda11.2
k80 x 8
Serving with Docker using your GPU
Install nvidia-docker
Before serving with a GPU, in addition to installing Docker, you will need:
Up-to-date NVIDIA drivers for your system
nvidia-docker: You can follow the installation instructions here
Running a GPU serving image
Running a GPU serving image is identical to running a CPU image. For more details, see running a serving image.
GPU Serving example
Let’s run through a full example where we load a model with GPU-bound ops and call it using the REST API.
First install nvidia-docker. Next you can pull the latest TensorFlow Serving GPU docker image by running:
docker pull tensorflow/s