TensorRT多GPU的使用

来自于开发者手册

Q: How do I use TensorRT on multiple GPUs?
如何在多GPU环境中使用TensorRT

A: Each ICudaEngine object is bound to a specific GPU when it is instantiated, either
by the builder or on deserialization. To select the GPU, use cudaSetDevice() before
calling the builder or deserializing the engine. Each IExecutionContext is bound
to the same GPU as the engine from which it was created. When calling execute()
or enqueue(), ensure that the thread is associated with the correct device by calling
cudaSetDevice() if necessary

每个ICudaEngine对象被实例化的时候（builder 或者deserialization）都会绑定在指定的GPU上。如果要选择GPU, 则应该在创建engine或者反序列化engine的时候使用cudaSetDevice（）进行设定。每一个IExecutionContext都被绑定在了engine被创建的那个GPU上。当使用execute()或者enqueue() 需要明确与当前显卡有关的线程