How do I use TensorRT on multiple GPUs?

Each ICudaEngine object is bound to a specific GPU when it is instantiated, either by the builder or on deserialization. To select the GPU, use cudaSetDevice() before calling the builder or deserializing the engine. Each IExecutionContext is bound to the same GPU as the engine from which it was created. When calling execute() or enqueue(), ensure that the thread is associated with the correct device by calling cudaSetDevice() if necessary.

译：每个ICudaEngine对象被实例化的时候（builder 或者deserialization）都会绑定在指定的GPU上。如果要选择GPU, 则应该在创建engine或者反序列化engine的时候使用cudaSetDevice（）进行设定。每一个IExecutionContext都被绑定在了engine被创建的那个GPU上。当使用execute()或者enqueue() 需要明确与当前显卡有关的线程