TensorRT多GPU的使用

最新推荐文章于 2024-07-05 11:23:00 发布

冬日and暖阳

最新推荐文章于 2024-07-05 11:23:00 发布

阅读量3.6k

点赞数 1

分类专栏： TensorRT

本文链接：https://blog.csdn.net/qq_29007291/article/details/110551881

版权

TensorRT 专栏收录该内容

20 篇文章 3 订阅

订阅专栏

来自于开发者手册

Q: How do I use TensorRT on multiple GPUs?
如何在多GPU环境中使用TensorRT

A: Each ICudaEngine object is bound to a specific GPU when it is instantiated, either
by the builder or on deserialization. To select the GPU, use cudaSetDevice() before
calling the builder or deserializing the engine. Each IExecutionContext is bound
to the same GPU as the engine from which it was created. When calling execute()
or enqueue(), ensure that the thread is associated with the correct device by calling
cudaSetDevice() if necessary

每个ICudaEngine对象被实例化的时候（builder 或者deserialization）都会绑定在指定的GPU上。如果要选择GPU, 则应该在创建engine或者反序列化engine的时候使用cudaSetDevice（）进行设定。每一个IExecutionContext都被绑定在了engine被创建的那个GPU上。当使用execute()或者enqueue() 需要明确与当前显卡有关的线程