解决pybind11分装的函数无法被python多线程处理问题

Naglio_

已于 2023-07-26 23:02:16 修改

阅读量361

点赞数

文章标签： python 开发语言 c++

于 2023-07-26 23:01:44 首次发布

本文链接：https://blog.csdn.net/Naglio_/article/details/131949500

版权

解决思路：在C++中解开GIL锁

C++封装pybind

#include <glog/logging.h>
#include <thread>
#include <chrono>
#include <iostream>

double SSpeedTime(int i) 
{
    LOG(INFO) << "start...";
    std::chrono::seconds speedtime(2);

    std::this_thread::sleep_for(speedtime);
    LOG(INFO) << "end !";

    return i;
}
double call_SpeedTime(int i) 
{

    py::gil_scoped_release release;
    
    // double a = calculator::Calculator::SpeedTime(i);
    double a = SSpeedTime(i);

    py::gil_scoped_acquire acquire;

    return a;

}

PYBIND11_MODULE(calculator_py, m) {
        m.def("call_SpeedTime", &call_SpeedTime);
}

python调用

import calculator_py
import concurrent.futures
import time
import functools

def worker(i):
    speed = calculator_py.call_SpeedTime(i)
    return speed

# 创建多个线程并启动它们
threads = []
  # 创建线程池
start_T = time.time()

with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
    # 提交任务给线程池
    futures = []
    for i in range(0, 10):
        # 创建偏函数，固定部分参数
        partial_worker = functools.partial(worker, i)
        # 提交任务给线程池
        future = executor.submit(partial_worker)
        futures.append(future)

# 等待所有任务完成
concurrent.futures.wait(futures)

# 获取任务执行结果
end_T = time.time()
print("所有任务完成时间 ： " + str((end_T - start_T)*1000) + ", 开始配置结果...")


all = 0
for res in concurrent.futures.as_completed(futures) :
    all = all + res.result()


print(all)
# print("所有任务完成时间 ： " + str((end_T - start_T)*1000) + ", 开始配置结果...")

结果输出：

I0726 22:56:20.110416 43808 calculator_py.cpp:28] start...
I0726 22:56:20.111053 43809 calculator_py.cpp:28] start...
I0726 22:56:20.111547 43810 calculator_py.cpp:28] start...
I0726 22:56:20.111605 43811 calculator_py.cpp:28] start...
I0726 22:56:20.112231 43812 calculator_py.cpp:28] start...
I0726 22:56:20.112677 43813 calculator_py.cpp:28] start...
I0726 22:56:20.127169 43814 calculator_py.cpp:28] start...
I0726 22:56:20.128896 43816 calculator_py.cpp:28] start...
I0726 22:56:20.129626 43817 calculator_py.cpp:28] start...
I0726 22:56:20.130002 43818 calculator_py.cpp:28] start...
I0726 22:56:22.110913 43808 calculator_py.cpp:32] end !
I0726 22:56:22.111416 43809 calculator_py.cpp:32] end !
I0726 22:56:22.112051 43810 calculator_py.cpp:32] end !
I0726 22:56:22.112452 43812 calculator_py.cpp:32] end !
I0726 22:56:22.113184 43813 calculator_py.cpp:32] end !
I0726 22:56:22.119035 43811 calculator_py.cpp:32] end !
I0726 22:56:22.127477 43814 calculator_py.cpp:32] end !
I0726 22:56:22.129300 43816 calculator_py.cpp:32] end !
I0726 22:56:22.129884 43817 calculator_py.cpp:32] end !
I0726 22:56:22.130395 43818 calculator_py.cpp:32] end !
所有任务完成时间 ： 2022.5729942321777, 开始配置结果...
45.0

成功！