我需要在python中使用多进程,可是我发现不论是multiprocessing.pool.Pool还是concurrent.futures.ProcessPoolExecutor都没有提供对子进程的终止?我现在急需要这样的功能,该如何实现?
我有看ProcessPoolExecutor的源代码,ProcessPoolExecutor大概就是在submit后会把参数视为一个work_item,添加到call_queue中去,再由_process_worker取出call_item并运行,感觉也没法用hack的手段来实现终止啊。。
下面是ProcessPoolExecutor中的部分核心源代码。。
def _process_worker(call_queue, result_queue):
"""Evaluates calls from call_queue and places the results in result_queue.
This worker is run in a separate process.
Args:
call_queue: A multiprocessing.Queue of _CallItems that will be read and
evaluated by the worker.
result_queue: A multiprocessing.Queue of _ResultItems that will written
to by the worker.
shutdown: A multiprocessing.Event that will be set as a signal to the
worker that it should exit when call_queue is empty.
"""
while True:
call_item = call_queue.get(block=True)
if call_item is None:
# Wake up queue management thread
result_queue.put(os.getpid())
return
try:
r = call_item.fn(*call_item.args, **call_item.kwargs)
except BaseException as e:
exc = _ExceptionWithTraceback(e, e.__traceback__)
result_queue.put(_ResultItem(call_item.work_id, exception=exc))
else:
result_queue.put(_ResultItem(call_item.work_id,
result=r))
在Python中使用多进程时,如何强制结束子进程是一个常见问题。`multiprocessing.pool.Pool`和`concurrent.futures.ProcessPoolExecutor`并未直接提供子进程终止功能。通过分析`ProcessPoolExecutor`源码,可以看到子任务被放入`call_queue`由`_process_worker`处理。要实现终止子进程,可能需要更复杂的方法,比如通过外部信号或者改变进程间的通信方式。目前,可以尝试使用`os.kill`或`terminate()`等方法来尝试结束子进程。
3万+

被折叠的 条评论
为什么被折叠?



