背景
import celery
import multiprocess as mp
def func(x):
return x**2
@celery.task
def add(self, x, y):
rst = []
with mp.Pool(processes=4) as pool:
for i in range(x,y):
res = pool.apply_async(func, i)
rst.append(res.get())
# 运行出错
# celery: daemonic processes are not allowed to have children
调整
根据stackflow上的了解是因为celery跟multiprocess的实现使用了不同系统接口库,要调整为celery自身使用的billiard库
# 将上述代码的multiprocess改成billiard
from billiard import pool as mp
参考:
billiard库介绍:https://pypi.org/project/billiard/