import threading
try:
def dsrw():
#并行,定时五分钟执行一次
Parallel(n_jobs=jobs)(delayed(bx)(i) for i in file_path)
timer = threading.Timer(300, dsrw)
timer.start()
dsrw()
except Exception as e :
print('并行执行有异常,错误如下')
print(e)
上诉try中的代码
Parallel(n_jobs=jobs)(delayed(bx)(i) for i in file_path)用来做for循环内容的并行
n_jobs为并行数目,delayed()为并行的方法,后面是传入bx的循环参数
使用threading.timer调用自身,实现每x分钟定时调用这个并行代码
举例
import time
from joblib import Parallel, delayed
import datetime
import threading
file_path = [1,2,3,4,5]
jobs = len(file_path)
def bx(test1):
print(test1)
time.sleep(10)
print(test1,datetime.datetime.now())
def dsrw():
# 并行,定时五分钟执行一次
Parallel(n_jobs=jobs)(delayed(bx)(i) for i in file_path)
timer = threading.Timer(300, dsrw)
timer.start()
dsrw()
效果
1
2
3
4
5
1 2021-07-15 10:02:39.219970
2 2021-07-15 10:02:39.313668
3 2021-07-15 10:02:39.455159
4 2021-07-15 10:02:39.528961
5 2021-07-15 10:02:39.610770
可以看得出,并行执行了print()然后再会执行time