最近遇到结巴分词文本,耗时比较久,于是想到多进程来提升数据处理的效率。
基础代码如下:
import os
import multiprocessing
# worker function
def worker(sign, lock):
lock.acquire()
print(sign, os.getpid())
lock.release()
# Multi-process
record = []
for i in range(5):
process = multiprocessing.Process(target=worker,args=('process',lock))
process.start()
record.append(process)
for process in record:
process.join()
other process
PS:
1、通过终端执行py文件,才能运行上述代码;
2、 other process 是在多线程执行完了之后,才能运行。