背景
日常开发中,难免遇到并发场景,而并发场景难免需要做流量控制,即需要对并发的进程或者线程的总量进行控制。 今天简单总结两种常用的控制线程个数的方法。
方法一:进程池/线程池
如下例demo所示, 创建了一个大小是4的进程池,然后创建5个进程,并启动
from multiprocessing import Pool
import os, time, random
def long_time_task(name):
print('Run task %s (%s)...' % (name, os.getpid()))
start = time.time()
time.sleep(random.random() * 3)
end = time.time()
print('Task %s runs %0.2f seconds.' % (name, (end - start)))
if __name__ == '__main__':
print('Parent process %s.' % os.getpid())
p = Pool(4)
for i in range(5):
p.apply_async(long_time_task, args=(i,))
print('Waiting for all subprocesses done...')
p.close()
p.join()
print('All subprocesses done.')
运行结果如下,可以看到第5个进程会等池子里的进程完成一个后才会被启动
Run task 0 (32952)...
Run task 1 (32951)...
Run task 2 (32953)...
Run task 3 (32954)...
Task 2 runs 0.68 seconds.
Run task 4 (32953)...
Task 1 runs 1.41 seconds.
Task 0 runs 1.44 seconds.
Task 4 runs 2.15 seconds.
Task 3 runs 2.98 seconds.
All subprocesses done.
方法二:queue
queue 模块即队列,特别适合处理信息在多个线程间安全交换的多线程程序中。 下面的demo展示了如何通过queue来限制线程的并发个数
import threading
import queue
import time
import random
import os
maxThreads = 4
class Store(threading.Thread):
def __init__(self, q):
threading.Thread.__init__(self)
self.queue = q
# self.store = store
def run(self):
try:
print('Run task (%s)...' % (os.getpid()))
start = time.time()
time.sleep(random.random() * 3)
end = time.time()
t = threading.currentThread()
# 线程ID
print('Thread id : %d' % t.ident)
print('Thread name : %s' % t.getName())
print('Task runs %0.2f seconds.' % (end - start))
except Exception as e:
print(e)
finally:
self.queue.get()
self.queue.task_done()
def main():
q = queue.Queue(maxThreads)
for s in range(6):
q.put(s)
t = Store(q)
t.start()
q.join()
print('over')
if __name__ == '__main__':
main()
运行结果如下:
Run task (33259)...
Run task (33259)...
Run task (33259)...
Run task (33259)...
Thread id : 123145444999168
Thread name : Thread-13
Task runs 0.04 seconds.
Run task (33259)...
Thread id : 123145394630656
Thread name : Thread-10
Task runs 1.02 seconds.
Run task (33259)...
Thread id : 123145428209664
Thread name : Thread-12
Task runs 1.20 seconds.
Thread id : 123145394630656
Thread name : Thread-17
Task runs 0.68 seconds.
Thread id : 123145444999168
Thread name : Thread-14
Task runs 1.79 seconds.
Thread id : 123145411420160
Thread name : Thread-11
Task runs 2.96 seconds.
over
以上就是python如何控制进程或者线程的个数的详细内容,更多关于python 控制进程或线程的资料请关注我们其它相关文章!
本文标题: python如何控制进程或者线程的个数
本文地址: http://www.cppcns.com/jiaoben/python/354832.html