Python的GIL(全局解释器锁)会组织Python代码同时在多个处理器核心上运行。于是有人误认为Python不支持多线程,或无法发挥多喝硬件的优势。对于“计算密集型”(CPU-bound)程序来说,可以用multiprocessing模块实现并发,该模块不受GIL限制,可以完全利用每个核心。这样一来,处理速度就很容易提高了(大致同CPU的核心数成正比)。对于“I/O密集型”程序来说,我们既可以用multiprocessing模块来做,也可以用threading模块或concurrent.futures模块来做。实际上,使用threading模块来编写I/O密集型程序时,并不用担心由GIL所带来的开销,因为网络延迟的影响更大。
# -*- coding: utf-8 -*-
#!/usr/bin/env python
# @Time : 2018/6/29 11:26
# @Software: PyCharm
import time
from concurrent.futures import ThreadPoolExecutor
import threading
import multiprocessing
'''
创建一个最大可容纳max_workers个的线程池
'''
pool1 = ThreadPoolExecutor(max_workers=300)
def return_future_result(message):
time.sleep(3)
return message
def go():
for num in range(1, 10):
# 往线程池里面加入一个task
pool1.submit(return_future_result, ("hello"+str(num)))
if __name__ == '__main__':
# 第一种
pool = multiprocessing.Pool(processes=10)
result = []
for i in xrange(60000):
msg = "hello %d" % (i)
print i
if int(i) % int(2000) == 0:
time.sleep(2)
result.append(pool.apply_async(return_future_result, (msg,)))
# 第二种
try:
threads = []
t1 = threading.Thread(target=return_future_result, args=(9101,))
threads.append(t1)
for port in (1,10):
threads.append(threading.Thread(target=return_future_result, args=(int(port[0]),)))
# 启动
for n in range(len(threads)):
threads[n].start()
except Exception, e:
print str(e)
# 第三种
for num in range(1, 1000):
print num
pool1.submit(return_future_result, num)
if int(num) % int(1000) == 0:
time.sleep(2)