1.多线程:python通过两个标准库thread和threading提供对线程的支持。thread提供了低级别的、原始的线程以及一个简单的锁。threading通过对thread模块进行二次封装,提供了更方便的API来操作线程。接下来只介绍threading的常见用法 import threading import time def Traversal_5(interval): for i in xrange(5): print 'Traversal_5:', i time.sleep(interval) def Traversal_10(interval): for i in xrange(10): print 'Traversal_10:', i time.sleep(interval) if __name__ == '__main__': print 'start time:' t1 = int(time.time()) tasks = [Traversal_5, Traversal_10] threads = [] for task in tasks: t = threading.Thread(target=task, args=(1,)) threads.append(t) for t in threads: t.setDaemon(True) t.start() for t in threads: t.join() print 'end main total time:', int(time.time()) - t1
2.多进程===由于GIL的存在,python中的多线程其实并不是真正的多线程,如果想要充分地使用多核CPU的资源,在python中大部分情况需要使用多进程。Python提供了非常好用的多进程包multiprocessing,与threading.Thread类似,它可以利用multiprocessing.Process对象来创建一个进程。接下来只介绍multiprocessing的常见用法。
import multiprocessing import time class Traversal(object): def __init__(self, interval, name): self.interval = interval self.name = name self._rungevent(self.interval, self.name) def _rungevent(self, interval, name): for i in xrange(5): print 'process name:', name, '\tindex:', i time.sleep(interval) if __name__ == '__main__': print 'start time:' t1 = int(time.time()) jobs = [] for x in xrange(2): p = multiprocessing.Process(target=Traversal, args=(1, 'Traversal_' + str(x))) p.start() jobs.append(p) for job in jobs: job.join() print 'end main total time:', int(time.time()) - t1
3.协程====协程,又称微线程,纤程。协程的特点在于是一个线程执行,那和多线程比,协程有何优势?最大的优势就是协程极高的执行效率。因为子程序切换不是线程切换,而是由程序自身控制,因此,没有线程切换的开销,和多线程比,线程数量越多,协程的性能优势就越明显。第三方的gevent为Python提供了比较完善的协程支持。接下来只介绍gevent用法
from gevent import monkey; monkey.patch_all(); from gevent.pool import Pool import time def Traversal(job): print 'job:', job time.sleep(1) if __name__ == '__main__': print 'start time:' t1 = int(time.time()) jobs = [i for i in xrange(10)] pool = Pool(5) pool.map(Traversal, jobs) print 'end main total time:', int(time.time()) - t1
4.多进程+协程===因为协程是一个线程执行,那怎么利用多核CPU呢?最简单的方法是多进程+协程,既充分利用多核,又充分发挥协程的高效率,可获得极高的性能。
import multiprocessing from gevent import monkey; monkey.patch_all(); from gevent.pool import Pool import time def Traver(job): print 'job:', job time.sleep(1) class Traversal(object): def __init__(self, interval, name): self.interval = interval self.name = name self._rungevent(self.interval, self.name) def _rungevent(self, interval, name): jobs = [i for i in xrange(5)] pool = Pool(5) pool.map(Traver, jobs) if __name__ == '__main__': print 'start time:' t1 = int(time.time()) jobs = [] for x in xrange(2): p = multiprocessing.Process(target=Traversal, args=(1, 'Traversal_' + str(x))) p.start() jobs.append(p) for job in jobs: job.join() print 'end main total time:', int(time.time()) - t1
5.