python 多进程队列

1、多进程与多线程

python由于GIL全局解释锁，即使多cpu的机器下，一次也只能运行一个线程，所以CPU密集型的程序，要用多进程

2、队列

multiprocessing支持进程之间两种通信方式，队列Queue()和管道Pipe()

完美克隆Queue.Queue的功能

from multiprocessing import Process,Queue

def f(q):
    q.put(1)

if __name__ == "__main__":
    q=Queue()
    p = Process(target=f,args=(q,))
    p.start()
    print(q.get())
    p.join()

3、锁

lock=multiprocessing.Lock()

lock.acquire()获取锁

lock.release()释放锁

4、例子

import multiprocessing
import time
def worker(name,lock,q_res):
    lock.acquire()
    print("%s 1" % name)
    time.sleep(1)
    print("%s 2" % name)
    time.sleep(1)
    print("%s 3" % name)
    if not q_res.empty():
        print(q_res.get())
    lock.release()

if __name__=="__main__":
    pool = multiprocessing.Pool(4)
    m = multiprocessing.Manager()
    print(multiprocessing.cpu_count())
    p_lock = m.Lock()
    q_res = m.Queue()

    for i in range(2):
        q_res.put(i)
    for i in range(5):
        pool.apply_async(worker,args=("进程%d:" % i ,p_lock,q_res))
    pool.close()