Parent process 15160.
Waiting for all subprocesses done...
Run task 0 (12588)...
Run task 1 (7948)...
Task 1 runs 0.37 seconds.
Run task 2 (7948)...
Task 0 runs 0.73 seconds.
Run task 3 (12588)...
Task 2 runs 0.48 seconds.
Run task 4 (7948)...
Task 3 runs 0.58 seconds.
Task 4 runs 0.54 seconds.
All subprocesses done.
进程池的方式可以控制运行中的进程数量,通过观察运行结果中进程号看得出进程池中只有两个进程。
进程间如何通信?使用Queue!现在有3个进程,2个负责写1个负责读:
from multiprocessing import Process, Queue
import os, time, random
# 写数据进程执行的代码:
def write(q):
print('Process to write: %s' % os.getpid())
for element in ['A', 'B', 'C','D','E','F','G','H']:
print('Write %s' %element)
q.put(element)
time.sleep(random.random())
# 读数据进程执行的代码:
def read(q):
while True:
#queue是阻塞队列,省去很多麻烦
element = q.get(True)
print('Read %s' %element)
if __name__=='__main__':
# 父进程创建Queue,并传给各个子进程:
q = Queue()
pw1 = Process(target=write, args=(q,))
pw2 = Process(target=write, args=(q,))
pr = Process(target=read, args=(q,))
# 启动子进程pw,写入:
pw1.start()
pw2.start()
# 启动子进程pr,读取:
pr.start()
# 等待pw结束:
pw1.join()
pw2.join()
# pr进程里是死循环,无法等待其结束,只能强行终止:
pr.terminate()
看完多进程我们再来看python多线程的开发,回到了Java人员熟悉的多线程算是如鱼得水,编码思路跟多进程基本一样,不同点是考虑到线程安全问题需要引入锁和ThreadLocal的概念。
Python锁的两个核心方法是:加锁lock.acquire()和释放锁lock.release(),为了保证锁一定要被释放我们往往将release()放在finally里
例子:
import threading
import time
lock = threading.Lock()
class myThread(threading.Thread):
def __init__(self, threadID, name, counter):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.counter = counter
def run(self):
# 获得锁,成功获得锁定后返回True
# 可选的timeout参数不填时将一直阻塞直到获得锁定
# 否则超时后将返回False
while self.counter>0 :
lock.acquire()
#打印机打印3次
print_time(self.name, 3)
# 释放锁
lock.release()
self.counter-=1
def print_time(threadName, counter):
while counter:
time.sleep(1)
print("%s: %s" % (threadName, time.ctime(time.time())))
counter -= 1
# 创建新线程
thread1 = myThread(1, "Thread-1", 3)
thread2 = myThread(2, "Thread-2", 3)
# 开启新线程
thread1.start()
thread2.start()
#等待线程执行完毕
thread1.join()
thread2.join()
执行结果:
Thread-1: Thu Dec 21 15:34:02 2017
Thread-1: Thu Dec 21 15:34:03 2017
Thread-1: Thu Dec 21 15:34:04 2017
Thread-1: Thu Dec 21 15:34:05 2017
Thread-1: Thu Dec 21 15:34:06 2017
Thread-1: Thu Dec 21 15:34:07 2017
Thread-2: Thu Dec 21 15:34:08 2017
Thread-2: Thu Dec 21 15:34:09 2017
Thread-2: Thu Dec 21 15:34:10 2017
Thread-1: Thu Dec 21 15:34:12 2017
Thread-1: Thu Dec 21 15:34:13 2017
Thread-1: Thu Dec 21 15:34:14 2017
Thread-2: Thu Dec 21 15:34:15 2017
Thread-2: Thu Dec 21 15:34:16 2017
Thread-2: Thu Dec 21 15:34:17 2017
Thread-2: Thu Dec 21 15:34:18 2017
Thread-2: Thu Dec 21 15:34:19 2017
Thread-2: Thu Dec 21 15:34:20 2017
锁虽然能解决线程间资源共享的问题,但是必然会带来竞争,而且处理不好容易产生死锁。
为了线程安全
Python
也有
Threadlocal
的解决思路,跟前面介绍的一期
Java
代码:
import threading,time,random
# 创建全局ThreadLocal对象:
local = threading.local()
class myThread(threading.Thread):
def __init__(self, threadID, name, counter):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.counter = counter
def run(self):
# 获得锁,成功获得锁定后返回True
# 可选的timeout参数不填时将一直阻塞直到获得锁定
# 否则超时后将返回False
while self.counter>0 :
local.name = self.name
local.count = 3
#打印机打印3次
print_time()
self.counter-=1
def print_time():
threadName = local.name
counter = local.count
while counter:
time.sleep(random.random())
print("%s: %s " % (threadName, time.ctime(time.time())))
counter -= 1
# 创建新线程
thread1 = myThread(1, "Thread-1", 3)
thread2 = myThread(2, "Thread-2", 3)
# 开启新线程
thread1.start()
thread2.start()
#等待线程执行完毕
thread1.join()
thread2.join()