python3多进程包_Python-多线程+多进程包（concurrent包，处理并发）

最新推荐文章于 2024-06-29 17:27:03 发布

weixin_39938875

最新推荐文章于 2024-06-29 17:27:03 发布

阅读量291

点赞数

文章标签： python3多进程包

1、concurrent包

此包3.2版本之后引入，只提供了一个模块futures

异步并行任务编程模块，提供了一个高级的异步可执行的便利接口。

提供了两个池执行器

ThreadPoolExecutor 异步调用的线程池的 Executor

ProcessPoolExeutor 异步调用的进程池的 Executor

2、ThreadPoolExecutor 对象 -- 线程

首先需要定义一个池的执行器对象，Executor类子类对象。

Future类

测试：IO 密集型测试，创建一个线城池，开启三个线程，因为此模块没有提供join方法，所以通过编程来实现主线程等待。

1 importthreading2 from concurrent importfutures3

4 importlogging5 importtime6

7 FORMAT = '%(asctime)-15s\t %(process)s %(threadName)s %(process)s %(message)s'

8 logging.basicConfig(level=logging.INFO, format=FORMAT)9

10 #这是一个IO 密集型的函数，建议使用多线程

11 defworker(n):12 logging.info('begin to work{}'.format(n))13 time.sleep(5)14 logging.info('finisdhed{}'.format(n))15 returnn16

17 #创建线程池，容量为3个

18 executor = futures.ThreadPoolExecutor(max_workers=3)19

20 fs =[]21 for i in range(3):22 #提交任务

23 future =executor.submit(worker, i)24 fs.append(future)25

27 whileTrue:28 time.sleep(1)29 logging.info(threading.enumerate())30

31 flag =True32 for f infs:33 logging.info(f.done())34 flag = flag andf.done()35

36 ifflag:37 for f infs:38 logging.info('the ans is {}'.format(f.result()))39 executor.shutdown()40 logging.info(threading.enumerate())41 break

43 logging.info('--------------------')

线程数=池容量

1 2018-10-30 10:29:03,486 9176 ThreadPoolExecutor-0_0 9176begin to work02 2018-10-30 10:29:03,486 9176 ThreadPoolExecutor-0_1 9176begin to work13 2018-10-30 10:29:03,486 9176 ThreadPoolExecutor-0_2 9176begin to work24 2018-10-30 10:29:04,486 9176 MainThread 9176 [<_mainthread started>, , , ]5 2018-10-30 10:29:04,486 9176 MainThread 9176False6 2018-10-30 10:29:04,486 9176 MainThread 9176False7 2018-10-30 10:29:04,486 9176 MainThread 9176False8 2018-10-30 10:29:05,486 9176 MainThread 9176 [<_mainthread started>, , , ]9 2018-10-30 10:29:05,486 9176 MainThread 9176False10 2018-10-30 10:29:05,486 9176 MainThread 9176False11 2018-10-30 10:29:05,487 9176 MainThread 9176False12 2018-10-30 10:29:06,488 9176 MainThread 9176 [<_mainthread started>, , , ]13 2018-10-30 10:29:06,488 9176 MainThread 9176False14 2018-10-30 10:29:06,488 9176 MainThread 9176False15 2018-10-30 10:29:06,488 9176 MainThread 9176False16 2018-10-30 10:29:07,488 9176 MainThread 9176 [<_mainthread started>, , , ]17 2018-10-30 10:29:07,488 9176 MainThread 9176False18 2018-10-30 10:29:07,488 9176 MainThread 9176False19 2018-10-30 10:29:07,489 9176 MainThread 9176False20 2018-10-30 10:29:08,487 9176 ThreadPoolExecutor-0_1 9176finisdhed121 2018-10-30 10:29:08,487 9176 ThreadPoolExecutor-0_0 9176finisdhed022 2018-10-30 10:29:08,487 9176 ThreadPoolExecutor-0_2 9176finisdhed223 2018-10-30 10:29:08,489 9176 MainThread 9176 [<_mainthread started>, , , ]24 2018-10-30 10:29:08,489 9176 MainThread 9176True25 2018-10-30 10:29:08,489 9176 MainThread 9176True26 2018-10-30 10:29:08,489 9176 MainThread 9176True27 2018-10-30 10:29:08,489 9176 MainThread 9176 the ans is028 2018-10-30 10:29:08,489 9176 MainThread 9176 the ans is 1

29 2018-10-30 10:29:08,489 9176 MainThread 9176 the ans is 2

30 2018-10-30 10:29:08,490 9176 MainThread 9176 [<_mainthread started>]31 2018-10-30 10:29:08,490 9176 MainThread 9176 --------------------

得出：线程池关闭之后，就没有线程空闲线程存在了

测试：IO 密集型测试，创建一个线城池，开启一个线程，因为此模块没有提供join方法，所以通过编程来实现主线程等待。

1 importthreading2 from concurrent importfutures3

4 importlogging5 importtime6

7 FORMAT = '%(asctime)-15s\t %(process)s %(threadName)s %(process)s %(message)s'

8 logging.basicConfig(level=logging.INFO, format=FORMAT)9

10 #这是一个IO 密集型的函数，建议使用多线程

11 defworker(n):12 logging.info('begin to work{}'.format(n))13 time.sleep(5)14 logging.info('finisdhed{}'.format(n))15 returnn16

17 #创建线程池，容量为3个

18 executor = futures.ThreadPoolExecutor(max_workers=3)19

20 fs =[]21 for i in range(1):22 #提交任务

23 future =executor.submit(worker, i)24 fs.append(future)25

27 whileTrue:28 time.sleep(1)29 logging.info(threading.enumerate())30

31 flag =True32 for f infs:33 logging.info(f.done())34 flag = flag andf.done()35

36 ifflag:37 for f infs:38 logging.info('the ans is {}'.format(f.result()))39 #logging.info(threading.enumerate())

40 #logging.info('====================')

41 executor.shutdown()42 logging.info(threading.enumerate())43 break

45 logging.info('--------------------')

线程数 < 池容量

1 D:\python3.7\python.exe E:/code_pycharm/tt2.py2 2018-10-30 10:31:01,309 8408 ThreadPoolExecutor-0_0 8408begin to work03 2018-10-30 10:31:02,309 8408 MainThread 8408 [<_mainthread started>, ]4 2018-10-30 10:31:02,309 8408 MainThread 8408False5 2018-10-30 10:31:03,309 8408 MainThread 8408 [<_mainthread started>, ]6 2018-10-30 10:31:03,309 8408 MainThread 8408False7 2018-10-30 10:31:04,309 8408 MainThread 8408 [<_mainthread started>, ]8 2018-10-30 10:31:04,309 8408 MainThread 8408False9 2018-10-30 10:31:05,309 8408 MainThread 8408 [<_mainthread started>, ]10 2018-10-30 10:31:05,309 8408 MainThread 8408False11 2018-10-30 10:31:06,309 8408 ThreadPoolExecutor-0_0 8408finisdhed012 2018-10-30 10:31:06,309 8408 MainThread 8408 [<_mainthread started>, ]13 2018-10-30 10:31:06,309 8408 MainThread 8408True14 2018-10-30 10:31:06,310 8408 MainThread 8408 the ans is015 2018-10-30 10:31:06,310 8408 MainThread 8408 [<_mainthread started>]16 2018-10-30 10:31:06,310 8408 MainThread 8408 --------------------

18 Process finished with exit code 0

得出：线程池处于懒惰模式，提交一个线程函数，就开启一个线程

测试：IO 密集型测试，创建一个线城池，开启六个线程，因为此模块没有提供join方法，所以通过编程来实现主线程等待。

1 importthreading2 from concurrent importfutures3

4 importlogging5 importtime6

7 FORMAT = '%(asctime)-15s\t %(process)s %(threadName)s %(process)s %(message)s'

8 logging.basicConfig(level=logging.INFO, format=FORMAT)9

10 #这是一个IO 密集型的函数，建议使用多线程

11 defworker(n):12 logging.info('begin to work{}'.format(n))13 time.sleep(5)14 logging.info('finisdhed{}'.format(n))15 returnn16

17 #创建线程池，容量为3个

18 executor = futures.ThreadPoolExecutor(max_workers=3)19

20 fs =[]21 for i in range(6):22 #提交任务

23 future =executor.submit(worker, i)24 fs.append(future)25

27 whileTrue:28 time.sleep(1)29 logging.info(threading.enumerate())30

31 flag =True32 for f infs:33 logging.info(f.done())34 flag = flag andf.done()35

36 ifflag:37 for f infs:38 logging.info('the ans is {}'.format(f.result()))39 #logging.info(threading.enumerate())

40 #logging.info('====================')

41 executor.shutdown()42 logging.info(threading.enumerate())43 break

45 logging.info('--------------------')

线程数 >池容量

1 D:\python3.7\python.exe E:/code_pycharm/tt2.py2 2018-10-30 10:36:29,200 7524 ThreadPoolExecutor-0_0 7524begin to work03 2018-10-30 10:36:29,201 7524 ThreadPoolExecutor-0_1 7524begin to work14 2018-10-30 10:36:29,201 7524 ThreadPoolExecutor-0_2 7524begin to work25 2018-10-30 10:36:30,201 7524 MainThread 7524 [<_mainthread started>, , , ]6 2018-10-30 10:36:30,201 7524 MainThread 7524False7 2018-10-30 10:36:30,201 7524 MainThread 7524False8 2018-10-30 10:36:30,202 7524 MainThread 7524False9 2018-10-30 10:36:30,202 7524 MainThread 7524False10 2018-10-30 10:36:30,202 7524 MainThread 7524False11 2018-10-30 10:36:30,202 7524 MainThread 7524False12 2018-10-30 10:36:31,204 7524 MainThread 7524 [<_mainthread started>, , , ]13 2018-10-30 10:36:31,204 7524 MainThread 7524False14 2018-10-30 10:36:31,204 7524 MainThread 7524False15 2018-10-30 10:36:31,204 7524 MainThread 7524False16 2018-10-30 10:36:31,204 7524 MainThread 7524False17 2018-10-30 10:36:31,204 7524 MainThread 7524False18 2018-10-30 10:36:31,204 7524 MainThread 7524False19 2018-10-30 10:36:32,204 7524 MainThread 7524 [<_mainthread started>, , , ]20 2018-10-30 10:36:32,204 7524 MainThread 7524False21 2018-10-30 10:36:32,204 7524 MainThread 7524False22 2018-10-30 10:36:32,204 7524 MainThread 7524False23 2018-10-30 10:36:32,204 7524 MainThread 7524False24 2018-10-30 10:36:32,204 7524 MainThread 7524False25 2018-10-30 10:36:32,204 7524 MainThread 7524False26 2018-10-30 10:36:33,204 7524 MainThread 7524 [<_mainthread started>, , , ]27 2018-10-30 10:36:33,204 7524 MainThread 7524False28 2018-10-30 10:36:33,204 7524 MainThread 7524False29 2018-10-30 10:36:33,204 7524 MainThread 7524False30 2018-10-30 10:36:33,205 7524 MainThread 7524False31 2018-10-30 10:36:33,205 7524 MainThread 7524False32 2018-10-30 10:36:33,206 7524 MainThread 7524False33 2018-10-30 10:36:34,200 7524 ThreadPoolExecutor-0_0 7524finisdhed034 2018-10-30 10:36:34,200 7524 ThreadPoolExecutor-0_0 7524begin to work335 2018-10-30 10:36:34,201 7524 ThreadPoolExecutor-0_1 7524finisdhed136 2018-10-30 10:36:34,201 7524 ThreadPoolExecutor-0_1 7524begin to work437 2018-10-30 10:36:34,201 7524 ThreadPoolExecutor-0_2 7524finisdhed238 2018-10-30 10:36:34,201 7524 ThreadPoolExecutor-0_2 7524begin to work539 2018-10-30 10:36:34,206 7524 MainThread 7524 [<_mainthread started>, , , ]40 2018-10-30 10:36:34,206 7524 MainThread 7524True41 2018-10-30 10:36:34,206 7524 MainThread 7524True42 2018-10-30 10:36:34,206 7524 MainThread 7524True43 2018-10-30 10:36:34,206 7524 MainThread 7524False44 2018-10-30 10:36:34,206 7524 MainThread 7524False45 2018-10-30 10:36:34,206 7524 MainThread 7524False46 2018-10-30 10:36:35,206 7524 MainThread 7524 [<_mainthread started>, , , ]47 2018-10-30 10:36:35,206 7524 MainThread 7524True48 2018-10-30 10:36:35,206 7524 MainThread 7524True49 2018-10-30 10:36:35,206 7524 MainThread 7524True50 2018-10-30 10:36:35,206 7524 MainThread 7524False51 2018-10-30 10:36:35,206 7524 MainThread 7524False52 2018-10-30 10:36:35,206 7524 MainThread 7524False53 2018-10-30 10:36:36,206 7524 MainThread 7524 [<_mainthread started>, , , ]54 2018-10-30 10:36:36,206 7524 MainThread 7524True55 2018-10-30 10:36:36,206 7524 MainThread 7524True56 2018-10-30 10:36:36,206 7524 MainThread 7524True57 2018-10-30 10:36:36,206 7524 MainThread 7524False58 2018-10-30 10:36:36,206 7524 MainThread 7524False59 2018-10-30 10:36:36,206 7524 MainThread 7524False60 2018-10-30 10:36:37,206 7524 MainThread 7524 [<_mainthread started>, , , ]61 2018-10-30 10:36:37,206 7524 MainThread 7524True62 2018-10-30 10:36:37,206 7524 MainThread 7524True63 2018-10-30 10:36:37,206 7524 MainThread 7524True64 2018-10-30 10:36:37,207 7524 MainThread 7524False65 2018-10-30 10:36:37,207 7524 MainThread 7524False66 2018-10-30 10:36:37,207 7524 MainThread 7524False67 2018-10-30 10:36:38,207 7524 MainThread 7524 [<_mainthread started>, , , ]68 2018-10-30 10:36:38,207 7524 MainThread 7524True69 2018-10-30 10:36:38,209 7524 MainThread 7524True70 2018-10-30 10:36:38,210 7524 MainThread 7524True71 2018-10-30 10:36:38,210 7524 MainThread 7524False72 2018-10-30 10:36:38,210 7524 MainThread 7524False73 2018-10-30 10:36:38,210 7524 MainThread 7524False74 2018-10-30 10:36:39,200 7524 ThreadPoolExecutor-0_0 7524finisdhed375 2018-10-30 10:36:39,201 7524 ThreadPoolExecutor-0_1 7524finisdhed476 2018-10-30 10:36:39,201 7524 ThreadPoolExecutor-0_2 7524finisdhed577 2018-10-30 10:36:39,210 7524 MainThread 7524 [<_mainthread started>, , , ]78 2018-10-30 10:36:39,210 7524 MainThread 7524True79 2018-10-30 10:36:39,210 7524 MainThread 7524True80 2018-10-30 10:36:39,211 7524 MainThread 7524True81 2018-10-30 10:36:39,211 7524 MainThread 7524True82 2018-10-30 10:36:39,211 7524 MainThread 7524True83 2018-10-30 10:36:39,211 7524 MainThread 7524True84 2018-10-30 10:36:39,211 7524 MainThread 7524 the ans is085 2018-10-30 10:36:39,212 7524 MainThread 7524 the ans is 1

86 2018-10-30 10:36:39,212 7524 MainThread 7524 the ans is 2

87 2018-10-30 10:36:39,212 7524 MainThread 7524 the ans is 3

88 2018-10-30 10:36:39,212 7524 MainThread 7524 the ans is 4

89 2018-10-30 10:36:39,212 7524 MainThread 7524 the ans is 5

90 2018-10-30 10:36:39,213 7524 MainThread 7524 [<_mainthread started>]91 2018-10-30 10:36:39,213 7524 MainThread 7524 --------------------

93 Process finished with exit code 0

得出：三个线程，只要有一个结束，就开启第四个线程

3、ProcessPoolExecutor 对象 -- 进程

方法和线程一样，就是线程变为进程

1 importthreading2 from concurrent importfutures3

4 importlogging5 importtime6

7 FORMAT = '%(asctime)-15s\t %(process)s %(processName)s %(message)s'

8 logging.basicConfig(level=logging.INFO, format=FORMAT)9

10 #这是一个IO 密集型的函数，建议使用多线程

11 defworker(n):12 logging.info('begin to work{}'.format(n))13 time.sleep(5)14 logging.info('finisdhed{}'.format(n))15 returnn16

17 #创建线程池，容量为3个

18 #executor = futures.ThreadPoolExecutor(max_workers=3)

20 if __name__ == '__main__':21 executor = futures.ProcessPoolExecutor(max_workers=3)22 fs =[]23 for i in range(3):24 #提交任务

25 future =executor.submit(worker, i)26 fs.append(future)27

29 whileTrue:30 time.sleep(1)31 logging.info(threading.enumerate())32

33 flag =True34 for f infs:35 logging.info(f.done())36 flag = flag andf.done()37

38 ifflag:39 for f infs:40 logging.info('the ans is {}'.format(f.result()))41 #logging.info(threading.enumerate())

42 #logging.info('====================')

43 executor.shutdown()44 logging.info(threading.enumerate())45 break

47 logging.info('--------------------')

进程测试

1 D:\python3.7\python.exe E:/code_pycharm/tt2.py2 2018-10-30 10:47:48,933 3224 SpawnProcess-1begin to work03 2018-10-30 10:47:48,951 7484 SpawnProcess-2begin to work14 2018-10-30 10:47:48,974 8848 SpawnProcess-3begin to work25 2018-10-30 10:47:49,829 6220 MainProcess [<_mainthread started>, , ]6 2018-10-30 10:47:49,829 6220MainProcess False7 2018-10-30 10:47:49,829 6220MainProcess False8 2018-10-30 10:47:49,829 6220MainProcess False9 2018-10-30 10:47:50,829 6220 MainProcess [<_mainthread started>, , ]10 2018-10-30 10:47:50,829 6220MainProcess False11 2018-10-30 10:47:50,829 6220MainProcess False12 2018-10-30 10:47:50,829 6220MainProcess False13 2018-10-30 10:47:51,829 6220 MainProcess [<_mainthread started>, , ]14 2018-10-30 10:47:51,829 6220MainProcess False15 2018-10-30 10:47:51,829 6220MainProcess False16 2018-10-30 10:47:51,829 6220MainProcess False17 2018-10-30 10:47:52,829 6220 MainProcess [<_mainthread started>, , ]18 2018-10-30 10:47:52,829 6220MainProcess False19 2018-10-30 10:47:52,829 6220MainProcess False20 2018-10-30 10:47:52,829 6220MainProcess False21 2018-10-30 10:47:53,829 6220 MainProcess [<_mainthread started>, , ]22 2018-10-30 10:47:53,829 6220MainProcess False23 2018-10-30 10:47:53,829 6220MainProcess False24 2018-10-30 10:47:53,829 6220MainProcess False25 2018-10-30 10:47:53,934 3224 SpawnProcess-1finisdhed026 2018-10-30 10:47:53,951 7484 SpawnProcess-2finisdhed127 2018-10-30 10:47:53,974 8848 SpawnProcess-3finisdhed228 2018-10-30 10:47:54,829 6220 MainProcess [<_mainthread started>, , ]29 2018-10-30 10:47:54,829 6220MainProcess True30 2018-10-30 10:47:54,829 6220MainProcess True31 2018-10-30 10:47:54,829 6220MainProcess True32 2018-10-30 10:47:54,829 6220 MainProcess the ans is033 2018-10-30 10:47:54,829 6220 MainProcess the ans is 1

34 2018-10-30 10:47:54,829 6220 MainProcess the ans is 2

35 2018-10-30 10:47:54,854 6220 MainProcess [<_mainthread started>]36 2018-10-30 10:47:54,854 6220 MainProcess --------------------

38 Process finished with exit code 0

查看 python进程数，占用三个CPU

4、支持上下文管理

1 importthreading2 from concurrent importfutures3

4 importlogging5 importtime6

7 FORMAT = '%(asctime)-15s\t %(process)s %(processName)s %(message)s'

8 logging.basicConfig(level=logging.INFO, format=FORMAT)9

10 #这是一个IO 密集型的函数，建议使用多线程

11 defworker(n):12 logging.info('begin to work{}'.format(n))13 time.sleep(5)14 logging.info('finisdhed{}'.format(n))15 returnn16

17 #创建线程池，容量为3个

18 #executor = futures.ThreadPoolExecutor(max_workers=3)

20 if __name__ == '__main__':21 executor = futures.ProcessPoolExecutor(max_workers=3)22

23 with executor:24 fs =[]25 for i in range(4):26 #提交任务

27 future =executor.submit(worker, i)28 fs.append(future)29

31 whileTrue:32 time.sleep(1)33 logging.info(threading.enumerate())34

35 flag =True36 for f infs:37 logging.info(f.done())38 flag = flag andf.done()39

40 ifflag:41 for f infs:42 logging.info('the ans is {}'.format(f.result()))43 #logging.info(threading.enumerate())

44 #logging.info('====================')

45 break

47 logging.info('--------------------')

with一般放到刚创建池对象后面，最终自动清除池

1 D:\python3.7\python.exe E:/code_pycharm/tt2.py2 2018-10-30 10:54:18,388 8244 SpawnProcess-1begin to work03 2018-10-30 10:54:18,404 5944 SpawnProcess-2begin to work14 2018-10-30 10:54:18,433 5608 SpawnProcess-3begin to work25 2018-10-30 10:54:19,294 8476 MainProcess [<_mainthread started>, , ]6 2018-10-30 10:54:19,294 8476MainProcess False7 2018-10-30 10:54:19,294 8476MainProcess False8 2018-10-30 10:54:19,294 8476MainProcess False9 2018-10-30 10:54:19,294 8476MainProcess False10 2018-10-30 10:54:20,294 8476 MainProcess [<_mainthread started>, , ]11 2018-10-30 10:54:20,294 8476MainProcess False12 2018-10-30 10:54:20,294 8476MainProcess False13 2018-10-30 10:54:20,294 8476MainProcess False14 2018-10-30 10:54:20,294 8476MainProcess False15 2018-10-30 10:54:21,294 8476 MainProcess [<_mainthread started>, , ]16 2018-10-30 10:54:21,294 8476MainProcess False17 2018-10-30 10:54:21,294 8476MainProcess False18 2018-10-30 10:54:21,294 8476MainProcess False19 2018-10-30 10:54:21,294 8476MainProcess False20 2018-10-30 10:54:22,294 8476 MainProcess [<_mainthread started>, , ]21 2018-10-30 10:54:22,294 8476MainProcess False22 2018-10-30 10:54:22,294 8476MainProcess False23 2018-10-30 10:54:22,294 8476MainProcess False24 2018-10-30 10:54:22,294 8476MainProcess False25 2018-10-30 10:54:23,294 8476 MainProcess [<_mainthread started>, , ]26 2018-10-30 10:54:23,294 8476MainProcess False27 2018-10-30 10:54:23,294 8476MainProcess False28 2018-10-30 10:54:23,294 8476MainProcess False29 2018-10-30 10:54:23,295 8476MainProcess False30 2018-10-30 10:54:23,388 8244 SpawnProcess-1finisdhed031 2018-10-30 10:54:23,388 8244 SpawnProcess-1begin to work332 2018-10-30 10:54:23,404 5944 SpawnProcess-2finisdhed133 2018-10-30 10:54:23,433 5608 SpawnProcess-3finisdhed234 2018-10-30 10:54:24,295 8476 MainProcess [<_mainthread started>, , ]35 2018-10-30 10:54:24,295 8476MainProcess True36 2018-10-30 10:54:24,295 8476MainProcess True37 2018-10-30 10:54:24,295 8476MainProcess True38 2018-10-30 10:54:24,295 8476MainProcess False39 2018-10-30 10:54:25,295 8476 MainProcess [<_mainthread started>, , ]40 2018-10-30 10:54:25,295 8476MainProcess True41 2018-10-30 10:54:25,295 8476MainProcess True42 2018-10-30 10:54:25,295 8476MainProcess True43 2018-10-30 10:54:25,295 8476MainProcess False44 2018-10-30 10:54:26,295 8476 MainProcess [<_mainthread started>, , ]45 2018-10-30 10:54:26,295 8476MainProcess True46 2018-10-30 10:54:26,295 8476MainProcess True47 2018-10-30 10:54:26,295 8476MainProcess True48 2018-10-30 10:54:26,295 8476MainProcess False49 2018-10-30 10:54:27,296 8476 MainProcess [<_mainthread started>, , ]50 2018-10-30 10:54:27,296 8476MainProcess True51 2018-10-30 10:54:27,296 8476MainProcess True52 2018-10-30 10:54:27,296 8476MainProcess True53 2018-10-30 10:54:27,296 8476MainProcess False54 2018-10-30 10:54:28,296 8476 MainProcess [<_mainthread started>, , ]55 2018-10-30 10:54:28,296 8476MainProcess True56 2018-10-30 10:54:28,296 8476MainProcess True57 2018-10-30 10:54:28,296 8476MainProcess True58 2018-10-30 10:54:28,296 8476MainProcess False59 2018-10-30 10:54:28,391 8244 SpawnProcess-1finisdhed360 2018-10-30 10:54:29,296 8476 MainProcess [<_mainthread started>, , ]61 2018-10-30 10:54:29,296 8476MainProcess True62 2018-10-30 10:54:29,296 8476MainProcess True63 2018-10-30 10:54:29,296 8476MainProcess True64 2018-10-30 10:54:29,296 8476MainProcess True65 2018-10-30 10:54:29,296 8476 MainProcess the ans is066 2018-10-30 10:54:29,296 8476 MainProcess the ans is 1

67 2018-10-30 10:54:29,296 8476 MainProcess the ans is 2

68 2018-10-30 10:54:29,296 8476 MainProcess the ans is 3

69 2018-10-30 10:54:29,327 8476 MainProcess --------------------

71 Process finished with exit code 0

结果

1 importthreading2 from concurrent importfutures3

4 importlogging5 importtime6

7 FORMAT = '%(asctime)-15s\t %(process)s %(processName)s %(message)s'

8 logging.basicConfig(level=logging.INFO, format=FORMAT)9

10 #这是一个IO 密集型的函数，建议使用多线程

11 defworker(n):12 logging.info('begin to work{}'.format(n))13 time.sleep(5)14 logging.info('finisdhed{}'.format(n))15 returnn16

17 #创建线程池，容量为3个

19 if __name__ == '__main__':20 #executor = futures.ProcessPoolExecutor(max_workers=3)

21 executor = futures.ThreadPoolExecutor(max_workers=3)22

23 with executor:24 fs =[]25 for i in range(4):26 #提交任务

27 future =executor.submit(worker, i)28 fs.append(future)29

31 whileTrue:32 time.sleep(1)33 logging.info(threading.enumerate())34

35 flag =True36 for f infs:37 logging.info(f.done())38 flag = flag andf.done()39

40 ifflag:41 for f infs:42 logging.info('the ans is {}'.format(f.result()))43 #logging.info(threading.enumerate())

44 #logging.info('====================')

45 break

46 logging.info(threading.enumerate())47

48 logging.info('--------------------')

多线程测试

1 D:\python3.7\python.exe E:/code_pycharm/tt2.py2 2018-10-30 10:58:12,862 5604MainProcess begin to work03 2018-10-30 10:58:12,862 5604MainProcess begin to work14 2018-10-30 10:58:12,862 5604MainProcess begin to work25 2018-10-30 10:58:13,862 5604 MainProcess [<_mainthread started>, , , ]6 2018-10-30 10:58:13,862 5604MainProcess False7 2018-10-30 10:58:13,862 5604MainProcess False8 2018-10-30 10:58:13,862 5604MainProcess False9 2018-10-30 10:58:13,862 5604MainProcess False10 2018-10-30 10:58:14,863 5604 MainProcess [<_mainthread started>, , , ]11 2018-10-30 10:58:14,863 5604MainProcess False12 2018-10-30 10:58:14,863 5604MainProcess False13 2018-10-30 10:58:14,863 5604MainProcess False14 2018-10-30 10:58:14,863 5604MainProcess False15 2018-10-30 10:58:15,863 5604 MainProcess [<_mainthread started>, , , ]16 2018-10-30 10:58:15,863 5604MainProcess False17 2018-10-30 10:58:15,863 5604MainProcess False18 2018-10-30 10:58:15,863 5604MainProcess False19 2018-10-30 10:58:15,863 5604MainProcess False20 2018-10-30 10:58:16,863 5604 MainProcess [<_mainthread started>, , , ]21 2018-10-30 10:58:16,863 5604MainProcess False22 2018-10-30 10:58:16,863 5604MainProcess False23 2018-10-30 10:58:16,863 5604MainProcess False24 2018-10-30 10:58:16,863 5604MainProcess False25 2018-10-30 10:58:17,863 5604MainProcess finisdhed226 2018-10-30 10:58:17,863 5604MainProcess begin to work327 2018-10-30 10:58:17,863 5604 MainProcess [<_mainthread started>, , , ]28 2018-10-30 10:58:17,863 5604MainProcess finisdhed129 2018-10-30 10:58:17,863 5604MainProcess finisdhed030 2018-10-30 10:58:17,863 5604MainProcess False31 2018-10-30 10:58:17,864 5604MainProcess True32 2018-10-30 10:58:17,864 5604MainProcess True33 2018-10-30 10:58:17,864 5604MainProcess False34 2018-10-30 10:58:18,864 5604 MainProcess [<_mainthread started>, , , ]35 2018-10-30 10:58:18,864 5604MainProcess True36 2018-10-30 10:58:18,864 5604MainProcess True37 2018-10-30 10:58:18,864 5604MainProcess True38 2018-10-30 10:58:18,864 5604MainProcess False39 2018-10-30 10:58:19,864 5604 MainProcess [<_mainthread started>, , , ]40 2018-10-30 10:58:19,864 5604MainProcess True41 2018-10-30 10:58:19,864 5604MainProcess True42 2018-10-30 10:58:19,864 5604MainProcess True43 2018-10-30 10:58:19,864 5604MainProcess False44 2018-10-30 10:58:20,864 5604 MainProcess [<_mainthread started>, , , ]45 2018-10-30 10:58:20,864 5604MainProcess True46 2018-10-30 10:58:20,864 5604MainProcess True47 2018-10-30 10:58:20,864 5604MainProcess True48 2018-10-30 10:58:20,864 5604MainProcess False49 2018-10-30 10:58:21,864 5604 MainProcess [<_mainthread started>, , , ]50 2018-10-30 10:58:21,864 5604MainProcess True51 2018-10-30 10:58:21,864 5604MainProcess True52 2018-10-30 10:58:21,864 5604MainProcess True53 2018-10-30 10:58:21,864 5604MainProcess False54 2018-10-30 10:58:22,863 5604MainProcess finisdhed355 2018-10-30 10:58:22,864 5604 MainProcess [<_mainthread started>, , , ]56 2018-10-30 10:58:22,864 5604MainProcess True57 2018-10-30 10:58:22,864 5604MainProcess True58 2018-10-30 10:58:22,864 5604MainProcess True59 2018-10-30 10:58:22,864 5604MainProcess True60 2018-10-30 10:58:22,864 5604 MainProcess the ans is061 2018-10-30 10:58:22,864 5604 MainProcess the ans is 1

62 2018-10-30 10:58:22,864 5604 MainProcess the ans is 2

63 2018-10-30 10:58:22,864 5604 MainProcess the ans is 3

64 2018-10-30 10:58:22,864 5604 MainProcess [<_mainthread started>]65 2018-10-30 10:58:22,864 5604 MainProcess --------------------

多线程测试，最终线程都关闭了

总结：

该库统一了线程池，进程池的调用，简化了编程。

缺点是无法设置线程名。