python中done_python基础-线程创建、线程池、进\线程异步回调(add_done_callback)、进\线程数据共享...

线程创建

进程只是用来把资源集中到一起(进程只是一个资源单位，或者说资源集合)，而线程才是cpu上的执行单位。

每个进程有一个地址空间，而且默认就有一个控制线程

线程就是一条流水线工作的过程，一条流水线必须属于一个车间，一个车间的工作过程是一个进程

多线程(即多个控制线程)的概念是，在一个进程中存在多个控制线程，多个控制线程共享该进程的地址空间，相当于一个车间内有多条流水线，都共用一个车间的资源

我们之前了解过进程的2种创建方式

下面的代码是2种创建线程的方式

from threading import Thread

from multiprocessing import Process

import time,os

def task():

print('%s is running' %os.getpid())

time.sleep(2)

print('%s is done' %os.getpid())

class Mythread(Thread):

def __init__(self,name):

super().__init__()

self.name=name

def run(self):

print('%s is running' % os.getpid())

time.sleep(5)

print('%s is done' % os.getpid())

if __name__ == '__main__':

t=Thread(target=task)

# t=Mythread('xxxxx')

t.start()

print('主')

输出如下：

E:\python\python_sdk\python.exe "E:/python/py_pro/1 开启线程的两种方式.py"

10336 is running

主

10336 is done

Process finished with exit code 0

线程进程pid

part1:在主进程下开启多个线程,每个线程都跟主进程的pid一样

from threading import Thread

from multiprocessing import Process

import time,os

def task():

print('partent：%s self：%s' %(os.getppid(),os.getpid()))

time.sleep(5)

if __name__ == '__main__':

t=Thread(target=task,)

# t=Process(target=task,)

t.start()

print('主',os.getppid(),os.getpid())

输出如下：

partent：9052 self：10120

主 9052 10120

开多个进程,每个进程都有不同的pid

from threading import Thread

from multiprocessing import Process

import time,os

def task():

print('partent：%s self：%s' %(os.getppid(),os.getpid()))

time.sleep(5)

if __name__ == '__main__':

t=Process(target=task,)

t.start()

print('主',os.getppid(),os.getpid())

输出如下：

主 9052 2668

partent：2668 self：8744

线程进程数据共享

进程之间数据不共享，但是进程之间可以通过ipc进行数据通讯

from threading import Thread

from multiprocessing import Process

import time,os

n=100

def task():

global n

n=0

if __name__ == '__main__':

t=Process(target=task,)

t.start()

t.join()

print('主',n)

输出如下：

主 100

线程之间内存空间共享

from threading import Thread

import time,os

n=100

def task():

global n

n=0

if __name__ == '__main__':

t=Thread(target=task,)

t.start()

t.join()

print('主',n)

输出如下：

主 0

线程池

from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutor

from threading import current_thread

import time,random

def task(n):

print('%s is running' %current_thread().getName())

time.sleep(random.randint(1,3))

return n**2

if __name__ == '__main__':

t=ThreadPoolExecutor(3) #默认是cpu的核数*5

objs=[]

for i in range(5):

obj=t.submit(task,i)

objs.append(obj)

t.shutdown(wait=True)

for obj in objs:

print(obj.result())

print('主',current_thread().getName())

输出如下：

E:\python\python_sdk\python.exe "E:/python/py_pro/4 线程池.py"

ThreadPoolExecutor-0_0 is running

ThreadPoolExecutor-0_1 is running

ThreadPoolExecutor-0_2 is running

ThreadPoolExecutor-0_0 is running

ThreadPoolExecutor-0_1 is running

0

1

4

9

16

主 MainThread

线程的一些其他方法

from threading import Thread,current_thread,enumerate,active_count

import time,os

def task():

print('%s is running' %current_thread().getName())

time.sleep(5)

print('%s is done' %current_thread().getName())

if __name__ == '__main__':

t=Thread(target=task,name='xxxx')

t.start()

print(t.name)

#查看当前活着的线程

print(enumerate()[0].getName())

print(active_count())

print('主',current_thread().getName())

print()

输出如下：

E:\python\python_sdk\python.exe "E:/python/py_pro/3 线程对象的其他属性或方法.py"

xxxx is running

xxxx

MainThread

2

主 MainThread

xxxx is done

异步-回调函数

ProcessPoolExecutor方式

我们之前总结的异步返回结果没有用到调用函数，接下来的是利用了回调函数

#pip install requests

import requests

from concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutor

from threading import current_thread

import time,os

def get(url):

print('%s GET %s' %(os.getpid(),url))

response=requests.get(url)

time.sleep(3)

if response.status_code == 200:

return {'url':url,'text':response.text}

def parse(obj):

res=obj.result()

print('[%s] (%s)' % (os.getpid(), res['url'],len(res['text'])))

if __name__ == '__main__':

urls = [

'https://www.python.org',

'https://www.baidu.com',

'https://www.jd.com',

'https://www.tmall.com',

]

t=ProcessPoolExecutor(2)

for url in urls:

t.submit(get,url).add_done_callback(parse)

t.shutdown(wait=True)

print('主',os.getpid())

代码思路是：

t=ProcessPoolExecutor(2)开一个进程池，然后去并发下载网络数据，下载完毕后，

在主进程中add_done_callback去解析

这里由于主进程、子进程不是同一个进程空间，所以在解析数据时候，在主进程

输出如下：

E:\python\python_sdk\python.exe "E:/python/py_pro/5 补充异步的概念.py"

5628 GET https://www.python.org

4816 GET https://www.baidu.com

4816 GET https://www.jd.com

[3204] (2443)

[3204] (48856)

5628 GET https://www.tmall.com

[3204] (124541)

[3204] (212080)

主 3204

Process finished with exit code 0

ThreadPoolExecutor方式

import requests

from concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutor

from threading import current_thread

import time

import os

def get(url):

print('%s GET %s,%s' %(current_thread().getName(),os.getpid(),url))

response=requests.get(url)

time.sleep(3)

if response.status_code == 200:

return {'url':url,'text':response.text}

def parse(obj):

res=obj.result()

print('[%s] (%s)' % (current_thread().getName(), res['url'],len(res['text'])))

if __name__ == '__main__':

urls = [

'https://www.python.org',

'https://www.baidu.com',

'https://www.jd.com',

'https://www.tmall.com',

]

t=ThreadPoolExecutor(2)

for url in urls:

t.submit(get,url).add_done_callback(parse)

t.shutdown(wait=True)

print('主',current_thread().getName(),os.getpid())

代码思路是：

t=ThreadPoolExecutor(2)开一个线程池，然后去并发下载网络数据，下载完毕后，

在主线程程中add_done_callback去解析

这里由于主线程、子线程是同一个进程空间，所以在解析数据时候，可能主线程、子线程都会解析

输出如下：

E:\python\python_sdk\python.exe "E:/python/py_pro/5 补充异步的概念.py"

ThreadPoolExecutor-0_0 GET 12956,https://www.python.org

ThreadPoolExecutor-0_1 GET 12956,https://www.baidu.com

[ThreadPoolExecutor-0_1] (2443)

ThreadPoolExecutor-0_1 GET 12956,https://www.jd.com

[ThreadPoolExecutor-0_0] (48856)

ThreadPoolExecutor-0_0 GET 12956,https://www.tmall.com

[ThreadPoolExecutor-0_1] (124541)

[ThreadPoolExecutor-0_0] (212079)

主 MainThread 12956

Process finished with exit code 0