目录
使用threading
Python中threading中的Thread类用来创建和管理线程对象,可以通过两种形式来使用多线程:直接使用Thread类、继承Thread类重写init和run方法;创建了Thread对象后可以调用start来启动,该方法会自动调用run,调用后处于alive状态直到线程结束
构造方法:
Thread(group=None, target=None, name=None, args=(), kwargs={})
其中
group: 线程组,目前还没有实现,库引用中提示必须是None;
target: 要执行的方法;
name: 线程名;
args/kwargs: 要传入方法的参数。
对象成员
实例方法:
isAlive(): 返回线程是否在运行。正在运行指启动后、终止前。
get/setName(name): 获取/设置线程名。
start(): 线程准备就绪,等待CPU调度
is/setDaemon(bool): 获取/设置是后台线程(默认前台线程(False))。(在start之前设置)
如果是后台线程,主线程执行过程中,后台线程也在进行,主线程执行完毕后,后台线程不论成功与否,主线程和后台线程均停止
如果是前台线程,主线程执行过程中,前台线程也在进行,主线程执行完毕后,等待前台线程也执行完成后,程序停止
start(): 启动线程。
join([timeout]): 阻塞当前上下文环境的线程,直到调用此方法的线程终止或到达指定的timeout(可选参数)。
直接使用Thread类创建线程
# !/usr/bin/env python
# encoding: utf-8
import threading
def do_soming(what):
print("what value:{0}, threading ident:{1}".format(what, threading.get_ident()))
if __name__ == '__main__':
for i in range(4):
t = threading.Thread(target=do_soming, args=(i, ))
t.start()
print("main end.")
继承Thread类重写init和run方法创建多个线程
# !/usr/bin/env python
# encoding: utf-8
import threading
import time
class MyThread(threading.Thread):
def __init__(self, arg):
super(MyThread, self).__init__() #一定要调用父类的初始化
self._arg = arg
def run(self): #run是线程跑起来之后调用的函数
time.sleep(1)
print('arg value is {0}, the ident is {1}'.format(self._arg, self._ident))
if __name__ == '__main__':
for i in range(4):
t = MyThread(i)
t.start()
print("main end.")
设置线程的daemon属性
线程的daemon属性默认为False,如果创建子线程的时候,给子线程的daemon属性设置了True。则主线程结束时候把所有的子线程都给带走了,因此你不会看到子线程的输出
# !/usr/bin/env python
# encoding: utf-8
import threading
import time
class MyThread(threading.Thread):
def __init__(self, arg):
super(MyThread, self).__init__() #一定要调用父类的初始化
self._arg = arg
def run(self): #run是线程跑起来之后调用的函数
time.sleep(10)
print('arg value is {0}, the ident is {1}, 我会在main end.下面输出'.format(self._arg, self._ident))
if __name__ == '__main__':
for i in range(4):
t = MyThread(i)
t.setDaemon(True)
t.start()
print("main end.")
反之,则会等待子线程结束,主线程才会退出,所以线程中的输出语句会出现在main end.下边
# !/usr/bin/env python
# encoding: utf-8
import threading
import time
class MyThread(threading.Thread):
def __init__(self, arg):
super(MyThread, self).__init__() #一定要调用父类的初始化
self._arg = arg
def run(self): #run是线程跑起来之后调用的函数
time.sleep(10)
print('arg value is {0}, the ident is {1}, 我会在main end.下面输出'.format(self._arg, self._ident))
if __name__ == '__main__':
for i in range(4):
t = MyThread(i)
t.setDaemon(False)
t.start()
print("main end.")
线程的同步
没做线程同步之前的,可能会出现问题,比如输出挤在一起
# !/usr/bin/env python
# encoding: utf-8
import threading
def do_soming(what):
print("what value:{0}, threading ident:{1}".format(what, threading.get_ident()))
if __name__ == '__main__':
for i in range(10):
t = threading.Thread(target=do_soming, args=(i, ))
t.start()
print("main end.")
Lock/RLock对象
一个锁有两种状态:locked和unlocked,如果一个锁处于unlocker状态,acquire()将其修改成locked并立即返回,如果处于locked状态,那么阻塞等待其他线程释放锁,然后将其修改成locked并立即返回
RLock是递归锁,RLock对象的acquire()和release()调用可以嵌套
在前面的例子中使用Lock/RLock
# !/usr/bin/env python
# encoding: utf-8
import threading
import time
def do_soming(what, lock):
lock.acquire()
print("what value:{0}, threading ident:{1}".format(what, threading.get_ident()))
time.sleep(2)#为了以示区分而加上
lock.release()
if __name__ == '__main__':
lock = threading.Lock()
# 也可以改成
# lock = threading.RLock()
for i in range(10):
t = threading.Thread(target=do_soming, args=(i, lock))
t.start()
print("main end.")
Condition对象
我们可以用Condition对象来轻易的模拟计算机操作系统一个常用的线程的例子—生产者消费者问题,Condition对象不值在获取了对象后,可以进入等待状态
当队列为空时,消费者进行等待,生产者进行生产,完成生产后唤醒所有的消费者,让所有的消费者进行“消费”;反过来则是生产者等待消费者唤醒
# !/usr/bin/env python
# encoding: utf-8
import threading
import time
#生产者
class Producer(threading.Thread):
def __init__(self, thread_name, condition_object, container):
threading.Thread.__init__(self, name=thread_name)
self._condition_object = condition_object
self._container = container
def run(self):
#判断当前的Queue是否满,如果满了就wait等待唤醒
#如果没有满,则锁定当前的Queue,往Queue里面添加数据
value = 1
while True:
self._condition_object.acquire()#获取对象
if self._container.__len__() > 4:
self._condition_object.wait()
print("producter waitring...")
else:
print("producter making...")
self._container.append(value)
value += 1
print("producter making end.")
self._condition_object.notify_all()
self._condition_object.release()#释放对象
time.sleep(2)
#消费者
class Consumer(threading.Thread):
def __init__(self, thread_name, condition_object, container):
threading.Thread.__init__(self, name=thread_name)
self._condition_object = condition_object
self._container = container
def run(self):
self._condition_object.acquire()#获取对象
#判断当前的Queue是否空,如果空就wait等待唤醒
#如果没有满,则锁定当前的Queue,往Queue里面添加数据
while True:
if not self._container:#队列为空
self._condition_object.wait()
print("{0} waitring...".format(self.getName()))
else:
print("{0} making...".format(self.getName()))
print("x.pop() value is {0}".format(self._container.pop()))
print("{0} making end...".format(self.getName()))
self._condition_object.notify_all()
self._condition_object.release()#释放对象
time.sleep(2)
if __name__ == '__main__':
container = []
condition_object = threading.Condition()
producer = Producer('producer', condition_object, container)
consumer1 = Consumer('consumer1', condition_object, container)
consumer2 = Consumer('consumer2', condition_object, container)
producer.start()
consumer1.start()
consumer2.start()
print("main end.")
Event对象
用Event对象来实现生产者、消费者之间的同步操作,但由于event没有锁,没办法实现多线程的同步阻塞,不过刚好能用在两个进程之间的同步中
# !/usr/bin/env python
# encoding: utf-8
import threading
import time
#生产者
class Producer(threading.Thread):
def __init__(self, thread_name, event, container):
threading.Thread.__init__(self, name=thread_name)
self._container = container
self._event = event
def run(self):
value = 1
while True:
if not self._event.isSet():#没有被设置
self._container.append(value)
value += 1
print("producer making end...")
self._event.set()
else:
print("producer waitting...")
time.sleep(2)
#消费者
class Consumer(threading.Thread):
def __init__(self, thread_name, event, container):
threading.Thread.__init__(self, name=thread_name)
self._event = event
self._container = container
def run(self):
while True:
if self._event.isSet():#没有被设置
self._event.wait()
print("consumer making x value {0}...".format(self._container.pop()))
self._event.clear()
else:
print("consumer waitting...")
time.sleep(2)
if __name__ == '__main__':
container = []
event = threading.Event()
producer = Producer('producer', event, container)
consumer = Consumer('consumer', event, container)
producer.start()
consumer.start()
print("main end.")
其他一些类
timer类
threading中的定时器的意思,作用就是能够实现让一个函数被延迟调用
# !/usr/bin/env python
# encoding: utf-8
import threading
def do_soming(what):
print("what value:{0}".format(what))
if __name__ == '__main__':
timer = threading.Timer(5, do_soming, args=("zeng", ))
timer.start()
local类
假如使用了threading下的local类,在thread中做出的修改是不会生效的
# !/usr/bin/env python
# encoding: utf-8
import threading
import time
local = threading.local()
local.tname = 'main'
name = "zeng"
def do_soming(what):
# local.tname = 'child'
global local
global name
local.tname = 'zengraoli'#虽然local被做成了全局,但依旧无法改变
name = "zengraoli" #为了对比,这里的name的的确确被改变了
print("what value:{0}, local value:{1}, name value:{2}".format(what, local.tname, name))#what value:zeng, local value:zengraoli, name value:zengraoli
time.sleep(2)
if __name__ == '__main__':
print("local value:{0}, name value:{1}".format(local.tname, name))#local value:main, name value:zeng
t = threading.Thread(target=do_soming, args=("zeng", ))
t.start()
t.join()#阻塞等待线程
print("local value:{0}, name value:{1}".format(local.tname, name))#local value:main, name value:zengraoli
print("main end.")#main end.
更多使用请查看
https://www.cnblogs.com/tkqasn/p/5700281.html
使用线程池
需要安装其他模块,如果需要使用到,请参考
https://blog.csdn.net/hehe123456zxc/article/details/52258358
或者使用下面的map来取代这里的进程池
map实现多线程
map在Python一般都会自动开启多线程、多进程的形式来处理分布式的应用,在Python中有两个库包含了map函数,一个是multiprocessing另外一个是multiprocessing.dummy,唯一的不同是前者使用的是进程,后者使用的是线程
下面的例子,使用了multiprocessing.dummy
# !/usr/bin/env python
# encoding: utf-8
from multiprocessing.dummy import Pool as ThreadPool
import time
def do_soming(what):
print("what value is {0}".format(what))
time.sleep(3)
return "what value is {0}".format(what)
if __name__ == '__main__':
urls = ['http://www.baidu.com', 'http://www.sina.com', 'http://www.qq.com']
pool = ThreadPool()
results = pool.map(do_soming, urls)
print(results)#['what value is http://www.baidu.com', 'what value is http://www.sina.com', 'what value is http://www.qq.com']
pool.close()
pool.join()
print("main end.")
还有使用multiprocessing的map,会开启N个多进程
# !/usr/bin/env python
# encoding: utf-8
import multiprocessing
import time
def do_soming(what):
print("what value is {0}".format(what))
time.sleep(30)
return "what value is {0}".format(what)
if __name__ == '__main__':
urls = ['http://www.baidu.com', 'http://www.sina.com', 'http://www.qq.com']
pool = multiprocessing.Pool(processes=3)
results = pool.map(do_soming, urls)
print(results)#['what value is http://www.baidu.com', 'what value is http://www.sina.com', 'what value is http://www.qq.com']
pool.close()
pool.join()
print("main end.")
Python多线程的缺陷
在Python中因为有GIL全局所,所以当你以为是多线程在并发运行的时候,其实当前只有一个线程在工作。因此,更建议去使用多进程
Python多线程的应用场景
在IO密集型任务中更适合使用多线程,如果IO密集型任务中使用的是多进程,则进程的切换涉及到IO的操作,会比较慢;
在计算密集型任务中适合使用多进程,毕竟可以并发。