python多线程与threading模块

python多线程与_thread模块　中介绍了线程的基本概念以及_thread模块的简单示例。然而，_thread模块过于简单，使得我们无法用它来准确地控制线程，本文介绍threading模块，它提供了更强大的多线程管理方案。

 

threading模块的对象

Thread　　表示一个执行线程的对象　

Lock　　锁原语

RLock　　可重入锁对象，使单一线程可以再次获得已持有的锁（递归锁）

Condition　　条件变量对象，使得一个线程等待另一个线程满足特定条件

Event　　条件变量的通用版本，任意数量的线程等待某个事件的发生，该事件发生后所有线程将被激活

Semaphore　　为线程间的共享资源提供了一个计数器，如果没有可用资源时会被阻塞

BoundedSemaphone　　与Semaphore相似，不过它不允许超过初始值

Timer　　与Thread相似，不过运行前要等待一段时间

Barrier　　创建一个”障碍“,必须要达到指定数量的线程才能继续

Thread对象

Thread类表示在单独的控制线程中运行的活动。

主要方法：

threading.Thread(group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None)

//target是将被run()方法调用的可调用对象。默认为None，表示不调用任何东西。

//name是线程的名字。默认情况下，以“Thread-N”的形式构造一个唯一的名字，N是一个小的十进制整数。

//args是给调用目标的参数元组。默认为()。

//kwargs是给调用目标的关键字参数的一个字典。默认为{}

//daemon表示是否为守护线程

 

start()　　开始执行线程　　

run()　　//定义线程功能

join(timeout=None)　　//直至启动的线程之前一直挂起，除非给出timeout时间，否则一直阻塞

gerName()　　//返回线程名

属性：

name　　//线程名

ident　　//线程标识符

daemon　　//是否为守护线程

 

Threading模块主要函数

threading.active_count()　　//返回当前处于alive状态的Thread对象的个数。返回的数目等于enumerate()返回的列表的长度。

threading.current_thread()　　//返回当前的Thread对象，对应于调用者控制的线程。

threading.get_ident()　　//返回当前线程的'线程标识符'。它是一个非零的整数。它的价值没有直接的意义。

threading.enumerate()　　//返回当前活着的Thread对象的列表。该列表包括守护线程、由current_thread()创建的虚假线程对象和主线程。它不包括已终止的线程和尚未开始的线程。

threading.main_thread()　　//返回主 Thread 对象。在正常情况下，主线程是从 Python 解释器中启动的线程。

守护线程

如果把一个线程设置为守护线程，就表示这个线程是不重要的，进程退出时不需要等待这些线程执行完成。

执行如下赋值语句可以将一个线程设置为守护线程。thread.daemon=True。

一个新线程会继承父线程的守护标记。 

使用锁

锁有两种状态：锁定和未锁定。同时也只支持两种函数，获得锁和释放锁。

lock.acquire()　　//获取锁

lock.release()　　//释放锁

当多线程竞争时，允许第一个获得锁的线程进入临界区，执行相应代码。所有之后到达的线程将被阻塞，直到第一个线程执行结束，退出临界区，释放锁。此时，等待的线程可以获得锁并进入临界区，哪个正在等待的线程获取锁是随机的。

#!/usr/bin/env python3
# coding:utf-8
from atexit import register
from random import randrange
from threading import Thread, current_thread, Lock
from time import sleep, ctime


class CleanOutputSet(set):
    def __str__(self):    //改变输出格式
        return ', '.join(x for x in self)


#lock = Lock()
loops = (randrange(2, 5) for x in range(randrange(3, 7)))
remaining = CleanOutputSet()


def loop(nsec):
    myname = current_thread().name    //获取当前线程名
    remaining.add(myname)    //添加到集合
    print('[{0}] Start {1}'.format(ctime(), myname))
    sleep(nsec)
    remaining.remove(myname)
    print('[{0}] Completed {1} ({2} secs)'.format(ctime(), myname, nsec))
    print('  (remaining: {0})'.format(remaining or 'None'))


def main():
    for pause in loops:
        Thread(target=loop, args=(pause,)).start()


@register
def _atexit():
    print('all DONE at:', ctime())


if __name__ == '__main__':
    main()
 
不使用锁的情况

[Wed Jan 24 20:41:48 2018] Start Thread-1
[Wed Jan 24 20:41:48 2018] Start Thread-2
[Wed Jan 24 20:41:48 2018] Start Thread-3
[Wed Jan 24 20:41:48 2018] Start Thread-4
[Wed Jan 24 20:41:48 2018] Start Thread-5
[Wed Jan 24 20:41:48 2018] Start Thread-6
[Wed Jan 24 20:41:50 2018] Completed Thread-3 (2 secs)
  (remaining: Thread-6, Thread-1, Thread-2, Thread-5, Thread-4)
[Wed Jan 24 20:41:50 2018] Completed Thread-4 (2 secs)
  (remaining: Thread-6, Thread-1, Thread-2, Thread-5)
[Wed Jan 24 20:41:51 2018] Completed Thread-5 (3 secs)
  (remaining: Thread-6, Thread-1, Thread-2)
[Wed Jan 24 20:41:52 2018] Completed Thread-1 (4 secs)
[Wed Jan 24 20:41:52 2018] Completed Thread-6 (4 secs)
  (remaining: Thread-2)
  (remaining: Thread-2)
[Wed Jan 24 20:41:52 2018] Completed Thread-2 (4 secs)
  (remaining: None)
all DONE at: Wed Jan 24 20:41:52 2018

输出结果

输出结果中可以看到，多个线程并行I/O导致结果混乱。I/O和访问相同的数据结构都属于临界区，因此需要用锁来防止多个线程同时进入临界区。

#!/usr/bin/env python3
# coding:utf-8
from atexit import register
from random import randrange
from threading import Thread, current_thread, Lock
from time import sleep, ctime


class CleanOutputSet(set):
    def __str__(self):  # 改变输出格式
        return ', '.join(x for x in self)


lock = Lock()
loops = (randrange(2, 5) for x in range(randrange(3, 7)))
remaining = CleanOutputSet()


def loop(nsec):
    myname = current_thread().name   #　获取当前线程名
    lock.acquire()
    remaining.add(myname)   #　加入集合
    print('[{0}] Start {1}'.format(ctime(), myname))
    lock.release()
    sleep(nsec)
    lock.acquire()
    remaining.remove(myname)
    print('[{0}] Completed {1} ({2} secs)'.format(ctime(), myname, nsec))
    print('  (remaining: {0})'.format(remaining or 'None'))
    lock.release()


def main():
    for pause in loops:
        Thread(target=loop, args=(pause,)).start()


@register
def _atexit():
    print('all DONE at:', ctime())


if __name__ == '__main__':
    main()
使用锁

[Wed Jan 24 20:51:11 2018] Start Thread-1
[Wed Jan 24 20:51:11 2018] Start Thread-2
[Wed Jan 24 20:51:11 2018] Start Thread-3
[Wed Jan 24 20:51:11 2018] Start Thread-4
[Wed Jan 24 20:51:11 2018] Start Thread-5
[Wed Jan 24 20:51:13 2018] Completed Thread-2 (2 secs)
  (remaining: Thread-3, Thread-1, Thread-4, Thread-5)
[Wed Jan 24 20:51:14 2018] Completed Thread-3 (3 secs)
  (remaining: Thread-1, Thread-4, Thread-5)
[Wed Jan 24 20:51:15 2018] Completed Thread-4 (4 secs)
  (remaining: Thread-1, Thread-5)
[Wed Jan 24 20:51:15 2018] Completed Thread-1 (4 secs)
  (remaining: Thread-5)
[Wed Jan 24 20:51:15 2018] Completed Thread-5 (4 secs)
  (remaining: None)
all DONE at: Wed Jan 24 20:51:15 2018

输出结果

使用信号量

锁只能控制临界区的访问，面对一些更复杂的情况就无能为力了。比如，读者与写者共享一段8单位长的缓冲区，读者每次取走1单位，写者每次产生1单位。使用锁显然无法解决，这就用到信号量。

信号量时最古老的原语之一。它是一个计数器，当资源消耗时递减，资源释放时递增。

threading.Semaphore(value=1)　　//构造函数（也可使用threading.BoundedSemaphore(value=1)，value可自行设置，BoundedSemaphore创建的信号量不可超过初值，超过将抛出ValueError）

semaphore.acquire(block=true,timeout=None)　　//资源消耗，block为true时，若无法获得资源，将阻塞timeout时间等待其他线程释放资源。block为false时，无法获得资源将抛出ValueError

semaphore.release()　　//资源增加

#!/usr/bin/env python3
# coding:utf-8
from atexit import register
from random import randrange
from threading import BoundedSemaphore, Lock, Thread
from time import sleep, ctime

lock = Lock()
MAX = 5
candytray = BoundedSemaphore(MAX)


def refill():
    lock.acquire()
    print('Refilling candy...')
    try:
        candytray.release()
    except ValueError:
        print('full, skipping')
    else:
        print('OK')
    lock.release()


def buy():
    lock.acquire()
    print('Buying candy...')
    if candytray.acquire(False):
        print('OK')
    else:
        print('empty, skipping')
    lock.release()


def producer(loops):
    for i in range(loops):
        refill()
        sleep(randrange(3))


def consumer(loops):
    for i in range(loops):
        buy()
        sleep(randrange(3))


def main():
    print('starting at:', ctime())
    nloops = randrange(2, 6)
    print('THE CANDY MACHINE (full with {0} bars)'.format(MAX))
    Thread(target=consumer, args=(randrange(nloops, nloops+MAX+2),)).start()
    Thread(target=producer, args=(nloops,)).start()


@register
def _atexit():
    print('all DONE at:', ctime())


if __name__ == '__main__':
    main()

使用信号量

以上为糖果机信号量示例。糖果机5个槽,refill()函数添加糖果，buy()函数买走糖果。这两个函数对型号量的作用是相反的。

starting at: Wed Jan 24 21:16:00 2018
THE CANDY MACHINE (full with 5 bars)
Buying candy...
OK
Refilling candy...
OK
Refilling candy...
full, skipping
Buying candy...
OK
Refilling candy...
OK
Buying candy...
OK
Buying candy...
OK
Buying candy...
OK
Buying candy...
OK
Buying candy...
OK
Buying candy...
empty, skipping
Buying candy...
empty, skipping
all DONE at: Wed Jan 24 21:16:05 2018

输出结果

使用Condition

可以把Condiftion理解为一把高级的琐，它提供了比Lock, RLock更高级的功能，允许我们能够控制复杂的线程同步问题。threadiong.Condition在内部维护一个琐对象（默认是RLock），可以在创建Condigtion对象的时候把琐对象作为参数传入。Condition也提供了acquire, release方法，其含义与琐的acquire, release方法一致，其实它只是简单的调用内部琐对象的对应的方法而已。Condition还提供了如下方法(特别要注意：这些方法只有在占用琐(acquire)之后才能调用，否则将会报RuntimeError异常。)：

Condition.wait([timeout])　　//wait方法释放内部所占用的琐，同时线程被挂起，直至接收到通知被唤醒或超时（如果提供了timeout参数的话）。当线程被唤醒并重新占有琐的时候，程序才会继续执行下去。

Condition.notify()　　//唤醒一个挂起的线程（如果存在挂起的线程）。注意：notify()方法不会释放所占用的琐。

Condition.notify_all()

Condition.notifyAll()　　//唤醒所有挂起的线程（如果存在挂起的线程）。注意：这些方法不会释放所占用的琐。

from threading import Condition, Thread
from time import ctime


con = Condition()


def get_odd():
    count = 3
    while(True):
        con.acquire(timeout=1)
        print(1)
        count -= 1
        if count is 0:
            break
        con.notify()
        con.wait()


def get_even():
    count = 3
    while(True):
        con.acquire(timeout=1)
        print(2)
        count -= 1
        if count is 0:
            break
        con.notify()
        con.wait()



def main():
    print('start at:', ctime())
    Thread(target=get_odd, name='').start()
    Thread(target=get_even, name='').start()


if __name__ == '__main__':
    main()

创建线程的三种方法

以生产者消费者模型为例，缓冲区大小为５，开始为空。生产者产生一个数置入缓冲区，消费者取走一个数。生产者运行10次，消费者运行5次 

方法一：创建Thread实例，并传给它一个函数(包括函数参数)

#!/usr/bin/env python3
# coding:utf-8'

from atexit import register
from time import ctime
from random import randint
from threading import BoundedSemaphore, Semaphore, Lock, Thread

lock = Lock()
buffer = [0 for x in range(5)]
MAX = len(buffer)
sem_full = Semaphore(0)
sem_empty = BoundedSemaphore(MAX)


def producer():
    sem_empty.acquire()
    lock.acquire()
    print("produce a number")
    for i in range(len(buffer)):
        if buffer[i] is 0:
            buffer[i] = randint(1, 10)
            break
    print(buffer)
    sem_full.release()
    lock.release()


def consumer():
    sem_full.acquire()
    lock.acquire()
    print("consume a number")
    for i in range(len(buffer)):
        if buffer[i] is not 0:
            buffer[i] = 0
            break
    print(buffer)
    sem_empty.release()
    lock.release()


def main():
    print('starting at:', ctime())
    for i in range(5):
        Thread(target=consumer, args='').start()
        Thread(target=producer, args='').start()
        Thread(target=producer, args='').start()


@register
def _atexit():
    print("all DOne at:", ctime())


if __name__ == '__main__':
    main()

方法二：创建Thread实例，传给它一个可调用的类实例（将调用__call__方法）

#!/usr/bin/env python3
# coding:utf-8

from atexit import register
from time import ctime
from random import randint
from threading import BoundedSemaphore, Semaphore, Lock, Thread

lock = Lock()
buffer = [0 for x in range(5)]
MAX = len(buffer)
sem_full = Semaphore(0)
sem_empty = BoundedSemaphore(MAX)


class ThreadFunc(object):

    def __init__(self, func, args, name=""):
        self.name = name
        self.func = func
        self.args = args

    def __call__(self, *args, **kwargs):
        self.func(*self.args)


def producer():
    sem_empty.acquire()
    lock.acquire()
    print("produce a number")
    for i in range(len(buffer)):
        if buffer[i] is 0:
            buffer[i] = randint(1, 10)
            break
    print(buffer)
    sem_full.release()
    lock.release()


def consumer():
    sem_full.acquire()
    lock.acquire()
    print("consume a number")
    for i in range(len(buffer)):
        if buffer[i] is not 0:
            buffer[i] = 0
            break
    print(buffer)
    sem_empty.release()
    lock.release()


threads = []


def main():
    print('starting at:', ctime())
    for i in range(5):
        c = Thread(target=ThreadFunc(consumer, '', consumer.__name__))
        threads.append(c)
        p1 = Thread(target=ThreadFunc(producer, '', producer.__name__))
        threads.append(p1)
        p2 = Thread(target=ThreadFunc(producer, '', producer.__name__))
        threads.append(p2)

    for i in range(len(threads)):
        threads[i].start()

    for i in range(len(threads)):
        threads[i].join()


@register
def _atexit():
    print("all DOne at:", ctime())


if __name__ == '__main__':
    main()

方法三：派生Thread子类，创建子类的实例（重写run方法）

#!/usr/bin/env python3
# coding:utf-8

from atexit import register
from time import ctime
from random import randint
from threading import BoundedSemaphore, Semaphore, Lock, Thread

lock = Lock()
buffer = [0 for x in range(5)]
MAX = len(buffer)
sem_full = Semaphore(0)
sem_empty = BoundedSemaphore(MAX)


class MyThread(Thread):

    def __init__(self, func, args, name=""):
        Thread.__init__(self)
        self.name = name
        self.func = func
        self.args = args

    def run(self):
        self.func(*self.args)


def producer():
    sem_empty.acquire()
    lock.acquire()
    print("produce a number")
    for i in range(len(buffer)):
        if buffer[i] is 0:
            buffer[i] = randint(1, 10)
            break
    print(buffer)
    sem_full.release()
    lock.release()


def consumer():
    sem_full.acquire()
    lock.acquire()
    print("consume a number")
    for i in range(len(buffer)):
        if buffer[i] is not 0:
            buffer[i] = 0
            break
    print(buffer)
    sem_empty.release()
    lock.release()


threads = []


def main():
    print('starting at:', ctime())
    for i in range(5):
        c = MyThread(consumer, '', consumer.__name__)
        threads.append(c)
        p1 = MyThread(producer, '', producer.__name__)
        threads.append(p1)
        p2 = MyThread(producer, '', producer.__name__)
        threads.append(p2)

    for i in range(len(threads)):
        threads[i].start()

    for i in range(len(threads)):
        threads[i].join()


@register
def _atexit():
    print("all DOne at:", ctime())


if __name__ == '__main__':
    main()

输出结果

starting at: Wed Jan 24 22:25:05 2018
produce a number
[7, 0, 0, 0, 0]
produce a number
[7, 9, 0, 0, 0]
consume a number
[0, 9, 0, 0, 0]
produce a number
[7, 9, 0, 0, 0]
consume a number
[0, 9, 0, 0, 0]
consume a number
[0, 0, 0, 0, 0]
produce a number
[9, 0, 0, 0, 0]
consume a number
[0, 0, 0, 0, 0]
produce a number
[9, 0, 0, 0, 0]
produce a number
[9, 5, 0, 0, 0]
produce a number
[9, 5, 6, 0, 0]
produce a number
[9, 5, 6, 10, 0]
consume a number
[0, 5, 6, 10, 0]
produce a number
[7, 5, 6, 10, 0]
produce a number
[7, 5, 6, 10, 2]
all DOne at: Wed Jan 24 22:25:05 2018