python多进程

基础概念

进程与线程

    进程:是系统资源分配的最小单位
    线程:是系统调度的最小单位

并行与并发

    并行:同时刻同时运行(多核CPU)
    并发:同时段同时运行(实际上是宏观并行,微观串行)

cpu状态

    用户态:cpu运行用户程序
    内核态:cpu运行内核程序

操作系统的状态模型

    就绪:
    运行:
    阻塞:

Linux的五种状态

	运行,可中断睡眠,不可中断睡眠状态,僵尸状态,停止状态
	D:uninterruptible sleep (usually IO)
	R:running or runnable (on run queue)
	S:interruptible sleep (waiting for an event to complete)
	T:stopped by job control signal
	t:stopped by debugger during the tracing
	X:dead (should never be seen)
	Z:defunct ("zombie") process, terminated but not - reaped by its parent

案例
创建线程

import requests
import time
from threading import Thread

def cost(f):
    def _cost(*args, **kwargs):
        start = time.time()
        result = f(*args, **kwargs)
        end = time.time()
        print(f"执行{f.__name__}花费{end-start}s")
        return result
    return _cost

def get_content(url):
    print("start......")
    text = requests.get(url).text
    time.sleep(0.5)
    print("get content")

@cost
def main():
    t_list = []
    for i in range(5):
        t = Thread(target=get_content,args=("https://www.baidu.com",))
        # 创建线程 target:执行函数,args:函数传入的参数
        t_list.append(t)
        # 设置为后台线程, 主线程退出,子线程也退出
        # t.setDaemon(True)
        # 启动线程
        t.start()  #自动调用t.run()
        # t.join()
    #阻塞当前环境上下文,直到为t的线程执行完成
    # [ t.join() for t in t_list]
    print("end.........")
main()

自定义线程类

import threading
# 自定义线程继承Thread
class MyThread(threading.Thread):
    def __init__(self, num):
        super().__init__() # 执行父类的init()方法
        self.num = num
    def run(self):
        print(f"runing on numbers:{self.num}")
t1 = MyThread(1)
t2 = MyThread(2)
t1.start()
t2.start()

互斥锁
解决资源争用,数据读取不一致的问题

import threading
import time

num = 0
def sum_num(i):
    print(f"lock start....{i}")
    # lock.acquire()  #获得锁
    with lock:
        global num
        time.sleep(0.5)
        num += i
        print(num)
    # lock.release()  #释放锁

lock = threading.RLock()  #互斥锁对象
for i in range(5):
    t = threading.Thread(target=sum_num, args=(i,))
    t.start()

Lock:原始锁,获取锁之前不做判断,知道获取锁为止
RLock:重入锁,获取锁之前先判断,如果自己有了锁,那就立即放回

import threading
# lock = threading.Lock()  #死锁
lock = threading.RLock()
lock.acquire()
print("lock acquire...")
lock.acquire()
print("lock acquire...2")
lock.release()
print("lock release")
lock.release()
print("lock release...2")

死锁现象:由于相互等待彼此的资源导致的都不能继续运行的现象
如何避免死锁?
避免一个线程锁定多个Lock
多个线程需要对多个Lock进行加锁行为的时候,尽量保证他们以相同的顺序进行加锁

信号锁

import threading
import time

num = 0
def sum_num(i):
    with lock:
        time.sleep(1)
        print(f"lock start....{i}")
    # lock.acquire()  #获得锁
    with lock2:
        global num
        time.sleep(1)
        num += i
        print(num)
    # lock.release()  #释放锁

lock = threading.BoundedSemaphore(2)  #信号锁,同时定义两把锁,可以有两个线程执行
lock2 = threading.RLock()
for i in range(5):
    t = threading.Thread(target=sum_num, args=(i,))
    t.start()

进程池

from multiprocessing import Pool, current_process
import time

lst = []
def task(i):
    print(current_process().name, i, 'start....')
    # time.sleep(2)
    lst.append(i)
    print(lst)
    print(current_process().name, i, "end....")

if __name__ == "__main__":
    #创建一个进程池,建议进程数和cpu核数一致
    p = Pool(processes=4, maxtasksperchild=3)
    for i in range(20):
        #进程池在接收任务
        p.apply_async(func = task, args=(i,))
    #关闭进程池,不接收任务
    p.close()
    #等待子进程执行完毕,父进程再退出
    p.join()
    print("end....")
# 进程池和process比较
from multiprocessing import Pool, Process
def task():
    for i in range(10**3):
        pass
if __name__ == "__main__":
    p_list = []
    for i in range(2000):
        p = Process(target=task)
        p.start()
        p_list.append(p)
    [p.join() for p in p_list]
    print("end.....")

# 进程池
from multiprocessing import Pool, Process
def task():
    for i in range(10**3):
        pass
if __name__ == "__main__":
    p = Pool(processes=4)
    for i in range(2000):
        p.apply_async(func= task, args=(i,))
    p.close()
    p.join()
    print("end......")
  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值