【Python】爬虫-----多线程案例

Python创建多线程的方法

1.准备一个函数

def myfunc(a,b):
 do_craw(a,b)

2.创建一个线程

import threading
t = threading.Thread(target=myfunc,args=(20,30))

3.启动线程

t.start()

4.等待结束

t.join()

案例

import threading
import requests

urls =[
    f"https://www.cnblogs.com/#p{page}"
    for page in range(1,50+1)
]
def craw(url):
    r = requests.get(url)
    print(url,len(r.text))

def mulit_thread():
    threads = []
    for url in urls:
        threads.append(
            threading.Thread(target=craw,args=(url,))
        )
    for thread in threads:
        thread.start()
    for thread in threads:
        thread.join()

if  __name__ == "__main__":
    mulit_thread()