python异步加载

最新推荐文章于 2023-12-10 22:54:17 发布

Pg-Man

最新推荐文章于 2023-12-10 22:54:17 发布

阅读量800

点赞数

分类专栏： Python 文章标签：异步加载

本文链接：https://blog.csdn.net/sinat_38832964/article/details/81911217

版权

Python 专栏收录该内容

28 篇文章 0 订阅

订阅专栏

import requests
import re
from bs4 import BeautifulSoup

headers = {
    'Accept':'text/html,application/xhtml+xm+plication/xml;q=0.9,*/*;q=0.8',
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:61.0) Gecko/201001'
}

urls = ['https://www.pexels.com/?page={}'.format(str(i)) for i in range(1,2)]
photos = []

for url in urls:
    wb_data = requests.get(url,headers=headers)
    soup = BeautifulSoup(wb_data.text,'lxml')
    imgs = soup.select('article.photo-item > a > img')
    for img in imgs:
        photo = img.get('src')
        photos.append(photo)

path = "F://yibu/"

for item in photos:
    data = requests.get(item,headers=headers)
    photo_name = re.findall('\/\d+\/(.*?)\?auto',item)
    if photo_name:
        fp = open(path+photo_name[0],'wb')
        print(path + photo_name[0])
        fp.write(data.content)
        fp.close()