爬取图片示例

最新推荐文章于 2022-12-15 23:11:20 发布

云月惊鸿来2

最新推荐文章于 2022-12-15 23:11:20 发布

阅读量82

点赞数

文章标签： python 开发语言网络爬虫

本文链接：https://blog.csdn.net/qq_64056169/article/details/124957243

版权

具体代码如下：

import requests
from lxml import etree
import os

url = 'http://pic.netbian.com/'
headers = {
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Safari/537.36 Edg/101.0.1210.32'
}

response = requests.get(url, headers)

# response.encoding = 'utf-8'  # 修改乱码方法一
page_text = response.text
tree = etree.HTML(page_text)
# 获取图片表格li
li_list = tree.xpath('//div[@class="slist"]/ul[@class="clearfix"]/li')
# 创建文件夹
if not os.path.exists('./bianpic'):
    os.mkdir('./bianpic')
for nav in li_list:
    # 详细图片网页
    img_url = 'http://pic.netbian.com' + nav.xpath('./a/@href')[0]
    # print(img_url)
    detail_text = requests.get(img_url, headers).text
    detail_tree = etree.HTML(detail_text)
    # 爬取地址，名称
    img_src ='http://pic.netbian.com' + detail_tree.xpath('//div[2]/div[1]/div[@class="photo"]/div[1]/div[2]/a/img/@src')[0]
    name = detail_tree.xpath('//div[2]/div[1]/div[@class="photo"]/div[1]/div[1]/h1/text()')[0]+'.jpg'
    # 修改乱码方法二
    name = name.encode('iso-8859-1').decode('gbk')
    img_data = requests.get(img_src, headers).content
    # 图片存储
    with open('./bianpic/' + name, 'wb') as fp:
        fp.write(img_data)
    print(name)
    print(img_src)