爬虫中的图片爬取

爬虫中的图片爬取

操作步骤

1. 指定url

2. 发出请求

3. 获取响应数据

4. 持久化存储

实例1

//单张图片爬取

import requests
url = "https://p5.ssl.qhimgs1.com/sdr/400__/t01df72ed01439728e5.jpg"
response = requests.post(url=url).content  #content表示以二进制的形式返回数据
with open('./国旗.jpg', "wb") as fp:  #图片的形式需定义为 .jpg 格式
    fp.write(response)

点击此处查看图片原图

实例2

//多张图片爬取

import requests
import os
from lxml import etree
if not os.path.exists('./风景'):
    os.mkdir("./风景")
url = "https://pic.netbian.com/4kfengjing/"
headers = {
    "user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36 Edg/95.0.1020.40"
​
}
response = requests.get(url=url, headers=headers)
page_text = response.text
tree = etree.HTML(page_text)
li_list = tree.xpath('//div[@class="slist"]//li')
​
for li in li_list:
    url1="https://pic.netbian.com"+li.xpath(".//img/@src")[0]
    name=li.xpath(".//img/@alt")[0]+".jpg"
    name= name.encode("iso-8859-1").decode("gbk")
    response1=requests.get(url=url1, headers=headers)
    content1=response1.content
    img_name="风景/"+name
    with open(img_name, "wb") as fp:
        fp.write(content1)
        print(name, url1)