聚焦爬虫案例4：斗图啦

需求：
获取斗图啦第一页所有图片，并将图片下载到本地
导入requests等

import requests
import os
from lxml import etree

定义请求头

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36'
}

发起请求，接收响应

response = requests.get(url='https://www.doutula.com/photo/list/',headers=headers)
# print(response.text)

将字符串转换成html对象

html = etree.HTML(response.text)

使用xpath提取数据
先获取a标签

a_list = html.xpath('//div[@class="page-content text-center"]/div/a[@class="col-xs-6 col-sm-3"]')
# print(a_list)

path = 'F:\Python夏令营\斗图啦\\'  # 新建文件夹，将下载的图片存入
os.mkdir(path)

for a in a_list:
# 获取表情包网址
    jpg = a.xpath('./img/@data-original')[0]  # 获取表情包网址
    # print(jpg)
    jpg_response = requests.get(url=jpg)  # 接收表情包请求
# 获取表情包名称
    name = a.xpath('./p[@style="display: none"]/text()')[0]
    # print(name)
    # 保存文件
    with open(path+'{}.jpg'.format(name),'wb') as fp:
        fp.write(jpg_response.content)