每天学习爬取小知识

大家好！

        嘿嘿好些天没有更新了，人嘛懒，还喜欢努力，这不找了个培训搞竞赛的，这可把我搞坏了，别走，今天教大家爬虫最有意思的事，爬取美女壁纸，相信大家都很喜欢吧

代码如下;

import requests
from lxml import etree
import os

if __name__ == '__main__':
    url = "https://pic.netbian.com/4kmeinv/"
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36'
    }
    result = requests.get(url, headers=headers)
    result.encoding = 'gbk'
    page_text = result.text
    # print(page_text)
    tree = etree.HTML(page_text)
    li_list = tree.xpath('//div[@class="slist"]/ul/li')
    if not os.path.exists("./meinvpic"):
        os.mkdir('./meinvpic')
    for nav in li_list:
        img_url = 'http://pic.netbian.com' + nav.xpath('./a/@href')[0]
        # print(img_url)
        detail_text = requests.get(img_url, headers).text
        detail_tree = etree.HTML(detail_text)
        
        # 爬取地址，名称
        img_src = 'http://pic.netbian.com' + \
                  detail_tree.xpath('//div[2]/div[1]/div[@class="photo"]/div[1]/div[2]/a/img/@src')[0]
        name = detail_tree.xpath('//div[2]/div[1]/div[@class="photo"]/div[1]/div[1]/h1/text()')[0] + '.jpg'
        # 修改乱码方法二
        name = name.encode('iso-8859-1').decode('gbk')
        img_data = requests.get(img_src, headers).content
        # 图片存储
        with open('./meinvpic/' + name, 'wb') as s:
            s.write(img_data)
        print(name)

s.close()
print("ok")

怎么样还可以，我这有个问题要是分页爬取，那要怎么做呢？

你们思考我我也想想吧，后面为大家继续更新，这xpath还是简单的