《封号码罗》python爬虫之猿人学第三题详解：访问逻辑 - 推心置腹（十六）_第四页,第五页已锁定。只能使用程序进行协议请求才能看到数据。-CSDN博客

本文链接：https://blog.csdn.net/Python_DJ/article/details/115150172

import requests
import json


def get_cookie():
    url = "http://match.yuanrenxue.com/logo"

    headers = {
        "Host": "match.yuanrenxue.com",
        "Connection": "keep-alive",
        "Content-Length": "0",
        "User-Agent": "yuanrenxue.project",
        # 第四页，第五页已锁定。只能使用程序进行协议请求才能看到数据。在使用程序请求这两个页面时请将User-Agent设置为： yuanrenxue.project
        "Accept": "*/*",
        "Origin": "http://match.yuanrenxue.com",
        "Referer": "http://match.yuanrenxue.com/match/3",
        "Accept-Encoding": "gzip, deflate",
        "Accept-Language": "zh-CN,zh;q=0.9",
    }

    session = requests.session()
    session.headers = headers
    res = session.post(url=url)
    # print(res.cookies)
    return session


def get_data(session, page_num):
    url = f"http://match.yuanrenxue.com/api/match/3?page={page_num}"

    response = session.get(url=url)
    # print(response.text)
    return response.json()


if __name__ == '__main__':
    data_list = list()
    for i in range(1, 6):
    	# 考得是请求逻辑，用fiddler抓包可以看到，每次都请求logo
        session = get_cookie()
        result = get_data(session, i)
        data_list.extend([_["value"] for _ in result["data"]])  # extend 列表合并
    max_label = max(data_list, key=data_list.count)  # 求列表中某个元素出现频率最高
    print(max_label)