猿人学第17题、第19题

最新推荐文章于 2024-07-16 16:50:01 发布

J ayus

最新推荐文章于 2024-07-16 16:50:01 发布

阅读量238

点赞数 2

分类专栏：猿人学文章标签：爬虫 python pycharm

本文链接：https://blog.csdn.net/m0_67409976/article/details/140389082

版权

猿人学专栏收录该内容

3 篇文章 0 订阅

订阅专栏

17题使用的是http2，而requests库使用的是http1，所以需要引入httpx库来做，注意需要将cookie中的sessionid改为自己的id。

pip install httpx
pip install httpx[http2]

import httpx

headers = {
    'accept': 'application/json, text/javascript, */*; q=0.01',
    'accept-language': 'zh-CN,zh;q=0.9',
    'cache-control': 'no-cache',
    'cookie': 'sessionid=t0h2emsm1blefhw20i88wobodk8uyz1x; Hm_lvt_c99546cf032aaa5a679230de9a95c7db=1720666756,1720683763,1720695275,1720751878; HMACCOUNT=135DEA475AA0E00B; no-alert3=true; Hm_lvt_9bcbda9cbf86757998a2339a0437208e=1720666878,1720683770,1720695279,1720751881; m=5HCsUCAhq6VU4HujMujwxaW0Ffxk1YzQkr%2Bmb4VWs9OmmtNxRpn7pWUl0hDVoxj9E3SXPHz8UvKLfkf272Q1yAmcB7XXZMjRCc1oTB1xJPUUjD2XstY8Gcga%2BPch5xQsj6iZg7t1Oh0QtXglfBhZKYpbuKplWhbEOvSd%2BkSJjCv8FvHg5tHypKumT5xgeJX5%2Fo%2FAsMjrTfUXJRtyBWhURmsTihFpCDhVqzQ7RWr15W9SleJCRXcUWNL%2FVxPnliTxnlnSlkomGds%2Fyd85E2Aqn1LePngevxRKNyROJ9HL70JV4qn%2FSygjhlNQE6lLDYfZfVcrt0TRpBXz365aD1nM0YQ%3D%3Dr; Hm_lpvt_9bcbda9cbf86757998a2339a0437208e=1720788578; Hm_lpvt_c99546cf032aaa5a679230de9a95c7db=1720788600',
    'pragma': 'no-cache',
    'priority': 'u=0, i',
    'referer': 'https://match.yuanrenxue.cn/match/17',
    'sec-ch-ua': '"Not/A)Brand";v="8", "Chromium";v="126", "Google Chrome";v="126"',
    'sec-ch-ua-mobile': '?0',
    'sec-ch-ua-platform': '"Windows"',
    'sec-fetch-dest': 'empty',
    'sec-fetch-mode': 'cors',
    'sec-fetch-site': 'same-origin',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36',
    'x-requested-with': 'XMLHttpRequest',
}
client = httpx.Client(http2=True)

data = 0
for page in range(1,6):
    params = (
        ('page', page),
    )

    response = client.get('https://match.yuanrenxue.cn/api/match/17', headers=headers, params=params)
    for num in response.json()['data']:
        data += num['value']
    print(data)

19题我直接请求成功了，原本还以为是header格式检测，后来搜索发现是ja3指纹检测。指纹检测可以使用curl_cffi库。利用curl_cffi也可以过五秒盾的免费版。

from curl_cffi import requests

# 注意这个 impersonate 参数，指定了模拟哪个浏览器
r = requests.get("https://tls.browserleaks.com/json", impersonate="chrome101")

print(r.json())