6.ModuleNotFoundError: No module named 'urllib3' ModuleNotFoundError: No module named 'idna' ModuleNotFoundError: No module named 'certifi'
底层环境中装有这些,但是我的虚拟环境中没有,会出现报错,同时pip install 这些模块会显示,主要原因是版本问题
Requirement already satisfied: requests in /home/u221080022/.local/lib/python3.10/site-packages (2.32.2) Requirement already satisfied: charset-normalizer<4,>=2 in /software/anaconda3/lib/python3.10/site-packages (from requests) (2.0.4) Requirement already satisfied: idna<4,>=2.5 in /home/u221080022/.local/lib/python3.10/site-packages (from requests) (3.7) Requirement already satisfied: urllib3<3,>=1.21.1 in /home/u221080022/.local/lib/python3.10/site-packages (from requests) (2.2.1) Requirement already satisfied: certifi>=2017.4.17 in /software/anaconda3/lib/python3.10/site-packages (from requests) (2022.12.7)
解决办法:把底层的删除,重新pip install 这些模块
import requests import json import time import random def fetch_data(): url = "" headers = { 'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6', 'Connection': 'keep-alive', 'Content-Length': '81', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Cookie': '_webvpn_key=eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyIjoiMjIxMDgwMDIyIiwiZ3JvdXBzIjpbMV0sImlhdCI6MTcwOTgxNTU2MiwiZXhwIjoxNzA5OTAxOTYyfQ._zMzNLEsKmjE8tTTzWpjWGi0J-m4Z3KJPCRR_ZG2ShM; webvpn_username=221080022%7C1709815562%7Cda2bf6abae7016bab2373766d240fdf5db433e38; UM_distinctid=18e18f393cdd62-07ea63f58e0739-4c657b58-1b0000-18e18f393cebf6; tfstk=eMwBmsiiceYQxPOH5wsacLipE7k7dy64J3i8mupe2vHpyUEx5Jlz8Wy_Nrrx9L3-J5T8uzRrzurJV0Zq7WyrYu37NVHKZWur84M7qrsNutW4xkDoBZ7V3phSV6kRi3LPakqnEhxwysPTxLTFvjsUSNO6MMqF5QdWE4TFP2_KKfysAfCavVC_zRmIOqELdLpRMDGIkk314rpqllwBNhGDFcgVfGOkZKJlV0Qs9WVI9cma3Gs6-QcKjcgVfGOkZXnibfS1feAl.; hb_MA-B701-2FC93ACD9328_source=entryhz.qiye.163.com; current_page_id=138369; website_id=89905; website_fid=23571; website_fid_login=0; mh_sign=dc9e4eb31ca922f7a0dd7d8748150313; lan=zh; goc=o; zh_choose=n', 'Host': '', 'Origin': '', 'Referer': '', 'Sec-Ch-Ua': '"Chromium";v="124", "Microsoft Edge";v="124", "Not-A.Brand";v="99"', 'Sec-Ch-Ua-Mobile': '?0', 'Sec-Ch-Ua-Platform': '"Windows"', 'Sec-Fetch-Dest': 'empty', 'Sec-Fetch-Mode': 'cors', 'Sec-Fetch-Site': 'same-origin', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0', 'X-Requested-With': 'XMLHttpRequest' } payload = { 'pageNum': 1, 'pageSize': 10 } try: # 随机延迟,模拟用户行为 time.sleep(random.uniform(1, 3)) # 发送POST请求 response = requests.post(url, headers=headers, data=json.dumps(payload)) # 检查响应状态码 response.raise_for_status() # 解析响应内容 data = response.json() return data except requests.exceptions.RequestException as e: print(f"HTTP请求失败: {e}") return None except json.JSONDecodeError: print("JSON解析错误") return None # 调用函数并打印结果 result = fetch_data() if result: print(json.dumps(result, indent=2, ensure_ascii=False)) else: print("请求失败,请稍后重试")
7.requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
先查看robts.txt https://www.chinaunicom.com/robots.txt
{'code': 0, 'data': None, 'description': '接口请求错误', 'message': '接口请求错误', 'requestId': '', 'status': 200} [23/May/2024 15:13:41] "GET /news/ HTTP/1.1" 200 156
被反爬虫了,一般伪装一下
去请求表头中,找到这些参数