Python爬虫json解析问题处理

在用爬虫获取12306的车票信息时（参考的文章），按链接博主所给出的代码遇到了json解析问题，最终通过添加cookie的方式解决了问题。

参考的文章中给出的url为https://kyfw.12306.cn/otn/resources/js/framework/station_name.js?station_version=1.9090

注意这里的url和之后所需要的cookie是对应的，获取url的方式可以参考上面的文章。

我在12306中选择的需要爬取的数据为从上海——北京的车票信息，对应的url为https://kyfw.12306.cn/otn/resources/js/framework/station_name.js?station_version=1.9255

 （将所给参考文章中代码的url改为你需要的url）

然后按照博主的代码最终运行后

train_url = 'https://kyfw.12306.cn/otn/leftTicket/query?leftTicketDTO.train_date={}&leftTicketDTO.from_station={}&leftTicketDTO.to_station={}&purpose_codes=ADULT'.format(
    chufa_riqi, chufa_code, daoda_code)
# 处理json格式的文件
web_data = requests.get(train_url)
# 获取data
json_data = web_data.json()['data']
# 获取result
json_result = json_data['result']

会出现如下报错：

Traceback (most recent call last): File "C:\Users\HUAWEI\PycharmProjects\爬虫\venv\lib\site-packages\requests\models.py", line 971, in json return complexjson.loads(self.text, **kwargs) File "D:\Python&Pycharm\Python\lib\json\__init__.py", line 346, in loads return _default_decoder.decode(s) File "D:\Python&Pycharm\Python\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "D:\Python&Pycharm\Python\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\Users\HUAWEI\PycharmProjects\爬虫\爬（2.1）.py", line 80, in <module> json_data = web_data.json()['data'] 

说白了就是json解析报错

下面分享一下获取cookie的方法以及如何在代码中正确添加cookie。

一、获取cookie

1.在12306界面按F12打开控制台

 

2.按Ctrl+R加载数据并找到图示信息后选中双击

 3.在Headers—Request Headers中可以找到对应的Cookie （那一大串）

复制该cookie

二、在代码中添加cookie 

原代码为：

train_url = 'https://kyfw.12306.cn/otn/leftTicket/query?leftTicketDTO.train_date={}&leftTicketDTO.from_station={}&leftTicketDTO.to_station={}&purpose_codes=ADULT'.format(
    chufa_riqi, chufa_code, daoda_code)
# 处理json格式的文件
web_data = requests.get(train_url)
# 获取data
json_data = web_data.json()['data']
# 获取result
json_result = json_data['result']

改为：

train_url = 'https://kyfw.12306.cn/otn/leftTicket/query?leftTicketDTO.train_date={}&leftTicketDTO.from_station={}&leftTicketDTO.to_station={}&purpose_codes=ADULT'.format(
    chufa_riqi, chufa_code, daoda_code)
headers = {
    "Cookie": "_uab_collina=167888383948119857248769; JSESSIONID=AA1790940F27638D86C1FEFAD08A02F4; RAIL_EXPIRATION=1679216016896; RAIL_DEVICEID=hYt8pnBJE5zjFEqXY14LduH9rIy9-UN4AbMn20bE81x4rRJC2kayjzg3BWPk6opyYPWmlfOnzEP20jhVPX5USWTbKkTDd_j9zwIuo090clPJ8GcGrIKna-oMGYNgN3JW1RAmYdQaEmhlijnY9GTcTMqB0LhkeNmX; guidesStatus=off; highContrastMode=defaltMode; cursorStatus=off; _jc_save_wfdc_flag=dc; _jc_save_fromStation=%u4E0A%u6D77%2CSHH; _jc_save_toStation=%u5317%u4EAC%2CBJP; BIGipServerotn=3939959050.24610.0000; BIGipServerpassport=887619850.50215.0000; fo=mzlwpnvs58igbr4yITzRhc1XpikMJb8t9jmKDMLXZw97swpAEJ3F3MVDeCRN3qiXIVS2bSU8Cd7FnzCq3JwN8VpXhZxvZJgMvxYzcm34sDrqiDlGpqr3UqvkLTHhqgaM5tn6XKrtAfMDvl1v7bDmAtWOlOuVxMkfmckCBvSxbrk15mEuJFgyNh_iuHs; route=9036359bb8a8a461c164a04f8f50b252; _jc_save_toDate=2023-03-16; _jc_save_fromDate=2023-03-17"}
# 处理json格式的文件
web_data = requests.get(train_url, headers=headers)
# 获取data
json_data = web_data.json()['data']
# 获取result
json_result = json_data['result']

最终可获得正常输出