Python作业8-网络爬虫

1.编程实现：利用requests爬虫库和json解析库实现疫情数据爬取。

已知：

（1）爬取网址：

url='https://api.inews.qq.com/newsqa/v1/query/inner/publish/modules/list?modules=statisGradeCityDetail,diseaseh5Shelf'

（2）爬取授权：

headers={

    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36'

}

（3）爬取解析：

json_data = response.json()['data']['diseaseh5Shelf']['areaTree'][0]['children']

2. 编程实现：利用xlrd库将以上爬取的数据（'地区'、'新增确诊'、'治愈人数'、'死亡人数'、'现存人数'、'累计确诊'）存入excel文件中。

3. 编程实现：利用xlwt库读取上述excel文件中的'地区'和'新增确诊'两列数据，利用matplotlib库折线图可视化这两种数据。

import requests
import json
import pandas as pd
import matplotlib.pyplot as plt
import xlrd
import xlwt
# 创建一个Excel
book = xlwt.Workbook(encoding='utf-8')
sheet = book.add_sheet('疫情显示')
# 进行爬虫操作
url = 'https://api.inews.qq.com/newsqa/v1/query/inner/publish/modules/list?modules=statisGradeCityDetail,diseaseh5Shelf'
headers = {
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36'
}
response = requests.get(url=url, headers=headers)
json_data = response.json()['data']['diseaseh5Shelf']['areaTree'][0]['children']
col = ('地区', '新增确诊', '治愈人数', '死亡人数', '现存人数', '累计确诊')
# 将爬取的数据放入Excel中
for i in range(6):
    sheet.write(0, i, col[i])
for i in range(len(json_data)):
    sheet.write(i+1, 0, json_data[i]['name'])
    sheet.write(i+1, 1, json_data[i]['today']['confirm'])
    sheet.write(i+1, 2, json_data[i]['total']['heal'])
    sheet.write(i+1, 3, json_data[i]['total']['dead'])
    sheet.write(i+1, 4, json_data[i]['total']['nowConfirm'])
    sheet.write(i+1, 5, json_data[i]['total']['confirm'])
book.save('疫情.xlsx')

# 读取Excel中的值
du = xlrd.open_workbook('疫情.xlsx')
sheet1 = du.sheets()[0]
list1 = sheet1.col_values(1)[1:]
list2 = sheet1.col_values(0)[1:]
len1 = len(list1)


# 作图
plt.rcParams['font.sans-serif'] = ['SimHei']
plt.figure(figsize=(20, 5))
plt.plot(range(1, len1+1), sheet1.col_values(1)[1:])
plt.xticks(range(1, len1+1), sheet1.col_values(0)[1:])
plt.xlabel("地区")
plt.ylabel("新增确诊")
plt.title("疫情显示")
plt.savefig("疫情.png")
plt.show()