爬取天气预报并对数据进行可视化分析

一、介绍

本文介绍爬虫的一个小用法，爬取天气预报

写爬虫时最好更改一下User-Agent，不然会被阻止访问。

本文用正则匹配获取天气预报的数据，并使用matplotlib进行绘图，然后进行分析天气预报的数据

二、具体代码

import requests
from bs4 import BeautifulSoup
import re
import pandas as pd
import matplotlib.pyplot as plt


url = 'https://www.tianqi.com/guangdong-guilin/28722/30/'
headers = {
    "User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0'
}
response = requests.get(url, headers=headers)
response.encoding = 'utf-8'
html = response.text
soup = BeautifulSoup(html, 'html.parser') #解析文档,html.parser为HTML解析器
weather = []
tem = []
for tag in soup.find_all('li',attrs={"class":"temp"}):
    if not bool(re.search(r'\d', tag.get_text())):
        tag_text = tag.get_text()
        weather.append(tag_text)
    else:
        tag_text = tag.get_text()
        tem.append(tag_text)
tem_str = ''.join(tem)
max_tem = re.findall(r'\d\d~', tem_str)
min_tem = re.findall(r'~\d\d', tem_str)
max_sum = 0
min_sum = 0
for i in range(len(max_tem)):
    max_tem[i] = int(max_tem[i].replace('~', ''))
    max_sum += max_tem[i]
for i in range(len(min_tem)):
    min_tem[i] = int(min_tem[i].replace('~', ''))
    min_sum += min_tem[i]

# 存储至excel表格中
# info = {'天气': weather, '温度': tem}
# dm_file = pd.DataFrame(info)
# dm_file.to_excel('weather.xlsx', sheet_name="数据分析")

# 绘制折线图
avg_max_tem = max_sum/len(max_tem)
avg_min_tem = min_sum/len(min_tem)
fig, ax = plt.subplots()
plt.title("Temperature changes in Guilin in the next 30 days")
plt.xlabel("Future days/day")
plt.ylabel("Celsius/℃")
day = []
for i in range(30):
    day.append(i)
ax.plot(day, max_tem)
ax.plot(day, min_tem)
ax.scatter(day, max_tem)
ax.scatter(day, min_tem)
ax.axhline(y=avg_max_tem, linestyle='--', color='black')
ax.axhline(y=avg_min_tem, linestyle='--', color='black')
plt.show()

三、总结

        爬虫的能力很强大，本文只是简单的运用。一定要注意不能非法运用爬虫技术，要遵纪守法。