当我们想要了解某个地区的天气趋势,或者进行天气数据的分析时,爬取并可视化天气数据是一个有趣而且有用的项目。在这篇博客中,我将介绍如何使用Python来爬取天气数据,并对其进行数据可视化分析。我们将使用Python中的一些流行的库,如Requests、pandas和flask来构建气象数据可视化网站。
步骤一:安装必要的库
首先,确保你已经安装了以下Python库,如果没有,可以使用pip进行安装:
- requests:用于从网站获取数据。
- pandas:用于处理数据
- flask:用于构造网站微型框架
步骤二:爬取天气数据
def scraw(code):
# 发送 HTTP 请求,获取网页内容
url = f'http://www.nmc.cn/rest/weather?stationid={code}&_=1675259309000'
response = requests.get(url, headers=headers)
try:
data = json.loads(response.text)
info = data['data']
# 24小时天气情况
passed = data['data']['passedchart']
# 一天
real = data['data']['real']
# 最近七天最高低温度
tempchart = data['data']['tempchart']
# 预测
predict = data['data']['predict']['detail']
for i in passed:
humidity = i['humidity'] # 相对湿度
pressure = i['pressure'] # 空气压力
rain1h = i['rain1h'] #
rain24h = i['rain24h'] #
temperature = i['temperature'] # 温度
windDirection = i['windDirection']
windSpeed = i['windSpeed']
time = i['time']
tempDiff = i['tempDiff'] # 体感温度
csv.writer(csv_obj).writerow(
[names[inx],humidity, pressure, rain1h, rain24h, temperature, windDirection, windSpeed, time, tempDiff])
csv.writer(csv_obj2).writerow(
[names[inx],datetime.datetime.now().date(), real['weather']['airpressure'], real['weather']['feelst'],
real['weather']['humidity'], real['weather']['info'], real['weather']['rain'], real['weather']['temperature'],
real['wind']['direct'], real['wind']['power'], real['wind']['speed']])
步骤三:将数据写入数据库
将数据处理并存入数据库
def save():
# 存入数据库
DB_STRING = 'mysql+pymysql://root:786120564@127.0.0.1:3306/tianqi'
engine = create_engine(DB_STRING)
df = pd.read_csv("csv/data24h.csv")
df2 = pd.read_csv("csv/dataday.csv")
df3 = pd.read_csv("csv/tempchart.csv")
df4 = pd.read_csv("csv/predict.csv")
df5 = pd.read_csv("csv/rain24h.csv")
df6 = pd.read_csv("csv/rain1h.csv")
#删除不正常值
# 删除部分列值等于9999的行
df = df.drop('24h降雨量',axis=1)
df2 = df2[df2['体感温度'] != 9999]
df3 = df3[df3['最高温度'] != 9999]
df.to_sql('24h', con=engine, if_exists='replace',index=False)
df2.to_sql('day', con=engine, if_exists='replace',index=False)
df3.to_sql('tempchart', con=engine, if_exists='replace',index=False)
df4.to_sql('predict', con=engine, if_exists='replace',index=False)
df5.to_sql('rain24h', con=engine, if_exists='replace',index=False)
df6.to_sql('rain1h', con=engine, if_exists='replace',index=False)
步骤四:构建flask应用
from flask import Flask, render_template
app = Flask(__name__)
@app.route("/")
def index():
return render_template("index.html")
@app.route("/echart1")
def echart1():
return jsonify(array.tolist())
if __name__ == '__main__':
app.run()