作为一名大三的准程序员,这学期学了一门叫做脚本程序设计的课程,刚好很感兴趣,于是做了一个相关的爬虫系统,目前只能算是半成品,因为还想到了很多可以跟进的地方,但奈何知识储备不够,所以先发布一下目前的版本。
我捕获的是某气象网的天气预报
成都天气预报,成都7天天气预报,成都15天天气预报,成都天气查询
下面我一 一讲解一下我写的代码
目录
观察连接
首先,我们要先观察这个连接,他有什么独特之处。成都天气预报,成都7天天气预报,成都15天天气预报,成都天气查询
通过观察,我们发现了两个独特之处。
首先,就是weather1d,我在这里给大家解释一下,其中的‘1d’代表当天的天气情况,之后还会用到‘15d’,它代表的便是未来14天的天气情况
然后就是101270101这串数字编码,这其实是成都的城市编码,我当时看了一下某气象网的源文件,并没有找到所以城市的编码,无奈只好问问度娘,下面我把城市编码放上来
北京:101010100都江堰:101270111朝阳:101010300顺义:101010400怀柔:101010500通州:101010600昌平:101010700延庆:101010800丰台:101010900石景山:101011000大兴:101011100房山:101011200密云:101011300门头沟:101011400平谷:101011500八达岭:101011600佛爷顶:101011700汤河口:101011800密云上甸子:101011900斋堂:101012000霞云岭:101012100北京城区:101012200海淀:101010200天津:101030100宝坻:101030300东丽:101030400西青:101030500北辰:101030600蓟县:101031400汉沽:101030800静海:101030900津南:101031000塘沽:101031100大港:101031200武清:101030200宁河:101030700上海:101020100宝山:101020300嘉定:101020500南汇:101020600浦东:101021300青浦:101020800松江:101020900奉贤:101021000崇明:101021100徐家汇:101021200闵行:101020200金山:101020700石家庄:101090101张家口:101090301承德:101090402唐山:101090501秦皇岛:101091101沧州:101090701衡水:101090801邢台:101090901邯郸:101091001保定:101090201廊坊:101090601郑州:101180101新乡:101180301许昌:101180401平顶山:101180501信阳:101180601南阳:101180701开封:101180801洛阳:101180901商丘:101181001焦作:101181101鹤壁:101181201濮阳:101181301周口:101181401漯河:101181501驻马店:101181601三门峡:101181701济源:101181801安阳:101180201合肥:101220101芜湖:101220301淮南:101220401马鞍山:101220501安庆:101220601宿州:101220701阜阳:101220801亳州:101220901黄山:101221001滁州:101221101淮北:101221201铜陵:101221301宣城:101221401六安:101221501巢湖:101221601池州:101221701蚌埠:101220201杭州:101210101舟山:101211101湖州:101210201嘉兴:101210301金华:101210901绍兴:101210501台州:101210601温州:101210701丽水:101210801衢州:101211001宁波:101210401重庆:101040100合川:101040300南川:101040400江津:101040500万盛:101040600渝北:101040700北碚:101040800巴南:101040900长寿:101041000黔江:101041100万州天城:101041200万州龙宝:101041300涪陵:101041400开县:101041500城口:101041600云阳:101041700巫溪:101041800奉节:101041900巫山:101042000潼南:101042100垫江:101042200梁平:101042300忠县:101042400石柱:101042500大足:101042600荣昌:101042700铜梁:101042800璧山:101042900丰都:101043000武隆:101043100彭水:101043200綦江:101043300酉阳:101043400秀山:101043600沙坪坝:101043700永川:101040200福州:101230101泉州:101230501漳州:101230601龙岩:101230701晋江:101230509南平:101230901厦门:101230201宁德:101230301莆田:101230401三明:101230801兰州:101160101平凉:101160301庆阳:101160401武威:101160501金昌:101160601嘉峪关:101161401酒泉:101160801天水:101160901武都:101161001临夏:101161101合作:101161201白银:101161301定西:101160201张掖:101160701广州:101280101惠州:101280301梅州:101280401汕头:101280501深圳:101280601珠海:101280701佛山:101280800肇庆:101280901湛江:101281001江门:101281101河源:101281201清远:101281301云浮:101281401潮州:101281501东莞:101281601中山:101281701阳江:101281801揭阳:101281901茂名:101282001汕尾:101282101韶关:101280201南宁:101300101柳州:101300301来宾:101300401桂林:101300501梧州:101300601防城港:101301401贵港:101300801玉林:101300901百色:101301001钦州:101301101河池:101301201北海:101301301崇左:101300201贺州:101300701贵阳:101260101安顺:101260301都匀:101260401兴义:101260906铜仁:101260601毕节:101260701六盘水:101260801遵义:101260201凯里:101260501昆明:101290101红河:101290301文山:101290601玉溪:101290701楚雄:101290801普洱:101290901昭通:101291001临沧:101291101怒江:101291201香格里拉:101291301丽江:101291401德宏:101291501景洪:101291601大理:101290201曲靖:101290401保山:101290501呼和浩特:101080101乌海:101080301集宁:101080401通辽:101080501阿拉善左旗:101081201鄂尔多斯:101080701临河:101080801锡林浩特:101080901呼伦贝尔:101081000乌兰浩特:101081101包头:101080201赤峰:101080601南昌:101240101上饶:101240301抚州:101240401宜春:101240501鹰潭:101241101赣州:101240701景德镇:101240801萍乡:101240901新余:101241001九江:101240201吉安:101240601武汉:101200101黄冈:101200501荆州:101200801宜昌:101200901恩施:101201001十堰:101201101神农架:101201201随州:101201301荆门:101201401天门:101201501仙桃:101201601潜江:101201701襄樊:101200201鄂州:101200301孝感:101200401黄石:101200601咸宁:101200701成都:101270101自贡:101270301绵阳:101270401南充:101270501达州:101270601遂宁:101270701广安:101270801巴中:101270901泸州:101271001宜宾:101271101内江:101271201资阳:101271301乐山:101271401眉山:101271501凉山:101271601雅安:101271701甘孜:101271801阿坝:101271901德阳:101272001广元:101272101攀枝花:101270201银川:101170101中卫:101170501固原:101170401石嘴山:101170201吴忠:101170301西宁:101150101黄南:101150301海北:101150801果洛:101150501玉树:101150601海西:101150701海东:101150201海南:101150401济南:101120101潍坊:101120601临沂:101120901菏泽:101121001滨州:101121101东营:101121201威海:101121301枣庄:101121401日照:101121501莱芜:101121601聊城:101121701青岛:101120201淄博:101120301德州:101120401烟台:101120501济宁:101120701泰安:101120801西安:101110101延安:101110300榆林:101110401铜川:101111001商洛:101110601安康:101110701汉中:101110801宝鸡:101110901咸阳:101110200渭南:101110501太原:101100101临汾:101100701运城:101100801朔州:101100901忻州:101101001长治:101100501大同:101100201阳泉:101100301晋中:101100401晋城:101100601吕梁:101101100乌鲁木齐:101130101石河子:101130301昌吉:101130401吐鲁番:101130501库尔勒:101130601阿拉尔:101130701阿克苏:101130801喀什:101130901伊宁:101131001塔城:101131101哈密:101131201和田:101131301阿勒泰:101131401阿图什:101131501博乐:101131601克拉玛依:101130201拉萨:101140101山南:101140301阿里:101140701昌都:101140501那曲:101140601日喀则:101140201林芝:101140401台北县:101340101高雄:101340201台中:101340401海口:101310101三亚:101310201东方:101310202临高:101310203澄迈:101310204儋州:101310205昌江:101310206白沙:101310207琼中:101310208定安:101310209屯昌:101310210琼海:101310211文昌:101310212保亭:101310214万宁:101310215陵水:101310216西沙:101310217南沙岛:101310220乐东:101310221五指山:101310222琼山:101310102长沙:101250101株洲:101250301衡阳:101250401郴州:101250501常德:101250601益阳:101250700娄底:101250801邵阳:101250901岳阳:101251001张家界:101251101怀化:101251201黔阳:101251301永州:101251401吉首:101251501湘潭:101250201南京:101190101镇江:101190301苏州:101190401南通:101190501扬州:101190601宿迁:101191301徐州:101190801淮安:101190901连云港:101191001常州:101191101泰州:101191201无锡:101190201盐城:101190701哈尔滨:101050101牡丹江:101050301佳木斯:101050401绥化:101050501黑河:101050601双鸭山:101051301伊春:101050801大庆:101050901七台河:101051002鸡西:101051101鹤岗:101051201齐齐哈尔:101050201大兴安岭:101050701长春:101060101延吉:101060301四平:101060401白山:101060901白城:101060601辽源:101060701松原:101060801吉林:101060201通化:101060501沈阳:101070101鞍山:101070301抚顺:101070401本溪:101070501丹东:101070601葫芦岛:101071401营口:101070801阜新:101070901辽阳:101071001铁岭:101071101朝阳:101071201盘锦:101071301大连:101070201锦州:101070701
判断城市是否存在
既然我们需要按城市查询天气预报,那么我们就需要判断一下该城市是否存在,函数如下
"""判断城市是否存在"""
def Judge(city_name):
city = re.compile(r'(?<={}:)\d*'.format(city_name))
str1 = open('城市代码.txt', 'r')
new_str = str1.readlines()
try:
city_coding = city.search(str(new_str)).group(0)
city_coding = city_name
return city_name
except:
return "你这儿是那个山卡卡"
# print(city_coding)
这里用了正则,如果不熟悉正则的小伙伴,可以先在菜鸟教程在线正则编译器进行正则的审核工作,这里也推荐一本关于正则的书《正则表达式必知必会》,这本书消化了的话,正则简直就是信手拈来。
获取url
接下来便是通过我们所需要查询的城市,获取相应的url,相应函数如下:
"""获取url"""
def getURL(city_name):
base_url = 'http://www.weather.com.cn/weather/{}.shtml'
city = re.compile(r'(?<={}:)\d*'.format(city_name))
str1 = open('城市代码.txt', 'r')
# print(str.readlines())
new_str = str1.readlines()
# print(new_str)
# print(city.search(str).group(0))
city_coding = city.search(str(new_str)).group(0)
# print(city_coding)
#
new_url = base_url.format(city_coding)
# print(new_url)
return new_url
def getURL2(city_name):
base_url = 'http://www.weather.com.cn/weather15d/{}.shtml'
city = re.compile(r'(?<={}:)\d*'.format(city_name))
str1 = open('城市代码.txt', 'r')
# print(str.readlines())
new_str = str1.readlines()
# print(new_str)
# print(city.search(str).group(0))
city_coding = city.search(str(new_str)).group(0)
# print(city_coding)
#
new_url = base_url.format(city_coding)
# print(new_url)
return new_url
这里说一句,以前看着大家用format不以为意,结果自己写了这个才知道format才是yyds。
这里面的open('城市代码.txt', 'r'),中的'城市代码.txt',需要大家自行创建一个txt文档,然后把本文开头发的城市代码全部复制进去才可运行
请求获取网页内容
我们获取了url之后,就该通过相应的url请求网页内容了,
相关函数如下
"""请求获得网页内容"""
def getHTMLtext(url,city_name):
"""请求获得网页内容"""
try:
headers = {
"User-Agent": UserAgent().random,
}
r = requests.get(url, timeout=30, headers=headers)
r.raise_for_status()
r.encoding = r.apparent_encoding
print("某气象网{}24小时天气访问成功".format(city_name))
return r.text
except:
print("某气象网{}24小时天气访问失败".format(city_name))
return " "
def getHTMLtext2(url,city_name):
"""请求获得网页内容"""
try:
headers = {
"User-Agent": UserAgent().random,
}
r = requests.get(url, timeout=30, headers=headers)
r.raise_for_status()
r.encoding = r.apparent_encoding
print("某气象网{}15天天气访问成功".format(city_name))
return r.text
except:
print("某气象网{}15天天气访问失败".format(city_name))
return " "
这里用了一组try/except,本人不是太喜欢看到红色报错,所以如果访问失败,会直接返回print中的内容 ,前文也有一组,同样的道理。
处理信息
处理有用的数据,并暂时保存起来,
相关代码如下
"""处理得到有用信息保存数据文件"""
def get_content(html):
final = [] # 初始化一个列表保存数据
bs = BeautifulSoup(html, "html.parser") # 创建BeautifulSoup对象
body = bs.body
data = body.find('div', {'id': '7d'}) # 找到div标签且id = 7d
# 下面捕获当天的数据
data2 = body.find_all('div', {'class': 'left-div'})
text = data2[2].find('script').string
text = text[text.index('=') + 1:-2] # 移除改var data=将其变为json数据
jd = json.loads(text)
dayone = jd['od']['od2'] # 找到当天的数据
final_day = [] # 存放当天的数据
count = 0
for i in dayone:
temp = []
if count <= 23:
temp.append(i['od21']) # 添加时间
temp.append(i['od22']) # 添加当前时刻温度
temp.append(i['od24']) # 添加当前时刻风力方向
temp.append(i['od25']) # 添加当前时刻风级
temp.append(i['od26']) # 添加当前时刻降水量
temp.append(i['od27']) # 添加当前时刻相对湿度
temp.append(i['od28']) # 添加当前时刻控制质量
# print(temp)
final_day.append(temp)
count = count + 1
return final_day
def get_content1_7(html):
# 下面捕获7天的数据
final = [] # 初始化一个列表保存数据
bs = BeautifulSoup(html, "html.parser") # 创建BeautifulSoup对象
body = bs.body
data = body.find('div', {'id': '7d'}) # 找到div标签且id = 7d
ul = data.find('ul') # 找到所有的ul标签
li = ul.find_all('li') # 找到左右的li标签
i = 0 # 控制捕获的天数
for day in li: # 遍历找到的每一个li
if 7 > i > 0:
temp = [] # 临时存放每天的数据
date = day.find('h1').string # 得到日期
date = date[0:date.index('日')] # 取出日期号
temp.append(date)
inf = day.find_all('p') # 找出li下面的p标签,提取第一个p标签的值,即天气
temp.append(inf[0].string)
tem_low = inf[1].find('i').string # 找到最低气温
if inf[1].find('span') is None: # 天气预报可能没有最高气温
tem_high = None
else:
tem_high = inf[1].find('span').string # 找到最高气温
temp.append(tem_low[:-1])
if tem_high[-1] == '℃':
temp.append(tem_high[:-1])
else:
temp.append(tem_high)
wind = inf[2].find_all('span') # 找到风向
for j in wind:
temp.append(j['title'])
wind_scale = inf[2].find('i').string # 找到风级
index1 = wind_scale.index('级')
temp.append(int(wind_scale[index1 - 1:index1]))
final.append(temp)
i = i + 1
return final
def get_content8_14(html):
"""处理得到有用信息保存数据文件"""
final = [] # 初始化一个列表保存数据
bs = BeautifulSoup(html, "html.parser") # 创建BeautifulSoup对象
body = bs.body
data = body.find('div', {'id': '15d'}) # 找到div标签且id = 15d
ul = data.find('ul') # 找到所有的ul标签
li = ul.find_all('li') # 找到左右的li标签
# final = []
i = 0 # 控制捕获的天数
for day in li: # 遍历找到的每一个li
if i < 8:
temp = [] # 临时存放每天的数据
date = day.find('span', {'class': 'time'}).string # 得到日期
date = date[date.index('(') + 1:-2] # 取出日期号
temp.append(date)
weather = day.find('span', {'class': 'wea'}).string # 找到天气
temp.append(weather)
tem = day.find('span', {'class': 'tem'}).text # 找到温度
temp.append(tem[tem.index('/') + 1:-1]) # 找到最低气温
temp.append(tem[:tem.index('/') - 1]) # 找到最高气温
wind = day.find('span', {'class': 'wind'}).string # 找到风向
if '转' in wind: # 如果有风向变化
temp.append(wind[:wind.index('转')])
temp.append(wind[wind.index('转') + 1:])
else: # 如果没有风向变化,前后风向一致
temp.append(wind)
temp.append(wind)
wind_scale = day.find('span', {'class': 'wind1'}).string # 找到风级
index1 = wind_scale.index('级')
temp.append(int(wind_scale[index1 - 1:index1]))
final.append(temp)
return final
一开始我用的是XPath,但是这个某气象网中网页源代码,和网页代码不一样,要用tbody,这个问题当时卡了我好几天。 最后实在不想被恶心下去,又学了一下beautifusoup,用beautifulsoup解决了这里的问题。
保存csv
保存数据到csv文件,
、
相关代码如下
"""保存为csv文件"""
def write_to_csv(file_name, data, day=14):
"""保存为csv文件"""
with open(file_name, 'w', errors='ignore', newline='') as f:
if day == 14:
header = ['日期', '天气', '最低气温', '最高气温', '风向1', '风向2', '风级']
else:
header = ['小时', '温度', '风力方向', '风级', '降水量', '相对湿度', '空气质量']
f_csv = csv.writer(f)
f_csv.writerow(header)
f_csv.writerows(data)
由于我们访问了两个网页,访问所暂存的数据也不一样,所以我们需要判断一下是24小时的数据还是14天的数据
创建数据库表单
"""创建表"""
def Creat_Table(city_name):
config = {
"host": "localhost",
"port": "3306",
"user": "root",
"password": "628",
"database": "python"
}
con = mysql.connector.connect(**config)
cursor = con.cursor()
cursor.execute('DROP TABLE IF EXISTS {}weather'.format(city_name))
sql_set = 'CREATE TABLE IF NOT EXISTS {}weather(Time INT,Temperature INT ,' \
'wind_direction VARCHAR(255), wind INT, rainfall INT, Humidity INT)'.format(city_name)
try:
cursor.execute(sql_set)
con.commit()
print('创建{}24小时天气情况数据表成功'.format(city_name))
except:
con.rollback()
print("创建{}24小时天气情况数据表失败".format(city_name))
con.close()
def Creat_Table14(city_name):
config = {
"host": "localhost",
"port": "3306",
"user": "root",
"password": "628",
"database": "python"
}
con = mysql.connector.connect(**config)
cursor = con.cursor()
cursor.execute('DROP TABLE IF EXISTS {}weather14d'.format(city_name))
sql_set = 'CREATE TABLE IF NOT EXISTS {}weather14d(Date INT,Weather VARCHAR(255) ,' \
'Min_temperature INT, Max_temperature INT, wind_direction_1 VARCHAR(255), wind_direction_2 VARCHAR(255), wind_level INT)'.format(city_name)
try:
cursor.execute(sql_set)
con.commit()
print('创建{}15天天气情况数据表成功'.format(city_name))
except:
con.rollback()
print('创建{}15天天气情况数据表失败'.format(city_name))
con.close()
如果大家需要copy的话,记得改这儿的数据库相应的值噢
存储数据到数据库
相关代码如下
"""在表中插入数据"""
def insert_data1(city_name):
config = {
"host": "localhost",
"port": "3306",
"user": "root",
"password": "628",
"database": "python"
}
con = mysql.connector.connect(**config)
cursor = con.cursor()
data1 = pd.read_csv('天气预报-{}24小时天气预报.csv'.format(city_name), encoding='gb2312')
for i in range(data1.shape[0]):
Time = list(data1['小时'])[i]
Temperature = list(data1['温度'])[i]
wind_direction = list(data1['风力方向'])[i]
wind = list(data1['风级'])[i]
rainfall = list(data1['降水量'])[i]
Humidity = list(data1['相对湿度'])[i]
# Time = list(data1['空气质量'])[1]
data = (Time, Temperature, wind_direction, wind, rainfall, Humidity)
sql = "INSERT INTO {}weather(Time, Temperature, wind_direction, wind, rainfall, Humidity) values ".format(city_name) + str(data) + ";"
try:
cursor.execute(sql) #执行sql语句
con.commit() #连接提交
except:
con.rollback()
print("插入{}24小时天气情况数据失败".format(city_name))
print('插入{}24小时天气情况数据成功'.format(city_name))
cursor.close()
con.close()
# sql = "INSERT INTO weather(Time, Temperature, wind_direction, wind, rainfall, Humidity) values('1', '1', 's', '1', '1', '1');"
# try:
# cursor.execute(sql)
# cursor.execute(sql)
# con.commit()
# print('插入数据成功')
# except:
# con.rollback()
# print("插入数据失败")
# con.close()
def insert_data14(city_name):
config = {
"host": "localhost",
"port": "3306",
"user": "root",
"password": "628",
"database": "python"
}
con = mysql.connector.connect(**config)
cursor = con.cursor()
data1 = pd.read_csv('天气预报-{}15天天气预报.csv'.format(city_name), encoding='gb2312')
for i in range(data1.shape[0]):
Date = list(data1['日期'])[i]
Weather = list(data1['天气'])[i]
Min_temperature = list(data1['最低气温'])[i]
Max_temperature = list(data1['最高气温'])[i]
wind_direction_1 = list(data1['风向1'])[i]
wind_direction_2 = list(data1['风向2'])[i]
wind_level = list(data1['风级'])[i]
# Time = list(data1['空气质量'])[1]
data = (Date, Weather, Min_temperature, Max_temperature, wind_direction_1, wind_direction_2,wind_level)
sql = "INSERT INTO {}weather14d(Date, Weather, Min_temperature, Max_temperature, wind_direction_1, wind_direction_2,wind_level) values ".format(city_name) + str(data) + ";"
try:
cursor.execute(sql) #执行sql语句
con.commit() #连接提交
except:
con.rollback()
print("插入{}半个月天气情况数据失败")
print('插入{}15天天气情况数据成功'.format(city_name))
cursor.close()
con.close()
这个地方也给我卡了小半天,结果你们猜哪儿出问题了,最后检查出来是保存14天数据的时候,写sql语句时表单名字写错了。。。。。真的程序员的bug八成都是拼写错误
数据捕获和数据存储的事情我们都已经完成了,接下来就需要数据可视化了,我用的是matplotlib,首先捕获天气预报肯定要把每天的天气温度放首位,那么接下来我们继续
绘制当天温度曲线
def tem_curve(data,city_name ):
"""温度曲线绘制"""
# plt.style.use('seaborn')
hour = list(data['小时'])
# print(hour)
tem = list(data['温度'])
for i in range(0, 24):
if math.isnan(tem[i]):
tem[i] = tem[i - 1]
tem_ave = sum(tem) / 24 # 求平均温度
tem_max = max(tem)
tem_max_hour = hour[tem.index(tem_max)] # 求最高温度
tem_min = min(tem)
tem_min_hour = hour[tem.index(tem_min)] # 求最低温度
x = []
y = []
for i in range(0, 24):
x.append(i)
y.append(tem[hour.index(i)])
plt.style.use('seaborn')
matplotlib.rcParams['font.sans-serif'] = ['SimHei']
matplotlib.rcParams['axes.unicode_minus'] = False
plt.figure(1)
plt.plot(x, y, color='red', label='温度',marker = '^') # 画出温度曲线
plt.scatter(x, y, color='red') # 点出每个时刻的温度点
plt.plot([0, 24], [tem_ave, tem_ave], c='blue', linestyle='--', label='平均温度') # 画出平均温度虚线
plt.text(tem_max_hour + 0.15, tem_max + 0.15, str(tem_max), ha='center', va='bottom', fontsize=10.5) # 标出最高温度
plt.text(tem_min_hour + 0.15, tem_min + 0.15, str(tem_min), ha='center', va='bottom', fontsize=10.5) # 标出最低温度
plt.xticks(x)
plt.legend()
plt.title('一天温度变化曲线图')
plt.xlabel('时间/h')
plt.ylabel('摄氏度/℃')
plt.savefig('image/{}24小时温度曲线.png'.format(city_name), bbox_inches='tight')
plt.show()
这段到后面的知识都是用了有关爬虫可视化的知识,这里包括但不限于matplotlib,当时我学这个是在《Python从入门到实践》这本书的可视化部分学的,这是图灵出版的书,真的推荐大家去看看,特别适合新手入门
绘制15天温度曲线
"""15天温度曲线绘制"""
def tem_curve14(data,city_name):
date = list(data['日期'])
tem_low = list(data['最低气温'])
tem_high = list(data['最高气温'])
for i in range(0, 14):
if math.isnan(tem_low[i]):
tem_low[i] = tem_low[i - 1]
if math.isnan(tem_high[i]):
tem_high[i] = tem_high[i - 1]
tem_high_ave = sum(tem_high) / 14 # 求平均高温
tem_low_ave = sum(tem_low) / 14 # 求平均低温
tem_max = max(tem_high)
tem_max_date = tem_high.index(tem_max) # 求最高温度
tem_min = min(tem_low)
tem_min_date = tem_low.index(tem_min) # 求最低温度
x = range(1, 15)
plt.figure(1)
plt.plot(x, tem_high, color='red', label='高温',marker = '^') # 画出高温度曲线
plt.scatter(x, tem_high, color='red') # 点出每个时刻的温度点
plt.plot(x, tem_low, color='blue', label='低温',marker = '^') # 画出低温度曲线
plt.scatter(x, tem_low, color='blue') # 点出每个时刻的温度点
plt.plot([1, 15], [tem_high_ave, tem_high_ave], c='black', linestyle='--') # 画出平均温度虚线
plt.plot([1, 15], [tem_low_ave, tem_low_ave], c='black', linestyle='--') # 画出平均温度虚线
plt.legend()
plt.text(tem_max_date + 0.15, tem_max + 0.15, str(tem_max), ha='center', va='bottom', fontsize=10.5) # 标出最高温度
plt.text(tem_min_date + 0.15, tem_min + 0.15, str(tem_min), ha='center', va='bottom', fontsize=10.5) # 标出最低温度
plt.xticks(x)
plt.title('未来14天高温低温变化曲线图')
plt.xlabel('未来天数/天')
plt.ylabel('摄氏度/℃')
plt.savefig('image/{}15天温度曲线.png'.format(city_name), bbox_inches='tight')
plt.show()
相对湿度曲线绘制
"""相对湿度曲线绘制"""
def hum_curve(data,city_name):
"""相对湿度曲线绘制"""
hour = list(data['小时'])
hum = list(data['相对湿度'])
for i in range(0, 24):
if math.isnan(hum[i]) == True:
hum[i] = hum[i - 1]
hum_ave = sum(hum) / 24 # 求平均相对湿度
hum_max = max(hum)
hum_max_hour = hour[hum.index(hum_max)] # 求最高相对湿度
hum_min = min(hum)
hum_min_hour = hour[hum.index(hum_min)] # 求最低相对湿度
x = []
y = []
for i in range(0, 24):
x.append(i)
y.append(hum[hour.index(i)])
plt.figure(2)
plt.plot(x, y, color='blue', label='相对湿度',marker = '^') # 画出相对湿度曲线
plt.scatter(x, y, color='blue') # 点出每个时刻的相对湿度
plt.plot([0, 24], [hum_ave, hum_ave], c='red', linestyle='--', label='平均相对湿度',marker = '*') # 画出平均相对湿度虚线
plt.text(hum_max_hour + 0.15, hum_max + 0.15, str(hum_max), ha='center', va='bottom', fontsize=10.5) # 标出最高相对湿度
plt.text(hum_min_hour + 0.15, hum_min + 0.15, str(hum_min), ha='center', va='bottom', fontsize=10.5) # 标出最低相对湿度
plt.xticks(x)
plt.legend()
plt.title('一天相对湿度变化曲线图')
plt.xlabel('时间/h')
plt.ylabel('百分比/%')
plt.savefig('image/{}相对湿度曲线.png'.format(city_name), bbox_inches='tight')
plt.show()
空气质量曲线绘制
"""空气质量曲线绘制"""
def air_curve(data,city_name):
"""空气质量曲线绘制"""
hour = list(data['小时'])
air = list(data['空气质量'])
# print(type(air[0]))
for i in range(0, 24):
if math.isnan(air[i]) == True:
air[i] = air[i - 1]
air_ave = sum(air) / 24 # 求平均空气质量
air_max = max(air)
air_max_hour = hour[air.index(air_max)] # 求最高空气质量
air_min = min(air)
air_min_hour = hour[air.index(air_min)] # 求最低空气质量
x = []
y = []
for i in range(0, 24):
x.append(i)
y.append(air[hour.index(i)])
plt.figure(3)
for i in range(0, 24):
if y[i] <= 50:
plt.bar(x[i], y[i], color='lightgreen', width=0.7) # 1等级
elif y[i] <= 100:
plt.bar(x[i], y[i], color='wheat', width=0.7) # 2等级
elif y[i] <= 150:
plt.bar(x[i], y[i], color='orange', width=0.7) # 3等级
elif y[i] <= 200:
plt.bar(x[i], y[i], color='orangered', width=0.7) # 4等级
elif y[i] <= 300:
plt.bar(x[i], y[i], color='darkviolet', width=0.7) # 5等级
elif y[i] > 300:
plt.bar(x[i], y[i], color='maroon', width=0.7) # 6等级
plt.plot([0, 24], [air_ave, air_ave], c='black', linestyle='--') # 画出平均空气质量虚线
plt.text(air_max_hour + 0.15, air_max + 0.15, str(air_max), ha='center', va='bottom', fontsize=10.5) # 标出最高空气质量
plt.text(air_min_hour + 0.15, air_min + 0.15, str(air_min), ha='center', va='bottom', fontsize=10.5) # 标出最低空气质量
plt.xticks(x)
plt.title('一天空气质量变化曲线图')
plt.xlabel('时间/h')
plt.ylabel('空气质量指数AQI')
plt.savefig('image/{}空气质量曲线.png'.format(city_name), bbox_inches='tight')
plt.show()
风向雷达图绘制
"""改变风向"""
def change_wind(wind):
for i in range(0, 14):
if wind[i] == "北风":
wind[i] = 90
elif wind[i] == "南风":
wind[i] = 270
elif wind[i] == "西风":
wind[i] = 180
elif wind[i] == "东风":
wind[i] = 360
elif wind[i] == "东北风":
wind[i] = 45
elif wind[i] == "西北风":
wind[i] = 135
elif wind[i] == "西南风":
wind[i] = 225
elif wind[i] == "东南风":
wind[i] = 315
return wind
"""风向雷达图"""
def wind_radar(data,city_name):
wind1 = list(data['风向1'])
wind2 = list(data['风向2'])
wind_speed = list(data['风级'])
wind1 = change_wind(wind1)
wind2 = change_wind(wind2)
degs = np.arange(45, 361, 45)
temp = []
for deg in degs:
speed = []
# 获取 wind_deg 在指定范围的风速平均值数据
for i in range(0, 14):
if wind1[i] == deg:
speed.append(wind_speed[i])
if wind2[i] == deg:
speed.append(wind_speed[i])
if len(speed) == 0:
temp.append(0)
else:
temp.append(sum(speed) / len(speed))
# print(temp)
N = 8
theta = np.arange(0. + np.pi / 8, 2 * np.pi + np.pi / 8, 2 * np.pi / 8)
# 数据极径
radii = np.array(temp)
# 绘制极区图坐标系
plt.axes(polar=True)
# 定义每个扇区的RGB值(R,G,B),x越大,对应的颜色越接近蓝色
colors = [(1 - x / max(temp), 1 - x / max(temp), 0.6) for x in radii]
plt.bar(theta, radii, width=(2 * np.pi / N), bottom=0.0, color=colors)
plt.title('未来14天风级图', x=0.2, fontsize=20)
plt.savefig('image/{}风向曲线.png'.format(city_name), bbox_inches='tight')
plt.show()
绘制天气饼图
"""绘制天气饼图"""
def weather_pie(data,city_name):
weather = list(data['天气'])
dic_wea = {}
for i in range(0, 14):
if weather[i] in dic_wea.keys():
dic_wea[weather[i]] += 1
else:
dic_wea[weather[i]] = 1
# print(dic_wea)
explode = [0.01] * len(dic_wea.keys())
color = ['lightskyblue', 'silver', 'yellow', 'salmon', 'grey', 'lime', 'gold', 'red', 'green', 'pink']
plt.pie(dic_wea.values(), explode=explode, labels=dic_wea.keys(), autopct='%1.1f%%', colors=color)
plt.title('未来14天气候分布饼图')
plt.savefig('image/{}天气饼图.png'.format(city_name), bbox_inches='tight')
plt.show()
main主函数
def main():
"""主函数"""
city_name = input("请输入要查询的城市:")
judge = Judge(city_name)
while judge == "你这儿是那个山卡卡":
print("你勒儿是那个山卡卡,我查不到,你换个塔塔诶")
city_name = input("请输入要查询的城市:")
judge = Judge(city_name)
print(judge)
if judge == city_name:
url_1 = getURL(city_name) # 7天天气某天气网
# print(url_1)
url_2 = getURL2(city_name)
html1 = getHTMLtext(url_1,city_name)
data1 = get_content(html1) # 获得1-7天和当天的数据
data1_7 = get_content1_7(html1)
html2 = getHTMLtext2(url_2,city_name)
data8_14 = get_content8_14(html2)
data14 = data1_7 + data8_14
write_to_csv('天气预报-{}24小时天气预报.csv'.format(city_name), data1, 1)
write_to_csv('天气预报-{}15天天气预报.csv'.format(city_name), data14, 14)
Creat_Table(city_name)
Creat_Table14(city_name)
insert_data1(city_name)
insert_data14(city_name)
plt.style.use('seaborn')
matplotlib.rcParams['font.sans-serif'] = ['SimHei'] # 解决中文显示问题
matplotlib.rcParams['axes.unicode_minus'] = False # 解决负号显示问题
data1 = pd.read_csv('天气预报-{}24小时天气预报.csv'.format(city_name), encoding='gb2312')
data14 = pd.read_csv('天气预报-{}15天天气预报.csv'.format(city_name), encoding='gb2312')
# tem_curve(data1,city_name) #24小时温度曲线绘制
# hum_curve(data1,city_name) #相对湿度曲线绘制
# air_curve(data1,city_name) #空气质量曲线绘制
# corr_tem_hum(data1,city_name) #温湿度相关性分析
# tem_curve14(data14,city_name) #15天温度曲线绘制
# wind_radar(data14,city_name) #风向雷达图
# weather_pie(data14,city_name) #绘制天气饼图
while True:
num = int(input("请输入要查询的图表(输入1查询24小时温度曲线绘制、输入2查询相对湿度曲线绘制、"
"输入3查询空气质量曲线绘制、输入4查询15天温度曲线、输入5风向雷达图、"
"输入6天气饼图):"))
if num == 1:
tem_curve(data1,city_name) #24小时温度曲线绘制
elif num == 2:
hum_curve(data1,city_name) #相对湿度曲线绘制
elif num == 3:
air_curve(data1,city_name) #空气质量曲线绘制
elif num == 4:
tem_curve14(data14, city_name) # 15天温度曲线绘制
elif num == 5:
wind_radar(data14, city_name) # 风向雷达图
elif num == 6:
weather_pie(data14, city_name) # 绘制天气饼图
else:
print("查询结束...")
return False
第一个whlie这儿做了一个判断,如果搜索的城市不存在,会继续让你进行搜索,如果存在,就会往下继续运行。
第二个while是做了个简单的查询,这个思路来自老师开课的第一课给我们的点名系统,对应的数字有对应的绘图 ,输入以外的数字将弹出循环,程序结束
全部代码展示
import requests
from fake_useragent import UserAgent
import json
from bs4 import BeautifulSoup
import csv
import pandas as pd
import matplotlib.pyplot as plt
import math
import mysql.connector
import re
import numpy as np
import matplotlib
"""判断城市是否存在"""
def Judge(city_name):
city = re.compile(r'(?<={}:)\d*'.format(city_name))
str1 = open('城市代码.txt', 'r')
new_str = str1.readlines()
try:
city_coding = city.search(str(new_str)).group(0)
city_coding = city_name
return city_name
except:
return "你这儿是那个山卡卡"
# print(city_coding)
"""获取url"""
def getURL(city_name):
base_url = 'http://www.weather.com.cn/weather/{}.shtml'
city = re.compile(r'(?<={}:)\d*'.format(city_name))
str1 = open('城市代码.txt', 'r')
# print(str.readlines())
new_str = str1.readlines()
# print(new_str)
# print(city.search(str).group(0))
city_coding = city.search(str(new_str)).group(0)
# print(city_coding)
#
new_url = base_url.format(city_coding)
# print(new_url)
return new_url
def getURL2(city_name):
base_url = 'http://www.weather.com.cn/weather15d/{}.shtml'
city = re.compile(r'(?<={}:)\d*'.format(city_name))
str1 = open('城市代码.txt', 'r')
# print(str.readlines())
new_str = str1.readlines()
# print(new_str)
# print(city.search(str).group(0))
city_coding = city.search(str(new_str)).group(0)
# print(city_coding)
#
new_url = base_url.format(city_coding)
# print(new_url)
return new_url
"""请求获得网页内容"""
def getHTMLtext(url,city_name):
"""请求获得网页内容"""
try:
headers = {
"User-Agent": UserAgent().random,
}
r = requests.get(url, timeout=30, headers=headers)
r.raise_for_status()
r.encoding = r.apparent_encoding
print("某气象网{}24小时天气访问成功".format(city_name))
return r.text
except:
print("某气象网{}24小时天气访问失败".format(city_name))
return " "
def getHTMLtext2(url,city_name):
"""请求获得网页内容"""
try:
headers = {
"User-Agent": UserAgent().random,
}
r = requests.get(url, timeout=30, headers=headers)
r.raise_for_status()
r.encoding = r.apparent_encoding
print("某气象网{}15天天气访问成功".format(city_name))
return r.text
except:
print("某气象网{}15天天气访问失败".format(city_name))
return " "
"""处理得到有用信息保存数据文件"""
def get_content(html):
final = [] # 初始化一个列表保存数据
bs = BeautifulSoup(html, "html.parser") # 创建BeautifulSoup对象
body = bs.body
data = body.find('div', {'id': '7d'}) # 找到div标签且id = 7d
# 下面捕获当天的数据
data2 = body.find_all('div', {'class': 'left-div'})
text = data2[2].find('script').string
text = text[text.index('=') + 1:-2] # 移除改var data=将其变为json数据
jd = json.loads(text)
dayone = jd['od']['od2'] # 找到当天的数据
final_day = [] # 存放当天的数据
count = 0
for i in dayone:
temp = []
if count <= 23:
temp.append(i['od21']) # 添加时间
temp.append(i['od22']) # 添加当前时刻温度
temp.append(i['od24']) # 添加当前时刻风力方向
temp.append(i['od25']) # 添加当前时刻风级
temp.append(i['od26']) # 添加当前时刻降水量
temp.append(i['od27']) # 添加当前时刻相对湿度
temp.append(i['od28']) # 添加当前时刻控制质量
# print(temp)
final_day.append(temp)
count = count + 1
return final_day
def get_content1_7(html):
# 下面捕获7天的数据
final = [] # 初始化一个列表保存数据
bs = BeautifulSoup(html, "html.parser") # 创建BeautifulSoup对象
body = bs.body
data = body.find('div', {'id': '7d'}) # 找到div标签且id = 7d
ul = data.find('ul') # 找到所有的ul标签
li = ul.find_all('li') # 找到左右的li标签
i = 0 # 控制捕获的天数
for day in li: # 遍历找到的每一个li
if 7 > i > 0:
temp = [] # 临时存放每天的数据
date = day.find('h1').string # 得到日期
date = date[0:date.index('日')] # 取出日期号
temp.append(date)
inf = day.find_all('p') # 找出li下面的p标签,提取第一个p标签的值,即天气
temp.append(inf[0].string)
tem_low = inf[1].find('i').string # 找到最低气温
if inf[1].find('span') is None: # 天气预报可能没有最高气温
tem_high = None
else:
tem_high = inf[1].find('span').string # 找到最高气温
temp.append(tem_low[:-1])
if tem_high[-1] == '℃':
temp.append(tem_high[:-1])
else:
temp.append(tem_high)
wind = inf[2].find_all('span') # 找到风向
for j in wind:
temp.append(j['title'])
wind_scale = inf[2].find('i').string # 找到风级
index1 = wind_scale.index('级')
temp.append(int(wind_scale[index1 - 1:index1]))
final.append(temp)
i = i + 1
return final
def get_content8_14(html):
"""处理得到有用信息保存数据文件"""
final = [] # 初始化一个列表保存数据
bs = BeautifulSoup(html, "html.parser") # 创建BeautifulSoup对象
body = bs.body
data = body.find('div', {'id': '15d'}) # 找到div标签且id = 15d
ul = data.find('ul') # 找到所有的ul标签
li = ul.find_all('li') # 找到左右的li标签
# final = []
i = 0 # 控制捕获的天数
for day in li: # 遍历找到的每一个li
if i < 8:
temp = [] # 临时存放每天的数据
date = day.find('span', {'class': 'time'}).string # 得到日期
date = date[date.index('(') + 1:-2] # 取出日期号
temp.append(date)
weather = day.find('span', {'class': 'wea'}).string # 找到天气
temp.append(weather)
tem = day.find('span', {'class': 'tem'}).text # 找到温度
temp.append(tem[tem.index('/') + 1:-1]) # 找到最低气温
temp.append(tem[:tem.index('/') - 1]) # 找到最高气温
wind = day.find('span', {'class': 'wind'}).string # 找到风向
if '转' in wind: # 如果有风向变化
temp.append(wind[:wind.index('转')])
temp.append(wind[wind.index('转') + 1:])
else: # 如果没有风向变化,前后风向一致
temp.append(wind)
temp.append(wind)
wind_scale = day.find('span', {'class': 'wind1'}).string # 找到风级
index1 = wind_scale.index('级')
temp.append(int(wind_scale[index1 - 1:index1]))
final.append(temp)
return final
"""保存为csv文件"""
def write_to_csv(file_name, data, day=14):
"""保存为csv文件"""
with open(file_name, 'w', errors='ignore', newline='') as f:
if day == 14:
header = ['日期', '天气', '最低气温', '最高气温', '风向1', '风向2', '风级']
else:
header = ['小时', '温度', '风力方向', '风级', '降水量', '相对湿度', '空气质量']
f_csv = csv.writer(f)
f_csv.writerow(header)
f_csv.writerows(data)
"""创建表"""
def Creat_Table(city_name):
config = {
"host": "localhost",
"port": "3306",
"user": "root",
"password": "628",
"database": "python"
}
con = mysql.connector.connect(**config)
cursor = con.cursor()
cursor.execute('DROP TABLE IF EXISTS {}weather'.format(city_name))
sql_set = 'CREATE TABLE IF NOT EXISTS {}weather(Time INT,Temperature INT ,' \
'wind_direction VARCHAR(255), wind INT, rainfall INT, Humidity INT)'.format(city_name)
try:
cursor.execute(sql_set)
con.commit()
print('创建{}24小时天气情况数据表成功'.format(city_name))
except:
con.rollback()
print("创建{}24小时天气情况数据表失败".format(city_name))
con.close()
def Creat_Table14(city_name):
config = {
"host": "localhost",
"port": "3306",
"user": "root",
"password": "628",
"database": "python"
}
con = mysql.connector.connect(**config)
cursor = con.cursor()
cursor.execute('DROP TABLE IF EXISTS {}weather14d'.format(city_name))
sql_set = 'CREATE TABLE IF NOT EXISTS {}weather14d(Date INT,Weather VARCHAR(255) ,' \
'Min_temperature INT, Max_temperature INT, wind_direction_1 VARCHAR(255), wind_direction_2 VARCHAR(255), wind_level INT)'.format(city_name)
try:
cursor.execute(sql_set)
con.commit()
print('创建{}15天天气情况数据表成功'.format(city_name))
except:
con.rollback()
print('创建{}15天天气情况数据表失败'.format(city_name))
con.close()
"""在表中插入数据"""
def insert_data1(city_name):
config = {
"host": "localhost",
"port": "3306",
"user": "root",
"password": "628",
"database": "python"
}
con = mysql.connector.connect(**config)
cursor = con.cursor()
data1 = pd.read_csv('天气预报-{}24小时天气预报.csv'.format(city_name), encoding='gb2312')
for i in range(data1.shape[0]):
Time = list(data1['小时'])[i]
Temperature = list(data1['温度'])[i]
wind_direction = list(data1['风力方向'])[i]
wind = list(data1['风级'])[i]
rainfall = list(data1['降水量'])[i]
Humidity = list(data1['相对湿度'])[i]
# Time = list(data1['空气质量'])[1]
data = (Time, Temperature, wind_direction, wind, rainfall, Humidity)
sql = "INSERT INTO {}weather(Time, Temperature, wind_direction, wind, rainfall, Humidity) values ".format(city_name) + str(data) + ";"
try:
cursor.execute(sql) #执行sql语句
con.commit() #连接提交
except:
con.rollback()
print("插入{}24小时天气情况数据失败".format(city_name))
print('插入{}24小时天气情况数据成功'.format(city_name))
cursor.close()
con.close()
# sql = "INSERT INTO weather(Time, Temperature, wind_direction, wind, rainfall, Humidity) values('1', '1', 's', '1', '1', '1');"
# try:
# cursor.execute(sql)
# cursor.execute(sql)
# con.commit()
# print('插入数据成功')
# except:
# con.rollback()
# print("插入数据失败")
# con.close()
def insert_data14(city_name):
config = {
"host": "localhost",
"port": "3306",
"user": "root",
"password": "628",
"database": "python"
}
con = mysql.connector.connect(**config)
cursor = con.cursor()
data1 = pd.read_csv('天气预报-{}15天天气预报.csv'.format(city_name), encoding='gb2312')
for i in range(data1.shape[0]):
Date = list(data1['日期'])[i]
Weather = list(data1['天气'])[i]
Min_temperature = list(data1['最低气温'])[i]
Max_temperature = list(data1['最高气温'])[i]
wind_direction_1 = list(data1['风向1'])[i]
wind_direction_2 = list(data1['风向2'])[i]
wind_level = list(data1['风级'])[i]
# Time = list(data1['空气质量'])[1]
data = (Date, Weather, Min_temperature, Max_temperature, wind_direction_1, wind_direction_2,wind_level)
sql = "INSERT INTO {}weather14d(Date, Weather, Min_temperature, Max_temperature, wind_direction_1, wind_direction_2,wind_level) values ".format(city_name) + str(data) + ";"
try:
cursor.execute(sql) #执行sql语句
con.commit() #连接提交
except:
con.rollback()
print("插入{}半个月天气情况数据失败")
print('插入{}15天天气情况数据成功'.format(city_name))
cursor.close()
con.close()
"""温度曲线绘制"""
def tem_curve(data,city_name ):
"""温度曲线绘制"""
# plt.style.use('seaborn')
hour = list(data['小时'])
# print(hour)
tem = list(data['温度'])
for i in range(0, 24):
if math.isnan(tem[i]):
tem[i] = tem[i - 1]
tem_ave = sum(tem) / 24 # 求平均温度
tem_max = max(tem)
tem_max_hour = hour[tem.index(tem_max)] # 求最高温度
tem_min = min(tem)
tem_min_hour = hour[tem.index(tem_min)] # 求最低温度
x = []
y = []
for i in range(0, 24):
x.append(i)
y.append(tem[hour.index(i)])
plt.style.use('seaborn')
matplotlib.rcParams['font.sans-serif'] = ['SimHei']
matplotlib.rcParams['axes.unicode_minus'] = False
plt.figure(1)
plt.plot(x, y, color='red', label='温度',marker = '^') # 画出温度曲线
plt.scatter(x, y, color='red') # 点出每个时刻的温度点
plt.plot([0, 24], [tem_ave, tem_ave], c='blue', linestyle='--', label='平均温度') # 画出平均温度虚线
plt.text(tem_max_hour + 0.15, tem_max + 0.15, str(tem_max), ha='center', va='bottom', fontsize=10.5) # 标出最高温度
plt.text(tem_min_hour + 0.15, tem_min + 0.15, str(tem_min), ha='center', va='bottom', fontsize=10.5) # 标出最低温度
plt.xticks(x)
plt.legend()
plt.title('一天温度变化曲线图')
plt.xlabel('时间/h')
plt.ylabel('摄氏度/℃')
plt.savefig('image/{}24小时温度曲线.png'.format(city_name), bbox_inches='tight')
plt.show()
"""相对湿度曲线绘制"""
def hum_curve(data,city_name):
"""相对湿度曲线绘制"""
hour = list(data['小时'])
hum = list(data['相对湿度'])
for i in range(0, 24):
if math.isnan(hum[i]) == True:
hum[i] = hum[i - 1]
hum_ave = sum(hum) / 24 # 求平均相对湿度
hum_max = max(hum)
hum_max_hour = hour[hum.index(hum_max)] # 求最高相对湿度
hum_min = min(hum)
hum_min_hour = hour[hum.index(hum_min)] # 求最低相对湿度
x = []
y = []
for i in range(0, 24):
x.append(i)
y.append(hum[hour.index(i)])
plt.figure(2)
plt.plot(x, y, color='blue', label='相对湿度',marker = '^') # 画出相对湿度曲线
plt.scatter(x, y, color='blue') # 点出每个时刻的相对湿度
plt.plot([0, 24], [hum_ave, hum_ave], c='red', linestyle='--', label='平均相对湿度',marker = '*') # 画出平均相对湿度虚线
plt.text(hum_max_hour + 0.15, hum_max + 0.15, str(hum_max), ha='center', va='bottom', fontsize=10.5) # 标出最高相对湿度
plt.text(hum_min_hour + 0.15, hum_min + 0.15, str(hum_min), ha='center', va='bottom', fontsize=10.5) # 标出最低相对湿度
plt.xticks(x)
plt.legend()
plt.title('一天相对湿度变化曲线图')
plt.xlabel('时间/h')
plt.ylabel('百分比/%')
plt.savefig('image/{}相对湿度曲线.png'.format(city_name), bbox_inches='tight')
plt.show()
"""空气质量曲线绘制"""
def air_curve(data,city_name):
"""空气质量曲线绘制"""
hour = list(data['小时'])
air = list(data['空气质量'])
# print(type(air[0]))
for i in range(0, 24):
if math.isnan(air[i]) == True:
air[i] = air[i - 1]
air_ave = sum(air) / 24 # 求平均空气质量
air_max = max(air)
air_max_hour = hour[air.index(air_max)] # 求最高空气质量
air_min = min(air)
air_min_hour = hour[air.index(air_min)] # 求最低空气质量
x = []
y = []
for i in range(0, 24):
x.append(i)
y.append(air[hour.index(i)])
plt.figure(3)
for i in range(0, 24):
if y[i] <= 50:
plt.bar(x[i], y[i], color='lightgreen', width=0.7) # 1等级
elif y[i] <= 100:
plt.bar(x[i], y[i], color='wheat', width=0.7) # 2等级
elif y[i] <= 150:
plt.bar(x[i], y[i], color='orange', width=0.7) # 3等级
elif y[i] <= 200:
plt.bar(x[i], y[i], color='orangered', width=0.7) # 4等级
elif y[i] <= 300:
plt.bar(x[i], y[i], color='darkviolet', width=0.7) # 5等级
elif y[i] > 300:
plt.bar(x[i], y[i], color='maroon', width=0.7) # 6等级
plt.plot([0, 24], [air_ave, air_ave], c='black', linestyle='--') # 画出平均空气质量虚线
plt.text(air_max_hour + 0.15, air_max + 0.15, str(air_max), ha='center', va='bottom', fontsize=10.5) # 标出最高空气质量
plt.text(air_min_hour + 0.15, air_min + 0.15, str(air_min), ha='center', va='bottom', fontsize=10.5) # 标出最低空气质量
plt.xticks(x)
plt.title('一天空气质量变化曲线图')
plt.xlabel('时间/h')
plt.ylabel('空气质量指数AQI')
plt.savefig('image/{}空气质量曲线.png'.format(city_name), bbox_inches='tight')
plt.show()
"""计算相关系数"""
def calc_corr(a, b):
"""计算相关系数"""
a_avg = sum(a) / len(a)
b_avg = sum(b) / len(b)
cov_ab = sum([(x - a_avg) * (y - b_avg) for x, y in zip(a, b)])
sq = math.sqrt(sum([(x - a_avg) ** 2 for x in a]) * sum([(x - b_avg) ** 2 for x in b]))
corr_factor = cov_ab / sq
return corr_factor
"""温湿度相关性分析"""
def corr_tem_hum(data,city_name):
"""温湿度相关性分析"""
tem = data['温度']
hum = data['相对湿度']
plt.scatter(tem, hum, color='blue')
plt.title("温湿度相关性分析图")
plt.xlabel("温度/℃")
plt.ylabel("相对湿度/%")
# plt.text(13, 40, "相关系数为:" + str(calc_corr(tem, hum)), fontdict={'size': '10', 'color': 'red'})
plt.savefig('image/{}温湿度相关性.png'.format(city_name), bbox_inches='tight')
plt.show()
# print("相关系数为:" + str(calc_corr(tem, hum)))
"""15天温度曲线绘制"""
def tem_curve14(data,city_name):
date = list(data['日期'])
tem_low = list(data['最低气温'])
tem_high = list(data['最高气温'])
for i in range(0, 14):
if math.isnan(tem_low[i]):
tem_low[i] = tem_low[i - 1]
if math.isnan(tem_high[i]):
tem_high[i] = tem_high[i - 1]
tem_high_ave = sum(tem_high) / 14 # 求平均高温
tem_low_ave = sum(tem_low) / 14 # 求平均低温
tem_max = max(tem_high)
tem_max_date = tem_high.index(tem_max) # 求最高温度
tem_min = min(tem_low)
tem_min_date = tem_low.index(tem_min) # 求最低温度
x = range(1, 15)
plt.figure(1)
plt.plot(x, tem_high, color='red', label='高温',marker = '^') # 画出高温度曲线
plt.scatter(x, tem_high, color='red') # 点出每个时刻的温度点
plt.plot(x, tem_low, color='blue', label='低温',marker = '^') # 画出低温度曲线
plt.scatter(x, tem_low, color='blue') # 点出每个时刻的温度点
plt.plot([1, 15], [tem_high_ave, tem_high_ave], c='black', linestyle='--') # 画出平均温度虚线
plt.plot([1, 15], [tem_low_ave, tem_low_ave], c='black', linestyle='--') # 画出平均温度虚线
plt.legend()
plt.text(tem_max_date + 0.15, tem_max + 0.15, str(tem_max), ha='center', va='bottom', fontsize=10.5) # 标出最高温度
plt.text(tem_min_date + 0.15, tem_min + 0.15, str(tem_min), ha='center', va='bottom', fontsize=10.5) # 标出最低温度
plt.xticks(x)
plt.title('未来14天高温低温变化曲线图')
plt.xlabel('未来天数/天')
plt.ylabel('摄氏度/℃')
plt.savefig('image/{}15天温度曲线.png'.format(city_name), bbox_inches='tight')
plt.show()
"""改变风向"""
def change_wind(wind):
for i in range(0, 14):
if wind[i] == "北风":
wind[i] = 90
elif wind[i] == "南风":
wind[i] = 270
elif wind[i] == "西风":
wind[i] = 180
elif wind[i] == "东风":
wind[i] = 360
elif wind[i] == "东北风":
wind[i] = 45
elif wind[i] == "西北风":
wind[i] = 135
elif wind[i] == "西南风":
wind[i] = 225
elif wind[i] == "东南风":
wind[i] = 315
return wind
"""风向雷达图"""
def wind_radar(data,city_name):
wind1 = list(data['风向1'])
wind2 = list(data['风向2'])
wind_speed = list(data['风级'])
wind1 = change_wind(wind1)
wind2 = change_wind(wind2)
degs = np.arange(45, 361, 45)
temp = []
for deg in degs:
speed = []
# 获取 wind_deg 在指定范围的风速平均值数据
for i in range(0, 14):
if wind1[i] == deg:
speed.append(wind_speed[i])
if wind2[i] == deg:
speed.append(wind_speed[i])
if len(speed) == 0:
temp.append(0)
else:
temp.append(sum(speed) / len(speed))
# print(temp)
N = 8
theta = np.arange(0. + np.pi / 8, 2 * np.pi + np.pi / 8, 2 * np.pi / 8)
# 数据极径
radii = np.array(temp)
# 绘制极区图坐标系
plt.axes(polar=True)
# 定义每个扇区的RGB值(R,G,B),x越大,对应的颜色越接近蓝色
colors = [(1 - x / max(temp), 1 - x / max(temp), 0.6) for x in radii]
plt.bar(theta, radii, width=(2 * np.pi / N), bottom=0.0, color=colors)
plt.title('未来14天风级图', x=0.2, fontsize=20)
plt.savefig('image/{}风向曲线.png'.format(city_name), bbox_inches='tight')
plt.show()
"""绘制天气饼图"""
def weather_pie(data,city_name):
weather = list(data['天气'])
dic_wea = {}
for i in range(0, 14):
if weather[i] in dic_wea.keys():
dic_wea[weather[i]] += 1
else:
dic_wea[weather[i]] = 1
# print(dic_wea)
explode = [0.01] * len(dic_wea.keys())
color = ['lightskyblue', 'silver', 'yellow', 'salmon', 'grey', 'lime', 'gold', 'red', 'green', 'pink']
plt.pie(dic_wea.values(), explode=explode, labels=dic_wea.keys(), autopct='%1.1f%%', colors=color)
plt.title('未来14天气候分布饼图')
plt.savefig('image/{}天气饼图.png'.format(city_name), bbox_inches='tight')
plt.show()
def main():
"""主函数"""
city_name = input("请输入要查询的城市:")
judge = Judge(city_name)
while judge == "你这儿是那个山卡卡":
print("你勒儿是那个山卡卡,我查不到,你换个塔塔诶")
city_name = input("请输入要查询的城市:")
judge = Judge(city_name)
print(judge)
if judge == city_name:
url_1 = getURL(city_name) # 7天天气某天气网
# print(url_1)
url_2 = getURL2(city_name)
html1 = getHTMLtext(url_1,city_name)
data1 = get_content(html1) # 获得1-7天和当天的数据
data1_7 = get_content1_7(html1)
html2 = getHTMLtext2(url_2,city_name)
data8_14 = get_content8_14(html2)
data14 = data1_7 + data8_14
write_to_csv('天气预报-{}24小时天气预报.csv'.format(city_name), data1, 1)
write_to_csv('天气预报-{}15天天气预报.csv'.format(city_name), data14, 14)
Creat_Table(city_name)
Creat_Table14(city_name)
insert_data1(city_name)
insert_data14(city_name)
plt.style.use('seaborn')
matplotlib.rcParams['font.sans-serif'] = ['SimHei'] # 解决中文显示问题
matplotlib.rcParams['axes.unicode_minus'] = False # 解决负号显示问题
data1 = pd.read_csv('天气预报-{}24小时天气预报.csv'.format(city_name), encoding='gb2312')
data14 = pd.read_csv('天气预报-{}15天天气预报.csv'.format(city_name), encoding='gb2312')
# tem_curve(data1,city_name) #24小时温度曲线绘制
# hum_curve(data1,city_name) #相对湿度曲线绘制
# air_curve(data1,city_name) #空气质量曲线绘制
# corr_tem_hum(data1,city_name) #温湿度相关性分析
# tem_curve14(data14,city_name) #15天温度曲线绘制
# wind_radar(data14,city_name) #风向雷达图
# weather_pie(data14,city_name) #绘制天气饼图
while True:
num = int(input("请输入要查询的图表(输入1查询24小时温度曲线绘制、输入2查询相对湿度曲线绘制、"
"输入3查询空气质量曲线绘制、输入4查询15天温度曲线、输入5风向雷达图、"
"输入6天气饼图图):"))
if num == 1:
tem_curve(data1,city_name) #24小时温度曲线绘制
elif num == 2:
hum_curve(data1,city_name) #相对湿度曲线绘制
elif num == 3:
air_curve(data1,city_name) #空气质量曲线绘制
elif num == 4:
tem_curve14(data14, city_name) # 15天温度曲线绘制
elif num == 5:
wind_radar(data14, city_name) # 风向雷达图
elif num == 6:
weather_pie(data14, city_name) # 绘制天气饼图
else:
print("查询结束...")
return False
if __name__ == '__main__':
main()
这算是我第一次真正意义上自己独立完成的python程序,做下来还是比较兴奋的,同时还有许多想法,比如加个scrapy框架,再搞个pyqt5的gui,让这个程序更加完美,可惜现在的知识储备不足,所以暂时只能做这么多。这也是我发布的第一篇csdn,以后编程中遇到的,收获的,我都会继续在csdn发布
本文链接:https://blog.csdn.net/weixin_45821611/article/details/121798585