Python查询天气预报
分类: Python 2013-12-02 08:59 1357人阅读 评论(2) 收藏 举报
Python天气预报
目录(?)[+]
一. 实现过程
1.1 查询外网IP
通过这个网址查询到外网IP http://ip.dnsexit.com/index.php
1.2 查询IP所在省份和城市
通过这个地址查询到IP所在省份和城市 http://int.dpool.sina.com.cn/iplookup/iplookup.php?format=json&ip=54.54.194.134
1.3 查询所在城市的天气URL
所在省份和城市, 查找到城市天气的URL
1.4 查询所在城市的天气情况.
通过这个网址查询天气的URL查询天气信息
http://m.weather.com.cn/data/101280101.html
(前面3个步骤都是为了这一步准备)
二. 实现技术
2.1 网页数据抓取与提取
所有网页数据通过Python抓取, 然后使用正则表达式或者BeautifulSoup或者json来解析.
2.2 城市天气URL的获取
利用这个网站上的信息http://www.weather.com.cn/
先获得城市的省份URL, 在通过省份信息获得该城市的URL
2.3 天气信息的获得
利用这个网站上的信息http://www.weather.com.cn/
使用json解析.
三. 实现
3.1 查询外网IP
这个简单
[python] view plaincopy
#!/usr/bin/env python
# coding=utf-8
# Python 2.7.3
# File: GetIP.py
# 获得外网IP地址
import urllib2
import httplib
def GetIP():
response = urllib2.urlopen('http://ip.dnsexit.com/index.php')
htmlStr = response.read()
return htmlStr
'''''
# 测试代码
print GetIP()
'''
3.2 获得IP所在省份和城市
这个也很简单
[python] view plaincopy
#!/usr/bin/env python
# coding=utf-8
# Python 2.7.3
# File: GetCity.py
# 获取IP所在国家/省份/城市
import urllib2
import httplib
import json
'''''
返回信息的结构
{"ret":1,"start":"54.52.163.0","end":"54.57.3.255","country":"美国","province":"新泽西州","city":"Woodbridge","district":"","isp":"联通","type":"","desc":""}
'''
def GetCity(ip, city):
response = urllib2.urlopen('http://int.dpool.sina.com.cn/iplookup/iplookup.php?format=json&ip=' + ip)
htmlStr = response.read()
cityInfo = htmlStr.decode("unicode-escape");
st = json.loads(cityInfo);
city[0] = st["country"]
city[1] = st["province"]
city[2] = st["city"]
'''''
# 测试代码
city = ["", "", ""]
GetCity("54.54.194.134", city)
print city
'''
3.3 获取城市的天气的URL
先获得省份信息, 在查找城市信息.
3.3.1 获得省份URL
看这个网址 http://www.weather.com.cn/textFC/hb.shtml
分析html(保存为GetCityID1.html)知道 <div class="lqcontentBoxheader">包含省份信息. 由于这个html比较复杂和有些字符不是唯一的, 所以这里使用的是BeautifulSoup分析.
3.3.2 获得城市URL
上面步骤获得省份URL后, 例如 http://www.weather.com.cn/textFC/xizang.shtml
分析html(保存为GetCityID2.html)知道 <div class="hanml"> 包含城市URL信息. 同样使用BeautifulSoup分析.
这里获得的URL是这样的格式 http://www.weather.com.cn/weather/101280101.shtml 你需要修改成这样的格式http://m.weather.com.cn/data/101280101.html
3.3.3 实现代码
这个代码有一个明显的缺点, 就是运行速度很慢(一个是网站数据比较多所以慢, 还有就是BeautifulSoup分析也有一点慢(HTML的数据太多了)). 所以动态或者这个就比较慢了, 先把这些URL下载再来保存到本地也是一个好方法.
[python] view plaincopy
#!/usr/bin/env python
# coding=utf-8
# Python 2.7.3
# File: GetCityID.py
# 获取城市的天气的URL地址
import urllib2
import HTMLParser
import httplib
from bs4 import BeautifulSoup
def GetProvinceURL(province):
response = urllib2.urlopen('http://www.weather.com.cn/textFC/hn.shtml')
htmlByte = response.read()
htmlStr = htmlByte.decode("utf8")
soup2 = BeautifulSoup(htmlStr)
div = soup2.find("div", class_ = "lqcontentBoxheader")
lista = div.find_all("a")
provinceURL = "http://www.weather.com.cn"
for aItem in lista:
if aItem.text == province:
provinceURL = provinceURL + aItem["href"]
break
return provinceURL
def GetCityURL(provinceURL, city):
response = urllib2.urlopen(provinceURL)
htmlByte = response.read()
htmlStr = htmlByte.decode("utf8")
soup2 = BeautifulSoup(htmlStr)
div = soup2.find("div", class_ = "hanml")
lista = div.find_all("a", text = city)
cityURL = lista[0]["href"].replace("www.weather.com.cn/weather", "m.weather.com.cn/data")
cityURL = cityURL.replace("shtml", "html")
return cityURL
'''''
# GetProvinceURL 测试代码
print GetProvinceURL(u"广东")
'''
# GetProvinceURL 测试代码
provinceURL = GetProvinceURL(u"广东")
print provinceURL
cityURL = GetCityURL(provinceURL, u"广州")
print cityURL
3.4 天气数据的获取
3.4.1 天气数据的解析
从获http://m.weather.com.cn/data/101280101.html得到的数据是Json格式, 需要进行解析. (有了这些数据, 你喜欢怎么显示都可以了)
{"weatherinfo":{"city":"广州","city_en":"guangzhou","date_y":"2013年11月29日","date":"","week":"星期五","fchh":"11","cityid":"101280101","temp1":"18℃~5℃","temp2":"20℃~7℃","temp3":"21℃~8℃","temp4":"21℃~9℃","temp5":"22℃~10℃","temp6":"23℃~10℃","tempF1":"64.4℉~41℉","tempF2":"68℉~44.6℉","tempF3":"69.8℉~46.4℉","tempF4":"69.8℉~48.2℉","tempF5":"71.6℉~50℉","tempF6":"73.4℉~50℉","weather1":"晴","weather2":"晴","weather3":"晴","weather4":"晴","weather5":"晴","weather6":"晴","img1":"0","img2":"99","img3":"0","img4":"99","img5":"0","img6":"99","img7":"0","img8":"99","img9":"0","img10":"99","img11":"0","img12":"99","img_single":"0","img_title1":"晴","img_title2":"晴","img_title3":"晴","img_title4":"晴","img_title5":"晴","img_title6":"晴","img_title7":"晴","img_title8":"晴","img_title9":"晴","img_title10":"晴","img_title11":"晴","img_title12":"晴","img_title_single":"晴","wind1":"北风3-4级转微风","wind2":"微风","wind3":"微风","wind4":"微风","wind5":"微风","wind6":"微风","fx1":"北风","fx2":"微风","fl1":"3-4级转小于3级","fl2":"小于3级","fl3":"小于3级","fl4":"小于3级","fl5":"小于3级","fl6":"小于3级","index":"较冷","index_d":"建议着大衣、呢外套加毛衣、卫衣等服装。体弱者宜着厚外套、厚毛衣。因昼夜温差较大,注意增减衣服。","index48":"较冷","index48_d":"建议着大衣、呢外套加毛衣、卫衣等服装。体弱者宜着厚外套、厚毛衣。因昼夜温差较大,注意增减衣服。","index_uv":"中等","index48_uv":"中等","index_xc":"适宜","index_tr":"适宜","index_co":"舒适","st1":"16","st2":"6","st3":"19","st4":"8","st5":"20","st6":"9","index_cl":"不宜","index_ls":"适宜","index_ag":"易发"}}
[python] view plaincopy
#!/usr/bin/env python
# coding=utf-8
# Python 2.7.3
# File: GetCityWeather.py
# 获得城市天气数据
import urllib2
import httplib
import json
def GetCityWeather(cityURL):
response = urllib2.urlopen(cityURL)
htmlByte = response.read()
htmlStr = htmlByte.decode("utf8")
st = json.loads(htmlStr);
return st
'''''
# http://m.weather.com.cn/data/101280101.html
{"weatherinfo":{"city":"广州","city_en":"guangzhou","date_y":"2013年11月29日","date":"","week":"星期五","fchh":"11","cityid":"101280101","temp1":"18℃~5℃","temp2":"20℃~7℃","temp3":"21℃~8℃","temp4":"21℃~9℃","temp5":"22℃~10℃","temp6":"23℃~10℃","tempF1":"64.4℉~41℉","tempF2":"68℉~44.6℉","tempF3":"69.8℉~46.4℉","tempF4":"69.8℉~48.2℉","tempF5":"71.6℉~50℉","tempF6":"73.4℉~50℉","weather1":"晴","weather2":"晴","weather3":"晴","weather4":"晴","weather5":"晴","weather6":"晴","img1":"0","img2":"99","img3":"0","img4":"99","img5":"0","img6":"99","img7":"0","img8":"99","img9":"0","img10":"99","img11":"0","img12":"99","img_single":"0","img_title1":"晴","img_title2":"晴","img_title3":"晴","img_title4":"晴","img_title5":"晴","img_title6":"晴","img_title7":"晴","img_title8":"晴","img_title9":"晴","img_title10":"晴","img_title11":"晴","img_title12":"晴","img_title_single":"晴","wind1":"北风3-4级转微风","wind2":"微风","wind3":"微风","wind4":"微风","wind5":"微风","wind6":"微风","fx1":"北风","fx2":"微风","fl1":"3-4级转小于3级","fl2":"小于3级","fl3":"小于3级","fl4":"小于3级","fl5":"小于3级","fl6":"小于3级","index":"较冷","index_d":"建议着大衣、呢外套加毛衣、卫衣等服装。体弱者宜着厚外套、厚毛衣。因昼夜温差较大,注意增减衣服。","index48":"较冷","index48_d":"建议着大衣、呢外套加毛衣、卫衣等服装。体弱者宜着厚外套、厚毛衣。因昼夜温差较大,注意增减衣服。","index_uv":"中等","index48_uv":"中等","index_xc":"适宜","index_tr":"适宜","index_co":"舒适","st1":"16","st2":"6","st3":"19","st4":"8","st5":"20","st6":"9","index_cl":"不宜","index_ls":"适宜","index_ag":"易发"}}
'''
'''''
# GetCityWeather测试代码
# GetProvinceURL 测试代码
cityURL = "http://m.weather.com.cn/data/101280101.html"
st = GetCityWeather(cityURL)
ss = st["weatherinfo"]
print ss["city"]
print ss["date_y"]
print ss["week"]
print ss["temp1"]
print ss["weather1"]
'''
'''''
# 输出
广州
2013年11月29日
星期五
18℃~5℃
晴
'''
3.5 主程序代码
[python] view plaincopy
#!/usr/bin/env python
# coding=utf-8
# Python 2.7.3
import GetIP
import GetCity
import GetCityID
import GetCityWeather
ip = GetIP.GetIP()
print ip
# 国家/省份/城市
city = ["", "", ""]
GetCity.GetCity(ip, city)
print city[0], city[1], city[2]
provinceURL = GetCityID.GetProvinceURL(city[1])
cityURL = GetCityID.GetCityURL(provinceURL, city[2])
print provinceURL
print cityURL
st = GetCityWeather.GetCityWeather(cityURL)
ss = st["weatherinfo"]
print ss["city"]
print ss["date_y"]
print ss["week"]
print ss["temp1"]
print ss["weather1"]
这两段代码运行的非常慢
provinceURL = GetCityID.GetProvinceURL(city[1])
cityURL = GetCityID.GetCityURL(provinceURL, city[2])
四. 小结
4.1 国外的城市可能查不到, 因为天气数据依赖于http://www.weather.com.cn/
4.2 获取城市url的速度实在太慢了. 的确先提取保存可能会更快吧.
4.3 通过实现这样的功能, 了解了json.
4.4 网上有很多有用的数据, 特别是一些动态的海量的数据, 你不可能手动去取, 就看你能不能抓, 找出规律, 两手抓, 两手都要硬
4.5 本文是参考http://blog.csdn.net/x_iya/article/details/8583015
分类: Python 2013-12-02 08:59 1357人阅读 评论(2) 收藏 举报
Python天气预报
目录(?)[+]
一. 实现过程
1.1 查询外网IP
通过这个网址查询到外网IP http://ip.dnsexit.com/index.php
1.2 查询IP所在省份和城市
通过这个地址查询到IP所在省份和城市 http://int.dpool.sina.com.cn/iplookup/iplookup.php?format=json&ip=54.54.194.134
1.3 查询所在城市的天气URL
所在省份和城市, 查找到城市天气的URL
1.4 查询所在城市的天气情况.
通过这个网址查询天气的URL查询天气信息
http://m.weather.com.cn/data/101280101.html
(前面3个步骤都是为了这一步准备)
二. 实现技术
2.1 网页数据抓取与提取
所有网页数据通过Python抓取, 然后使用正则表达式或者BeautifulSoup或者json来解析.
2.2 城市天气URL的获取
利用这个网站上的信息http://www.weather.com.cn/
先获得城市的省份URL, 在通过省份信息获得该城市的URL
2.3 天气信息的获得
利用这个网站上的信息http://www.weather.com.cn/
使用json解析.
三. 实现
3.1 查询外网IP
这个简单
[python] view plaincopy
#!/usr/bin/env python
# coding=utf-8
# Python 2.7.3
# File: GetIP.py
# 获得外网IP地址
import urllib2
import httplib
def GetIP():
response = urllib2.urlopen('http://ip.dnsexit.com/index.php')
htmlStr = response.read()
return htmlStr
'''''
# 测试代码
print GetIP()
'''
3.2 获得IP所在省份和城市
这个也很简单
[python] view plaincopy
#!/usr/bin/env python
# coding=utf-8
# Python 2.7.3
# File: GetCity.py
# 获取IP所在国家/省份/城市
import urllib2
import httplib
import json
'''''
返回信息的结构
{"ret":1,"start":"54.52.163.0","end":"54.57.3.255","country":"美国","province":"新泽西州","city":"Woodbridge","district":"","isp":"联通","type":"","desc":""}
'''
def GetCity(ip, city):
response = urllib2.urlopen('http://int.dpool.sina.com.cn/iplookup/iplookup.php?format=json&ip=' + ip)
htmlStr = response.read()
cityInfo = htmlStr.decode("unicode-escape");
st = json.loads(cityInfo);
city[0] = st["country"]
city[1] = st["province"]
city[2] = st["city"]
'''''
# 测试代码
city = ["", "", ""]
GetCity("54.54.194.134", city)
print city
'''
3.3 获取城市的天气的URL
先获得省份信息, 在查找城市信息.
3.3.1 获得省份URL
看这个网址 http://www.weather.com.cn/textFC/hb.shtml
分析html(保存为GetCityID1.html)知道 <div class="lqcontentBoxheader">包含省份信息. 由于这个html比较复杂和有些字符不是唯一的, 所以这里使用的是BeautifulSoup分析.
3.3.2 获得城市URL
上面步骤获得省份URL后, 例如 http://www.weather.com.cn/textFC/xizang.shtml
分析html(保存为GetCityID2.html)知道 <div class="hanml"> 包含城市URL信息. 同样使用BeautifulSoup分析.
这里获得的URL是这样的格式 http://www.weather.com.cn/weather/101280101.shtml 你需要修改成这样的格式http://m.weather.com.cn/data/101280101.html
3.3.3 实现代码
这个代码有一个明显的缺点, 就是运行速度很慢(一个是网站数据比较多所以慢, 还有就是BeautifulSoup分析也有一点慢(HTML的数据太多了)). 所以动态或者这个就比较慢了, 先把这些URL下载再来保存到本地也是一个好方法.
[python] view plaincopy
#!/usr/bin/env python
# coding=utf-8
# Python 2.7.3
# File: GetCityID.py
# 获取城市的天气的URL地址
import urllib2
import HTMLParser
import httplib
from bs4 import BeautifulSoup
def GetProvinceURL(province):
response = urllib2.urlopen('http://www.weather.com.cn/textFC/hn.shtml')
htmlByte = response.read()
htmlStr = htmlByte.decode("utf8")
soup2 = BeautifulSoup(htmlStr)
div = soup2.find("div", class_ = "lqcontentBoxheader")
lista = div.find_all("a")
provinceURL = "http://www.weather.com.cn"
for aItem in lista:
if aItem.text == province:
provinceURL = provinceURL + aItem["href"]
break
return provinceURL
def GetCityURL(provinceURL, city):
response = urllib2.urlopen(provinceURL)
htmlByte = response.read()
htmlStr = htmlByte.decode("utf8")
soup2 = BeautifulSoup(htmlStr)
div = soup2.find("div", class_ = "hanml")
lista = div.find_all("a", text = city)
cityURL = lista[0]["href"].replace("www.weather.com.cn/weather", "m.weather.com.cn/data")
cityURL = cityURL.replace("shtml", "html")
return cityURL
'''''
# GetProvinceURL 测试代码
print GetProvinceURL(u"广东")
'''
# GetProvinceURL 测试代码
provinceURL = GetProvinceURL(u"广东")
print provinceURL
cityURL = GetCityURL(provinceURL, u"广州")
print cityURL
3.4 天气数据的获取
3.4.1 天气数据的解析
从获http://m.weather.com.cn/data/101280101.html得到的数据是Json格式, 需要进行解析. (有了这些数据, 你喜欢怎么显示都可以了)
{"weatherinfo":{"city":"广州","city_en":"guangzhou","date_y":"2013年11月29日","date":"","week":"星期五","fchh":"11","cityid":"101280101","temp1":"18℃~5℃","temp2":"20℃~7℃","temp3":"21℃~8℃","temp4":"21℃~9℃","temp5":"22℃~10℃","temp6":"23℃~10℃","tempF1":"64.4℉~41℉","tempF2":"68℉~44.6℉","tempF3":"69.8℉~46.4℉","tempF4":"69.8℉~48.2℉","tempF5":"71.6℉~50℉","tempF6":"73.4℉~50℉","weather1":"晴","weather2":"晴","weather3":"晴","weather4":"晴","weather5":"晴","weather6":"晴","img1":"0","img2":"99","img3":"0","img4":"99","img5":"0","img6":"99","img7":"0","img8":"99","img9":"0","img10":"99","img11":"0","img12":"99","img_single":"0","img_title1":"晴","img_title2":"晴","img_title3":"晴","img_title4":"晴","img_title5":"晴","img_title6":"晴","img_title7":"晴","img_title8":"晴","img_title9":"晴","img_title10":"晴","img_title11":"晴","img_title12":"晴","img_title_single":"晴","wind1":"北风3-4级转微风","wind2":"微风","wind3":"微风","wind4":"微风","wind5":"微风","wind6":"微风","fx1":"北风","fx2":"微风","fl1":"3-4级转小于3级","fl2":"小于3级","fl3":"小于3级","fl4":"小于3级","fl5":"小于3级","fl6":"小于3级","index":"较冷","index_d":"建议着大衣、呢外套加毛衣、卫衣等服装。体弱者宜着厚外套、厚毛衣。因昼夜温差较大,注意增减衣服。","index48":"较冷","index48_d":"建议着大衣、呢外套加毛衣、卫衣等服装。体弱者宜着厚外套、厚毛衣。因昼夜温差较大,注意增减衣服。","index_uv":"中等","index48_uv":"中等","index_xc":"适宜","index_tr":"适宜","index_co":"舒适","st1":"16","st2":"6","st3":"19","st4":"8","st5":"20","st6":"9","index_cl":"不宜","index_ls":"适宜","index_ag":"易发"}}
[python] view plaincopy
#!/usr/bin/env python
# coding=utf-8
# Python 2.7.3
# File: GetCityWeather.py
# 获得城市天气数据
import urllib2
import httplib
import json
def GetCityWeather(cityURL):
response = urllib2.urlopen(cityURL)
htmlByte = response.read()
htmlStr = htmlByte.decode("utf8")
st = json.loads(htmlStr);
return st
'''''
# http://m.weather.com.cn/data/101280101.html
{"weatherinfo":{"city":"广州","city_en":"guangzhou","date_y":"2013年11月29日","date":"","week":"星期五","fchh":"11","cityid":"101280101","temp1":"18℃~5℃","temp2":"20℃~7℃","temp3":"21℃~8℃","temp4":"21℃~9℃","temp5":"22℃~10℃","temp6":"23℃~10℃","tempF1":"64.4℉~41℉","tempF2":"68℉~44.6℉","tempF3":"69.8℉~46.4℉","tempF4":"69.8℉~48.2℉","tempF5":"71.6℉~50℉","tempF6":"73.4℉~50℉","weather1":"晴","weather2":"晴","weather3":"晴","weather4":"晴","weather5":"晴","weather6":"晴","img1":"0","img2":"99","img3":"0","img4":"99","img5":"0","img6":"99","img7":"0","img8":"99","img9":"0","img10":"99","img11":"0","img12":"99","img_single":"0","img_title1":"晴","img_title2":"晴","img_title3":"晴","img_title4":"晴","img_title5":"晴","img_title6":"晴","img_title7":"晴","img_title8":"晴","img_title9":"晴","img_title10":"晴","img_title11":"晴","img_title12":"晴","img_title_single":"晴","wind1":"北风3-4级转微风","wind2":"微风","wind3":"微风","wind4":"微风","wind5":"微风","wind6":"微风","fx1":"北风","fx2":"微风","fl1":"3-4级转小于3级","fl2":"小于3级","fl3":"小于3级","fl4":"小于3级","fl5":"小于3级","fl6":"小于3级","index":"较冷","index_d":"建议着大衣、呢外套加毛衣、卫衣等服装。体弱者宜着厚外套、厚毛衣。因昼夜温差较大,注意增减衣服。","index48":"较冷","index48_d":"建议着大衣、呢外套加毛衣、卫衣等服装。体弱者宜着厚外套、厚毛衣。因昼夜温差较大,注意增减衣服。","index_uv":"中等","index48_uv":"中等","index_xc":"适宜","index_tr":"适宜","index_co":"舒适","st1":"16","st2":"6","st3":"19","st4":"8","st5":"20","st6":"9","index_cl":"不宜","index_ls":"适宜","index_ag":"易发"}}
'''
'''''
# GetCityWeather测试代码
# GetProvinceURL 测试代码
cityURL = "http://m.weather.com.cn/data/101280101.html"
st = GetCityWeather(cityURL)
ss = st["weatherinfo"]
print ss["city"]
print ss["date_y"]
print ss["week"]
print ss["temp1"]
print ss["weather1"]
'''
'''''
# 输出
广州
2013年11月29日
星期五
18℃~5℃
晴
'''
3.5 主程序代码
[python] view plaincopy
#!/usr/bin/env python
# coding=utf-8
# Python 2.7.3
import GetIP
import GetCity
import GetCityID
import GetCityWeather
ip = GetIP.GetIP()
print ip
# 国家/省份/城市
city = ["", "", ""]
GetCity.GetCity(ip, city)
print city[0], city[1], city[2]
provinceURL = GetCityID.GetProvinceURL(city[1])
cityURL = GetCityID.GetCityURL(provinceURL, city[2])
print provinceURL
print cityURL
st = GetCityWeather.GetCityWeather(cityURL)
ss = st["weatherinfo"]
print ss["city"]
print ss["date_y"]
print ss["week"]
print ss["temp1"]
print ss["weather1"]
这两段代码运行的非常慢
provinceURL = GetCityID.GetProvinceURL(city[1])
cityURL = GetCityID.GetCityURL(provinceURL, city[2])
四. 小结
4.1 国外的城市可能查不到, 因为天气数据依赖于http://www.weather.com.cn/
4.2 获取城市url的速度实在太慢了. 的确先提取保存可能会更快吧.
4.3 通过实现这样的功能, 了解了json.
4.4 网上有很多有用的数据, 特别是一些动态的海量的数据, 你不可能手动去取, 就看你能不能抓, 找出规律, 两手抓, 两手都要硬
4.5 本文是参考http://blog.csdn.net/x_iya/article/details/8583015