这几天瞎看hadoop, 但是没有数据可以用, 所以考虑用shell 在服务器上采集点天气信息
先写一个脚本用curl命令简单取一下信息
curl "http://cdn.weather.hao.360.cn/api_weather_info.php?app=hao360&_jsonp=smartloaddata101190101&code=101070201" >> /data/lzcronscripts/data.txt
可以得到如下数据
smartloaddata101190101({"pubdate":"2017-04-06","pubtime":"18:00:00","time":1491492236,"area":[["\u8fbd\u5b81","07"],["\u5927\u8fde","0702"],["\u5927\u8fde","101070201"]],"weather":[{"date":"2017-04-06","info":{"day":["0","\u6674","14","\u897f\u5317\u98ce","4-5 \u7ea7"],"night":["0","\u6674","7","\u897f\u5317\u98ce","4-5 \u7ea7"]}},{"date":"2017-04-07","info":{"dawn":["0","\u6674","7","\u897f\u5317\u98ce","4-5 \u7ea7"],"day":["1","\u591a\u4e91","14","\u5357\u98ce","4-5 \u7ea7"],"night":["1","\u591a\u4e91","8","\u5357\u98ce","4-5 \u7ea7"]}},{"date":"2017-04-08","info":{"dawn":["1","\u591a\u4e91","8","\u5357\u98ce","4-5 \u7ea7"],"day":["1","\u591a\u4e91","14","\u65e0\u6301\u7eed\u98ce\u5411","5-6 \u7ea7"],"night":["0","\u6674","8","\u5317\u98ce","4-5 \u7ea7"]}},{"date":"2017-04-09","info":{"dawn":["0","\u6674","8","\u5317\u98ce","4-5 \u7ea7"],"day":["0","\u6674","15","\u5317\u98ce","4-5 \u7ea7"],"night":["0","\u6674","8","\u5317\u98ce","4-5 \u7ea7"]}},{"date":"2017-04-10","info":{"dawn":["0","\u6674","8","\u5317\u98ce","4-5 \u7ea7"],"day":["0","\u6674","14","\u4e1c\u5357\u98ce","4-5 \u7ea7"],"night":["1","\u591a\u4e91","8","\u897f\u5357\u98ce","4-5 \u7ea7"]}}],"pm25":{"area":["\u5927\u8fde","101070201"],"pm25":[146,"2017040621"]}});
最后省点事就分隔以下逗号取个温度,用awk分隔以下
echo "a,b,c,d"|awk -F ',' '{print $1}' 输出a
最高最低气温分别是18和13
[root@VM_167_46_centos lzcronscripts]# cat ./getWeather.sh
#!/bin/sh
message=`curl "http://cdn.weather.hao.360.cn/api_weather_info.php?app=hao360&_jsonp=smartloaddata101190101&code=101070201"`
low=`echo $message|awk -F ',' '{print $13}'`
high=`echo $message|awk -F ',' '{print $18}'`
echo $message
low=${low//'"'/''};
high=${high//'"'/''};
echo $low
echo $high
time=$(date +%y%m%d);
echo $time","$low","$high >> /data/lzcronscripts/data.txt
[root@VM_167_46_centos lzcronscripts]# ./getWeather.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1250 0 1250 0 0 6718 0 --:--:-- --:--:-- --:--:-- 6756
smartloaddata101190101({"pubdate":"2017-04-06","pubtime":"18:00:00","time":1491492236,"area":[["\u8fbd\u5b81","07"],["\u5927\u8fde","0702"],["\u5927\u8fde","101070201"]],"weather":[{"date":"2017-04-06","info":{"day":["0","\u6674","14","\u897f\u5317\u98ce","4-5 \u7ea7"],"night":["0","\u6674","7","\u897f\u5317\u98ce","4-5 \u7ea7"]}},{"date":"2017-04-07","info":{"dawn":["0","\u6674","7","\u897f\u5317\u98ce","4-5 \u7ea7"],"day":["1","\u591a\u4e91","14","\u5357\u98ce","4-5 \u7ea7"],"night":["1","\u591a\u4e91","8","\u5357\u98ce","4-5 \u7ea7"]}},{"date":"2017-04-08","info":{"dawn":["1","\u591a\u4e91","8","\u5357\u98ce","4-5 \u7ea7"],"day":["1","\u591a\u4e91","14","\u65e0\u6301\u7eed\u98ce\u5411","5-6 \u7ea7"],"night":["0","\u6674","8","\u5317\u98ce","4-5 \u7ea7"]}},{"date":"2017-04-09","info":{"dawn":["0","\u6674","8","\u5317\u98ce","4-5 \u7ea7"],"day":["0","\u6674","15","\u5317\u98ce","4-5 \u7ea7"],"night":["0","\u6674","8","\u5317\u98ce","4-5 \u7ea7"]}},{"date":"2017-04-10","info":{"dawn":["0","\u6674","8","\u5317\u98ce","4-5 \u7ea7"],"day":["0","\u6674","14","\u4e1c\u5357\u98ce","4-5 \u7ea7"],"night":["1","\u591a\u4e91","8","\u897f\u5357\u98ce","4-5 \u7ea7"]}}],"pm25":{"area":["\u5927\u8fde","101070201"],"pm25":[146,"2017040621"]}});
14
7
[root@VM_167_46_centos lzcronscripts]# cat data.txt
170407,14,7
[root@VM_167_46_centos lzcronscripts]#
最后我们加个每天定时任务
crontab -e
*/1 * * * * /data/lzcronscripts/getWeather.sh >/dev/null 2>&1
不想服务器邮件文件夹被撑爆的话一定要重定向给null, 这个表达式是1分钟运行一次
如果不好用的话请确认crond服务是运行状态
0 11 * * * /data/lzcronscripts/getWeather.sh >/dev/null 2>&1 (每天11点运行一次)