天气预报抓取

原创 2010年05月19日 15:30:00

看了很多网站需要显示天气预报,于是我决心整理下代码。

大部分网站是调用的天气预报插件,这样给我们带来很多不便。比如,不能上互联网,只能在局域网使用的,根本就连不上服务器,这时天气预报那个地方就显示“无法显示网页”,很不美观;你只能用他们的图片,不能用自己的图片等。我的思路是把他们的数据定时抓取到数据库中,然后程序读数据库即可。当然得先把他们的图片下载下来。

我用的是webClient方法将文件从他网页下载到本地。

 WebClient myWebClient = new WebClient();

int i=0;

for(;i<32;i++){
            myWebClient.DownloadFile(uriString, "D://c"+i+".gif");

}

共下载了32个图片,从c0.gif到c31.gif.

然后写一个数据抓取的exe

Form1.cs:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Net;
using System.Xml;
using System.IO;
using System.Data.OracleClient;
using System.Configuration;
using System.Text.RegularExpressions;

namespace 天气预报信息抓取
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }
        private string GetWebContent(string Url, Encoding encoding)
        {
            string strResult = "";
            try
            {
                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(Url);
                //声明一个HttpWebRequest请求
                request.Timeout = 90000;
                //设置连接超时时间
                request.Headers.Set("Pragma", "no-cache");
                HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                Stream streamReceive = response.GetResponseStream();
                //Encoding encoding = Encoding.UTF8;

                StreamReader streamReader = new StreamReader(streamReceive, encoding);
                strResult = streamReader.ReadToEnd();
                streamReceive.Dispose();
                streamReader.Dispose();

            }
            catch (Exception ex)
            {
                //将错误信息写到日志文件.txt中
                AppLog.WirteLog(ex.Message);
            }

            return strResult;
        }
        private void Form1_Load(object sender, EventArgs e)
        {
            try
            {
                DataSet ds = new DataSet();
                ds.ReadXml("WeatherUrl.xml");
                DataTable dt = ds.Tables[0];
                for (int i = 0; i < dt.Rows.Count; i++)
                {
                    getWeather(i, dt.Rows[i]["city"].ToString(), dt.Rows[i]["url"].ToString());
                }

                this.Close();
                this.Dispose();
            }
            catch (Exception ex)
            {
                //将错误信息写到日志文件.txt中
                AppLog.WirteLog(ex.Message);
            }
        }

        private void getWeather(int xh, string city,string weburl)
        {
            try
            {
                //得到指定Url的源码
                string strWebContent = GetWebContent(weburl, Encoding.UTF8);

                string cnnstr = System.Configuration.ConfigurationSettings.AppSettings["cstr"];
                OracleConnection dbconn = new OracleConnection(cnnstr);

                if (dbconn.State == ConnectionState.Open)
                {
                    dbconn.Close();
                }
                dbconn.Open();
                string deletesql = "delete from weather where xh='" + xh + "'";
                OracleCommand myCommand1 = new OracleCommand(deletesql, dbconn);
                myCommand1.ExecuteNonQuery();

                string insertsql = "insert into weather(xh,info,city) values('" + xh + "','" + strWebContent + "','" + city + "')";
                OracleCommand myCommand2 = new OracleCommand(insertsql, dbconn);
                myCommand2.ExecuteNonQuery();
                dbconn.Close();
            }
            catch (Exception ex)
            {
                //将错误信息写到日志文件.txt中
                AppLog.WirteLog(ex.Message);
            }
        }

    }
}
一个写日志的类AppLog.cs

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Net;
using System.IO;
using System.Data.OracleClient;
using System.Configuration;

namespace 天气预报信息抓取
{
    class AppLog
    {
        private static string LocalLogFilePath = Application.StartupPath + "//AppLog";

        public AppLog()
        {

        }
        // 判断日志文件是否存在
        public static bool CheckFile(string Path)
        {
            if (!File.Exists(Path))
            {
                try
                {
                    FileStream fs = File.Create(Path);
                    fs.Close();
                    fs.Dispose();

                }
                catch
                {
                    return false;
                }

            }
            return true;
        }

        //检测日志文件夹是否存在
        public static bool CheckDectoryExists(string DectoryName)
        {
            if (Directory.Exists(DectoryName))
            {
                return true;
            }
            else
            {
                try
                {
                    Directory.CreateDirectory(DectoryName);
                }
                catch
                {
                    return false;
                }

            }
            return true;
        }

        //删除日志文件
        public static void DeleteAppLogFile(DateTime LastIime)
        {
            try
            {
                if (Directory.Exists(LocalLogFilePath))
                {
                    foreach (string newstring in Directory.GetFiles(LocalLogFilePath))
                    {
                        if (Path.GetExtension(newstring).ToLower() == ".txt")
                        {
                            if (DateTime.Compare(LastIime, File.GetLastWriteTime(newstring)) > 0)
                            {
                                File.Delete(newstring);
                            }
                        }

                    }

                }
            }
            catch (Exception ex)
            {
                WirteLog("删除日志文件出错:" + ex.Message);
            }

        }

        //写日志文件
        public static void WirteLog(string msg)//向日志文件中插入数据
        {
            if (!CheckDectoryExists(LocalLogFilePath))
                return;
            string FileName = DateTime.Now.Year.ToString() + "年" + DateTime.Now.Month.ToString() + "月" + DateTime.Now.Day + "日.txt";
            FileName = LocalLogFilePath + "//" + FileName;

            if (!CheckFile(FileName))
                return;

            FileStream fs = new FileStream(FileName, FileMode.Append);
            if (File.Exists(FileName))
            {
                int i = 0;
                while (fs.Length > 2000000)//2M
                {
                    fs.Close();
                    FileName = DateTime.Now.Year.ToString() + "年" + DateTime.Now.Month.ToString() + "月" + DateTime.Now.Day + "日" + i.ToString() + ".txt";
                    if (File.Exists(FileName))
                    {
                        i = i + 1;
                        if (!CheckFile(FileName))
                            return;
                        fs = new FileStream(FileName, FileMode.Append);
                    }
                    else
                    {
                        if (!CheckFile(FileName))
                            return;
                        fs = new FileStream(FileName, FileMode.Append);
                        break;
                    }
                }
            }

            StreamWriter sw = new StreamWriter(fs);
            msg = DateTime.Now.ToString() + ":" + msg;
            sw.WriteLine(msg);
            sw.Close();
            fs.Close();
            sw.Dispose();
            fs.Dispose();
        }

        public static void WirteLog(string logFilePath, string msg)
        {
            string fullpath = Path.Combine(Application.StartupPath, logFilePath);
            if (!CheckFile(fullpath))
                return;
            System.IO.FileStream fs = new FileStream(fullpath, FileMode.Append);
            System.IO.StreamWriter sw = new StreamWriter(fs, System.Text.Encoding.GetEncoding("gb2312"));
            msg = DateTime.Now.ToString() + ":" + msg;
            sw.WriteLine(msg);
            sw.Close();
            fs.Close();
            sw.Dispose();
            fs.Dispose();
        }

        // 获得本机局域网IP地址
        private static string getIPAddress()
        {
            System.Net.IPAddress addr = new System.Net.IPAddress(System.Net.Dns.GetHostEntry(System.Net.Dns.GetHostName()).AddressList[0].Address);
            return addr.ToString();
        }
    }
}

xml文件WeatherUrl.xml:配置抓取某个城市的天气数据网页

<?xml version="1.0" encoding="utf-8" ?>
<DataSet>
    <weather>
      <city>潜江</city>
      <url>http://m.weather.com.cn/data/101201701.html</url>
    </weather>
</DataSet>

注意生成exe的时候要配置数据库的连接信息,比如你的exe名叫aa.exe,你生成的配置文件就叫aa.exe.config,

里面的内容格式如下

<configuration>
  <appSettings>
      <add key="cstr" value="xxxxxx"></add>
  </appSettings>
</configuration>

工作进行得差不多了,将你做好的exe还有所需文件放到服务器上。

关键是怎么让它定时执行呢?

添加任务计划。到控制面板里面,打开任务计划,找到你那个exe,设定早晨8:05和中午12:05,下午18:05执行一次,因为中国天气网在早8:00,中午12:00和下午18:00更新数据。

好了,数据库设计如下:

create table WEATHER
(
  XH   CHAR(1) not null,
  INFO VARCHAR2(2000) not null,
  CITY VARCHAR2(20) not null
)

-- Add comments to the table
comment on table WEATHER
  is '天气预报数据表';
-- Add comments to the columns
comment on column WEATHER.XH
  is '序号';
comment on column WEATHER.INFO
  is '天气数据';
comment on column WEATHER.CITY
  is '城市';

 

这样后台就ok了。注意,info里面是个JSON数据,比如{"weatherinfo":{"city":"长春","city_en":"changchun","date_y":"2010年3月22日","date":"庚寅年二月初七","week":"星期一","fchh":"08","cityid":"101060101","temp1":"0℃~-8℃","temp2":"1℃~-9℃","temp3":"0℃~-12℃","temp4":"-2℃~-13℃","temp5":"-2℃~-13℃","tempF1":"32℉~17.6℉","tempF2":"33.8℉~15.8℉","tempF3":"32℉~10.4℉","tempF4":"28.4℉~8.6℉","tempF5":"28.4℉~8.6℉","weather1":"小到中雪转多云","weather2":"多云","weather3":"晴","weather4":"晴","weather5":"晴","img1":"26","img2":"1","img3":"1","img4":"99","img5":"0","img6":"99","img7":"0","img8":"99","img9":"0","img10":"99","img_single":"26","img_title1":"小到中雪","img_title2":"多云","img_title3":"多云","img_title4":"多云","img_title5":"晴","img_title6":"晴","img_title7":"晴","img_title8":"晴","img_title9":"晴","img_title10":"晴","img_title_single":"小到中雪","wind1":"西风转西北风3-4级","wind2":"西北风小于3级","wind3":"西北风转西风小于3级","wind4":"西风小于3级","wind5":"西风转西南风小于3级","fx1":"西风","fx2":"西北风","fl1":"3-4级","fl2":"小于3级","fl3":"小于3级","fl4":"小于3级","fl5":"小于3级","index":"暂缺","index_d":"暂缺","index48":"暂缺","index48_d":"暂缺","index_uv":"最弱","index48_uv":"最弱","index_xc":"不宜","index_tr":"一般","index_co":"较不舒适","st1":"-1","st2":"-11","st3":"1","st4":"-7","st5":"-1","st6":"-10"}}。数据有点多,呵呵。

 

下边的工作就是做显示页了。

 

前台aspx:

<%@ Page Language="C#" AutoEventWireup="true" CodeBehind="weather.aspx.cs" Inherits="采集辅助管理系统.weather" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" >
<head>
    <title>天气预报</title>
    <script src="<%=JSPath %>ShareScript.js" type="text/javascript"> </script>
     <style type="text/css">
    <!--
    body
    {
        margin:0px;
        font-size: 12px;
     color: #000000;
     text-decoration: none;
 }
    a
    {
     color:#000;
     text-decoration:none;
 }
    a:hover
    {
     color:#FF6600;
     text-decoration:underline;
 }
    em
    {
     font-style:normal;
 }
 ul.row
    {
     clear:both;
  margin:0px;
 }
    ul.row li
    {
     float:left;
     margin:0 4px;
 }
    ul.row li a
    {
     display:inline;
     height:22px;
     line-height:22px;
 }
 img{border:none;}
    -->
</style>
</head>
<body>
    <form id="form1" runat="server">
    <ul class="row">
  <li><a id="url1" href="#" target="_blank"><em id="city"></em></a></li>
  <li><a id="url2" href="#" target="_blank"><img id="small" alt="天气图标"  /></a></li>
  <li><a id="url3" href="#" target="_blank"><em id="temp1"></em></a></li>
  <li><a id="url4" href="#" target="_blank"><em id="wd1"></em></a></li>
  <li><a id="url5" href="#" target="_blank">七天预报</a></li>
</ul>
    </form>
    <script language="javascript" type="text/javascript">
        /*
        功能:设置每个超级链接的地址
        */
        for (i = 0; i < document.getElementsByTagName("a").length; i++) {
            document.getElementsByTagName("a")[i].href = "http://www.weather.com.cn/html/weather/101201701.shtml";
        }
        var JsonString = '<% =WhetherInfo%>';
        if (JsonString != "") {
            var JSONobj = eval("(" + JsonString + ")");
            var weobj = JSONobj.weatherinfo;
            $("city").innerText = weobj.city; //城市
            $("small").src = "../images/weather/c" + weobj.img1 + ".gif"; //天气图片
            $("temp1").innerText = weobj.temp1.replace(/~/g, "~"); //温度
            $("wd1").innerText = weobj.wind1; //风力
        }
        else
            document.body.innerHTML = "<div style='text-align:center'><img src='../images/warn.gif' width='21' height='21' style='position:relative;top:5px;margin-right:2px;'>无数据,请检查服务器任务计划设置!</div>"
    </script>
</body>
</html>

后台cs:

using System;
using System.Collections.Generic;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Data;

namespace 采集辅助管理系统
{
    public partial class weather : Com.HanWeiKeJi.WebBase.SubSystem
 {
        public string WhetherInfo = "";

        protected void Page_Load(object sender, EventArgs e)
        {
            try
            {
                this.g_subsysid = Request.QueryString["g_subsysid"];
                this.g_moduleGuid = Request.QueryString["g_moduleGuid"];

                setString();
            }
            catch (Exception ex)
            {
                Response.Write("Page_Load():" + ex.Message);
            }
        }
        /// <summary>
        /// 设置天气参数
        /// </summary>
        private void setString()
        {
            string StrSql = "select info from weather where xh='0'";
            Object o = this.SubSysDbObj.ExecuteScalar(StrSql);
            if (o != null)
                WhetherInfo = this.SubSysDbObj.ExecuteScalar(StrSql).ToString();
        }
 }
}
呵呵。这样就大功告成了!里面有些类我没法公布了,比如Com.HanWeiKeJi,那是我们公司写的基类,这里就不公布了,呵呵。

祝你好运!

Java实现抓取相应地区中央天气预报

先上图:程序一开始运行默认的是显示本地的天气(界面好难看咯)当获取本地地址失败的时候,程序默认显示北京的天气,可以在上面的下拉列表里面选择相应的城市来获取相应城市的天气。全国各地区都是有编码的,所以当...

天气预报之抓取、解析、存入MYSQL数据库模块实现,不规范(代码)

package com.zzk.cn; import java.io.BufferedReader; import java.io.IOException; import java.io.I...

Android demo-->网易新闻风格的RSS新闻抓取项目(二) 新闻列表刷新、天气预报、设置页面滑动开关

新闻列表刷新 天气预报新闻列表刷新天气预报

jsp网页抓取天气预报源代码

  • 2009年06月02日 17:31
  • 5KB
  • 下载

scrpay 和 pyv8解析js,抓取天气预报数据

github项目地址:https://github.com/robertzhai/python/tree/master/scrapy/weather # -*- coding:utf-8 -*-...

天气预报程序-抓取和文件读写

package com.zzk.cn; import java.io.BufferedReader; import java.io.IOException; import java.io.I...

天气预报抓取的方法和源代码(包括从IP获取)

天气预报抓取的方法和源代码(包括从IP获取)
  • hsg77
  • hsg77
  • 2011年06月01日 00:46
  • 2455

天气预报抓取的方法和源代码(包括从IP获取)

天气预报抓取的方法和源代码(包括从IP获取) 作者: 毛虫    文章来源:www.cnblogs.com    发布日期:2009-7-22 11:14:21    浏览次数:16...

天气预报抓取、解析(传递抓取的info参数)功能模块实现(代码)

package com.zzk.cn; import java.io.BufferedReader; import java.io.IOException; import java.io.I...
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:天气预报抓取
举报原因:
原因补充:

(最多只允许输入30个字)