最近做微信公共平台开发,用到html、xml和JSON解析,特在这里总结一下。
解析html的方法有以下几种:
1.传统JDK自带的URLConnection
2.使用httpclient
使用传统JDK自带的URLConnection解析html步骤:
1、发起http、get请求,获得html网页
//不需要特别的引入包public static String getHtml(String requestUrl){
StringBuffer buffer = null;
try{
//1、建立连接
//创建url对象
URL url = new URL(requestUrl);
//利用HttpURLConnection对象获取网页数据
HttpURLConnection httpUrlConn = (HttpURLConnection)url.openConnection();
httpUrlConn.setDoInput(true);
httpUrlConn.setRequestMethod("GET");
//2.获得输入流
InputStream inputStream = httpUrlConn.getInputStream();
//以网页编码utf-8进行读入
InputStreamReader inputStreamReader = new InputStreamReader(inputStream,"utf-8");
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
//3、读取返回结果
buffer = new StringBuffer();
String str = null;
while((str=bufferedReader.readLine())!=null){
System.out.println(str);
buffer.append(str);
}
//4.释放资源
bufferedReader.close();
inputStreamReader.close();
inputStream.close();
httpUrlConn.disconnect();
}catch(Exception e){
e.printStackTrace();
}
return buffer.toString();
}
2、使用httpclient:
//需要引入httpclient-4.3.5和httpclient-core和commons-logging
public static String getHtml2(String requestUrl){
String html="未获得网页";
try{
//1.建立httpclient对象
CloseableHttpClient httpClient = HttpClients.createDefault();
//2.创建请求方法的实例,并指定请求URL。如果需要发送GET请求,创建HttpGet对象;如果需要发送POST请求,创建HttpPost对象。
HttpGet httpGet = new HttpGet(requestUrl);
//3.调用HttpClient对象的execute(HttpUriRequest request)发送请求,该方法返回一个HttpResponse。
CloseableHttpResponse response = httpClient.execute(httpGet);
//4.调用HttpResponse的getEntity()方法可获取HttpEntity对象,该对象包装了服务器的响应内容。
html = EntityUtils.toString(response.getEntity());
return html;
}catch(Exception e){
e.printStackTrace();
}
return html;
}
解析html,获得需要的数据
这也有几种方法:正则匹配、htmlparser、jsoup
1.采用正则匹配:
- private static String parseHtml(String html) {
- StringBuffer buffer = null;
- // 日期标签:区分是昨天还是今天
- String dateTag = getMonthDay(0);
- Pattern p = Pattern.compile("(.*)(<div class=\"listren\">)(.*?)(</div>)(.*)");
- Matcher m = p.matcher(html);
- if (m.matches()) {
- buffer = new StringBuffer();
- if (m.group(3).contains(getMonthDay(-1)))
- dateTag = getMonthDay(-1);
- // 拼装标题
- buffer.append("≡≡ ").append("历史上的").append(dateTag).append(" ≡≡").append("\n\n");
- // 抽取需要的数据
- for (String info : m.group(3).split(" ")) {
- info = info.replace(dateTag, "").replace("(图)", "").replaceAll("</?[^>]+>", "").trim();
- // 在每行末尾追加2个换行符
- if (!"".equals(info)) {
- buffer.append(info).append("\n\n");
- }
- }
- }
- // 将buffer最后两个换行符移除并返回
- return (null == buffer) ? null : buffer.substring(0, buffer.lastIndexOf("\n\n"));
- }
2.采用jsop
//解析获得网页源码,获得想要的数据
public static List<String> parseHtml(String html){
List list = new ArrayList<String>();
//获得网页对象
Document document = Jsoup.parse(html,"gbk");
Elements elements = document.getElementsByAttributeValue("class", "listren");
Elements elements2 = elements.select("a[title]");
int i=0;
for(org.jsoup.nodes.Element element:elements2){
System.out.println(elements2.eq(i).text());
list.add(elements2.eq(i).text());
i++;
}
return list;
}