由于网页内容的字符集编码不一定都采用UTF-8编码,所以通过HttpURLConnection获取的网页内容经常会出现乱码的问题。 网页内容的编码可能是UTF-8,也可能是GBK、GB2312,甚至其它编码方式。
从下面的截图可以看出,服务器会在HTTP头里面包含原始字符集编码信息,我们可以通过URLConnection类的getContentType()方法的返回值,然后用正则表达式从中提取出编码。
public String requestWebContenFromUrl(String urlStr) {
String result = "";
try {
URL url = new URL(urlStr);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setConnectTimeout(5 * 1000);
if (conn.getResponseCode() == 200) {
InputStream is = conn.getInputStream();
String charset = "UTF-8";
Pattern pattern = Pattern.compile("charset=\\S*");
Matcher matcher = pattern.matcher(conn.getContentType());
if (matcher.find()) {
charset = matcher.group().replace("charset=", "");
}
BufferedReader reader = new BufferedReader(new InputStreamReader(is, charset));
StringBuilder sb = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
result = sb.toString();
}
} catch (Exception e) {
e.printStackTrace();
}
return result;
}