java爬取异步数据_Java 关于抓取异步的网页数据

publicstaticStringgetHtmlCode(Stringurl,StringcharCode)throwsIOException{StringhtmlCode="";InputStreamin=null;HttpURLConnectionconnection=null;try{URLurlCon=newURL(url);c...

public static String getHtmlCode(String url,String charCode) throws IOException {

String htmlCode = "";

InputStream in = null;

HttpURLConnection connection=null;

try {

URL urlCon = new URL(url);

connection = (HttpURLConnection) urlCon.openConnection();

connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.25 Safari/534.24");

connection.setConnectTimeout(5000);

connection.setReadTimeout(5000);

connection.setFollowRedirects(true);

connection.setInstanceFollowRedirects(false);

connection.setDefaultUseCaches(false);

in = urlCon.openStream();

int index = 0;

byte[] bytes = new byte[1024*100];

int count = in.read(bytes, index, 1024 * 100);

while (count != -1) {

index += count;

count = in.read(bytes, index, 1);

}

htmlCode = new String(bytes, charCode);

} catch (Exception e) {

e.printStackTrace();

}

finally{

if(in!=null)

in.close();

if(connection!=null)

connection.disconnect();

}

return htmlCode;

}

我要抓取一个页面的数据，但是因为访问的页面采用AJAX。使我读取的数据不完全。请问下该怎么写才能读取到全部的数据

展开