Jsoup抓取网页内容超时问题的处理

最新推荐文章于 2024-06-28 16:55:19 发布

指尖de柔情

最新推荐文章于 2024-06-28 16:55:19 发布

阅读量6.2k

点赞数

分类专栏： Jsoup网络爬虫文章标签： jsoup 超时

本文链接：https://blog.csdn.net/u013456370/article/details/79169831

版权

Jsoup网络爬虫专栏收录该内容

2 篇文章 0 订阅

订阅专栏

在利用Jsoup抓取网页的时候，遇到超时问题：

java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(Unknown Source)
	at java.net.SocketInputStream.read(Unknown Source)
	at java.io.BufferedInputStream.fill(Unknown Source)
	at java.io.BufferedInputStream.read1(Unknown Source)
	at java.io.BufferedInputStream.read(Unknown Source)
	at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
	at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at java.net.HttpURLConnection.getResponseCode(Unknown Source)
	at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:655)
	at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:628)
	at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:260)
	at org.jsoup.helper.HttpConnection.get(HttpConnection.java:249)
	at com.accord.jsoup.JsoupUtils.getHtmlByUrl2(JsoupUtils.java:70)
	at com.accord.jsoup.JsoupMain.getListDatas(JsoupMain.java:63)
	at com.accord.jsoup.JsoupMain.main(JsoupMain.java:32)
Exception in thread "main" java.lang.NullPointerException
	at com.accord.jsoup.JsoupMain.parserHtml(JsoupMain.java:169)
	at com.accord.jsoup.JsoupMain.getListDatas(JsoupMain.java:64)
	at com.accord.jsoup.JsoupMain.main(JsoupMain.java:32)

我一开始代码里面是：

Jsoup.connect(url).cookie("JSESSIONID", sessionId).get();

方法获取网页内容，这种方式使用的是默认超时时间30秒。

由于请求数据慢，就会造成超时的，修改成如下就可以了：

Jsoup.connect(url).cookie("JSESSIONID", sessionId).timeout(50000).get(); // 50S

这个时间，自己看着设置！