One Cause of java.net.SocketTimeoutException: Read timed out

When I try to get document from a website using jsoup, I got the error after seconds of stucking.

java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(Unknown Source)
	at java.net.SocketInputStream.read(Unknown Source)
	at java.io.BufferedInputStream.fill(Unknown Source)
	at java.io.BufferedInputStream.read1(Unknown Source)
	at java.io.BufferedInputStream.read(Unknown Source)
	at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
	at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at java.net.HttpURLConnection.getResponseCode(Unknown Source)
	at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:453)
	at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:434)
	at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:181)
	at org.jsoup.helper.HttpConnection.get(HttpConnection.java:170)
	at com.yystar.crown.adapter.service.CrownScraper.main(CrownScraper.java:12)

Code is as simple as following:

public static void main(String[] args) {
		String url = "http://xxx.com/";
		try {
			Document doc = Jsoup.connect(url).get();
			System.out.println(doc.data());
			
		} catch (IOException e) {
			e.printStackTrace();
		}

	}

What could cause this error? Given that we can access the page normally using browser.

After comparison with browser's HTTP request header(METHODOLOGY), I set the USER AGENT and issue the request again, surprised, the server responded with the right document. The following snippet works:

public static void main(String[] args) {
		String url = "http://xxx.com/xxx";
		try {
			Document doc = Jsoup
					.connect(url)
					.followRedirects(true)
					.referrer("http://xxx.com/xxx")
					.userAgent(
							"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36")
					.get();
			System.out.println(doc.data());

		} catch (IOException e) {
			e.printStackTrace();
		}
	}
Note that the user agent value copied from browser developer tool.






  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值