java.net.SocketException: No buffer space available (maximum connections reached?): JVM_Bin

爬虫程序运行中出现异常，
java.net.SocketException: No buffer space available (maximum connections reached?): JVM_Bin

爬虫中使用httpclient，而且是使用了线程池。

搜了下发现，别人也有类似问题，解决办法如下，照着改了下程序，看看以后运行中是否还会出现这个问题。

 from http://9iopen.group.javaeye.com/group/blog/144545

 

 

在一个爬虫程序中遇到了以下异常:
java.net.SocketException No buffer space available (maximum connections reached?): JVM_Bind

我们知道,操作系统有它允许持有的最大文件句柄数,而在网络连接的过程中,每个socket请求都要占用一个文件句柄资源,如果没有及时释放,则可能会耗尽文件句柄资源.
通过检查代码,发现在使用HttpClient发送Get请求时没有释放资源,并且由于是多线程程序,很容易耗尽资源
改写后的代码如下:

java 代码

1. HttpClient client =  new  HttpClient();   
2. GetMethod method =  null ;   
3. try  {   
4. method =  new  GetMethod(crawlerURL.getUURI().getEscapedURI());   
5. statusCode = client.executeMethod(method);   
6. //...   
7. }   
8. finally  {   
9. if  ( null  != method)   
10. method.releaseConnection();   
11. }  

在sun的技术论坛中有一个解答是这样的:
Chances are you are forgetting to close a socket, a database connection, or some other connection that uses sockets internally. See example program below.
The alternative is that your program (or the sum of all programs running on your computer) really needs a lot of connections. In this case you'll need to find out how to increase the amount of socket buffer space that your operating system allocates. I'd start by googling for instructions. Or maybe you could redesign your program so that it doesn't use so much resources.

 

 

 

更多此问题的解答请看这里:
http://forum.java.sun.com/thread.jspa?threadID=556382