需求:类似于消息推送,在一定时间段内将固定数据发送给用户(用户量很大,抛开技术实现方面问题,只分析异常)
1.异常
[2020-10-28 17:44:03] [INFO] [com.eebbk.onlinelesson.common.util.HttpUtil] [284] - No buffer space available (maximum connections reached?): connect
java.net.SocketException: No buffer space available (maximum connections reached?): connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:121)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:326)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:605)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:440)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at com.eebbk.onlinelesson.common.util.HttpUtil.doHttpGet(HttpUtil.java:264)
at com.eebbk.onlinelesson.common.util.PushMsgUtil.pushMsgWithMachineId(PushMsgUtil.java:127)
at com.eebbk.onlinelesson.Call.PushCall.call(PushCall.java:44)
at com.eebbk.onlinelesson.Call.PushCall.call(PushCall.java:21)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2.异常原因:是因为机器端口号用尽导致tcp无法申请新的端口去建立连接
代码
ExecutorService pool = new ThreadPoolExecutor(20, 20,0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
try {
List<String> list= liveTelecastService.getStatisticsByActivityKey(coursePackageId+"");
list=new ArrayList<>();
for (int i = 0; i <50000 ; i++) {
list.add("700S520072701");
}
int pushSuccess=0;
Message message = new Message();
//初始化计数器大小
CountDownLatch latch=new CountDownLatch(list.size());
List< Future<Boolean>> futureList=new ArrayList<>();
for (String machineId:list) {
PushCall pushCall = new PushCall(message,machineId,latch);
Future<Boolean> future= pool.submit(pushCall);
futureList.add(future);
}
//等待所有线程执行完成
latch.await();
for (Future<Boolean> future:futureList) {
if(future.get()){
pushSuccess++;
}
}
五万条数据,创建20个线程去请求接口获取数据,根据异常情况,查看代码,频繁new httpclient,每次都要重新建立tcp连接,完成3次握手和4次挥手。
2.修改为http连接池
@Slf4j
public class HttpClientUtils {
// 客户端从服务端读取数据的超时时间
private static final int HTTP_TIMEOUT = 2000;
// 客户端与服务器建立连接的超时时间
private static final int HTTP_CON_TIMEOUT = 200;
// 客户端从连接池中获取连接的超时时间
private static final int HTTP_CON_REQ_TIMEOUT = 1000;
// 路由的默认最大连接
private static final int HTTP_MAX_PERROUTE = 500;
// 整个连接池连接的最大值
private static final int HTTP_MAX_TOTAL = 1000;
private static RequestConfig defaultRequestConfig = null;
private static CloseableHttpClient httpClient = null;
public static CloseableHttpClient getHttpClient()
{
if(null == httpClient)
{
// 创建连接池管理器
PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
// 设置socket配置
SocketConfig socketConfig = SocketConfig.custom()
.setTcpNoDelay(true)
.build();
connManager.setDefaultSocketConfig(socketConfig);
connManager.setMaxTotal(HTTP_MAX_TOTAL);
connManager.setDefaultMaxPerRoute(HTTP_MAX_PERROUTE);
// 设置获取连接超时时间、建立连接超时时间、从服务端读取数据的超时时间
defaultRequestConfig = RequestConfig.custom()
.setSocketTimeout(HTTP_TIMEOUT)
.setConnectTimeout(HTTP_CON_TIMEOUT)
.setConnectionRequestTimeout(HTTP_CON_REQ_TIMEOUT)
.build();
// 创建httpclient实例
httpClient = HttpClients.custom()
.setConnectionManager(connManager)
.setDefaultRequestConfig(defaultRequestConfig)
.build();
}
return httpClient;
}
}
代码
问题解决,不会出现上面的异常。
后记:上面问题解决了,但是由与并发高,又出现了响应超时问题,这个只要设置连接池的超时时间就好
- Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
at com.eebbk.onlinelesson.common.util.HttpUtil.doHttpGet(HttpUtil.java:264)
at com.eebbk.onlinelesson.common.util.PushMsgUtil.pushMsgWithMachineId(PushMsgUtil.java:127)
at com.eebbk.onlinelesson.Call.PushCall.call(PushCall.java:44)
at com.eebbk.onlinelesson.Call.PushCall.call(PushCall.java:21)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)