阿里云OSS线程增长问题分析

最新推荐文章于 2023-09-20 21:51:55 发布

瓜头居

最新推荐文章于 2023-09-20 21:51:55 发布

阅读量1.1k

点赞数 1

文章标签：阿里云云计算 java

本文链接：https://blog.csdn.net/lgd0602/article/details/122342742

版权

线上出现运行几个小时后，阿里云OSS无法进行操作问题，分析发现线程数线性增长，很快到达最大线程数，最后分析代码/查找OSS文档发现是使用OSSObject时没有close导致的，因为OSSObject封装了inputStream，未关闭会导致http连接一直处于open状态，无法进行回收，最后到达最大连接，彻底无法访问OSS。

随后对OSSClient代码进行分析，看下为何会线程线性增长；

1，查看OSSClient定义

public OSSClient(String endpoint, CredentialsProvider credsProvider, ClientConfiguration config) {
        this.credsProvider = credsProvider;
        config = config == null ? new ClientConfiguration() : config;
        if (config.isRequestTimeoutEnabled()) {
            this.serviceClient = new TimeoutServiceClient(config);
        } else {
            //新建DefaultServiceClient对象
            this.serviceClient = new DefaultServiceClient(config);
        }
        initOperations();
        setEndpoint(endpoint);
    }

2，查看DefaultServiceClient，定义了httpclient包里的PoolingHttpClientConnectionManager（http连接池管理器），并单独起线程进行连接回收策略

public DefaultServiceClient(ClientConfiguration config) {
        super(config);
        //使用了httpClient包里的连接管理器，重点看这里
        this.connectionManager = createHttpClientConnectionManager();
        this.httpClient = createHttpClient(this.connectionManager);
        RequestConfig.Builder requestConfigBuilder = RequestConfig.custom();
        //连接超时时间
        requestConfigBuilder.setConnectTimeout(config.getConnectionTimeout());
        requestConfigBuilder.setSocketTimeout(config.getSocketTimeout());
        requestConfigBuilder.setConnectionRequestTimeout(config.getConnectionRequestTimeout());
        
        //。。。。。省略代码
        this.requestConfig = requestConfigBuilder.build();
    }

protected HttpClientConnectionManager createHttpClientConnectionManager() {
        //。。。。省略代码
        Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
                .register(Protocol.HTTP.toString(), PlainConnectionSocketFactory.getSocketFactory())
                .register(Protocol.HTTPS.toString(), sslSocketFactory)
                .build();
        //使用httpclient的http连接池管理器
        PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);
        //设置最大连接，默认1024
        connectionManager.setDefaultMaxPerRoute(config.getMaxConnections());
        connectionManager.setMaxTotal(config.getMaxConnections());
        connectionManager.setValidateAfterInactivity(config.getValidateAfterInactivity());
        connectionManager.setDefaultSocketConfig(SocketConfig.custom().
                setSoTimeout(config.getSocketTimeout()).setTcpNoDelay(true).build());
        if (config.isUseReaper()) {
            //起新线程执行连接回收策略，重点关注
            IdleConnectionReaper.setIdleConnectionTime(config.getIdleConnectionTime());
            IdleConnectionReaper.registerConnectionManager(connectionManager);
        }
        return connectionManager;
    }

3，连接回收策略，定时去遍历连接管理器，然后调用httpclient的连接回收策略。

 public void run() {
        while (true) {
            if (shuttingDown) {
                getLog().debug("Shutting down reaper thread.");
                return;
            }
            
            //每次休眠五秒
            try {
                Thread.sleep(REAP_INTERVAL_MILLISECONDS);
            } catch (InterruptedException e) {
            }
            
            try {
                List<HttpClientConnectionManager> connectionManagers = null;
                synchronized (IdleConnectionReaper.class) {
                    connectionManagers = (List<HttpClientConnectionManager>)IdleConnectionReaper.connectionManagers.clone();
                }
                //遍历所有连接管理器
                for (HttpClientConnectionManager connectionManager : connectionManagers) {
                    try {
                        //调用httpclient的回收连接方法，回收过期连接与超时空闲连接
                        connectionManager.closeExpiredConnections();
                        connectionManager.closeIdleConnections(idleConnectionTime, TimeUnit.MILLISECONDS);
                    } catch (Exception ex) {
                        getLog().warn("Unable to close idle connections", ex);
                    }
                }
            } catch (Throwable t) {
                getLog().debug("Reaper thread: ",  t);
            }
        }
    }

4，总结，OSSClient实际是基于httpclient进行接口调用，并使用PoolingHttpClientConnectionManager进行连接管理，单独起线程进行连接的定时回收。