RestHighLevelClient高级客户端

系列文章目录

RestHighLevelClient粗略认识


提示:写完文章后,目录可以自动生成,如何生成可参考右边的帮助文档

 

文章目录

  • RestHighLevelClient粗略认识
  • 前言
  • 一、RestHighLevelClient是什么?
  • 二、请求处理原理(源代码)
    • 1.RestClient
    • 2.InternalHttpAsyncClient对象构造
    • 3.InternalHttpAsyncClient执行请求
    • 4.PoolingNHttpClientConnectionManager
    • 5.问题
  • 总结

 


前言

      近期在使用7.3.2 ES过程中使用到了RestHighLevelClient,也入了一些坑,对RestHighLevelClient做了下简单调研,特此记录,以供参考学习


一、RestHighLevelClient是什么?

        RestHighLevelClient是http请求高级客户端,依赖低级客户端RestClient。

public class RestHighLevelClient implements Closeable {

    private final RestClient client;
}

二、请求处理原理(源代码)

1.RestClient

public class RestClient implements Closeable {

    private static final Log logger = LogFactory.getLog(RestClient.class);

    private final CloseableHttpAsyncClient client;
}

public abstract class CloseableHttpAsyncClient implements HttpAsyncClient, Closeable;


abstract class CloseableHttpAsyncClientBase extends CloseableHttpPipeliningClient;

class InternalHttpAsyncClient extends CloseableHttpAsyncClientBase;

class MinimalHttpAsyncClient extends CloseableHttpAsyncClientBase;

本次开发使用的是InternalHttpAsyncClient类;

2、InternalHttpAsyncClient对象构造

    private final NHttpClientConnectionManager connmgr;
    public CloseableHttpAsyncClientBase(
            final NHttpClientConnectionManager connmgr,
            final ThreadFactory threadFactory,
            final NHttpClientEventHandler handler) {
        super();
        this.connmgr = connmgr;
        if (threadFactory != null && handler != null) {
            this.reactorThread = threadFactory.newThread(new Runnable() {

                @Override
                public void run() {
                    try {
                        final IOEventDispatch ioEventDispatch = new InternalIODispatch(handler);
                        connmgr.execute(ioEventDispatch);
                    } catch (final Exception ex) {
                        log.error("I/O reactor terminated abnormally", ex);
                    } finally {
                        status.set(Status.STOPPED);
                    }
                }

            });
        } else {
            this.reactorThread = null;
        }
        this.status = new AtomicReference<Status>(Status.INACTIVE);
    }
public class PoolingNHttpClientConnectionManager
       implements NHttpClientConnectionManager, ConnPoolControl<HttpRoute>;
public class DefaultConnectingIOReactor extends AbstractMultiworkerIOReactor
        implements ConnectingIOReactor

       

    PoolingNHttpClientConnectionManager连接池管理器类;
    DefaultConnectingIOReactor io事件反应器类;
    连接池管理器connmgr调用execute(IOEventDispatch eventDispatch),启动后台i/o反应器,初始化I/O event通知分发器;激活主I/O反应器和所有的worker i/o反应器;主i/o反应器:对i/o事件做反应并触发通知方法;worker i/o触发器:对i/o事件做反应并分发I/O通知给IOEventDispatch接口;

3. InternalHttpAsyncClient执行请求

         InternalHttpAsyncClient对象调用

public <T> Future<T> execute(
        final HttpAsyncRequestProducer requestProducer,
        final HttpAsyncResponseConsumer<T> responseConsumer,
        final HttpContext context,
        final FutureCallback<T> callback)

启动处理http请求。execute调用AbstractClientExchangeHandler::requestConnection方法,然后调用PoolingNHttpClientConnectionManager连接池管理器public Future<NHttpClientConnection> requestConnection(

        final HttpRoute route,
        final Object state,
        final long connectTimeout,
        final long leaseTimeout,
        final TimeUnit tunit,
        final FutureCallback<NHttpClientConnection> callback)方法处理http请求。

4. PoolingNHttpClientConnectionManager

CPool pool:

PoolingNHttpClientConnectionManager调用pool.lease()方法处理连接请求。

T:HttpRoute   C:object,用户token  E:CpoolEntry对象

private final LinkedList<LeaseRequest<T, C, E>> leasingRequests;
private final Set<SessionRequest> pending;
private final Set<E> leased;
private final LinkedList<E> available;
private final ConcurrentLinkedQueue<LeaseRequest<T, C, E>> completedRequests;
分四种情况:

1、有avaliable可用连接,则可以租到连接,leased缓存对应的CpoolEntry对象,completedRequests缓存LeaseRequest对象;

2、如果请求过了存活时间,抛出异常,completedRequests缓存LeaseRequest对象;

3、如果连接用完了,连接池最大连接数已经用完或者per route连接数已经分配完,则leasingRequests缓存LeaseRequest对象;

4、连接池最大连接数还未用完,还可以分配连接,则pending缓存SessionRequest对象;

然后处理completedRequests中的请求;

5、问题

      在此次开发中,请求并发很大,处理能力有限,导致很多请求连接积压在leasingRequests对象中,对象是一个无限双向链表,leasingRequests不断吞噬内存导致内存溢出,服务奔溃。

     在高并发开发过程中要注意此问题;

     解决方法:通过线程池限制并发请求量,防止内存溢出,这种方法有一定的缺陷,不太容易激发出异步发送的最好性能,高并发处理的性能有受损,由于并发量不好确定,目前还未找到更好的方法处理。

 

总结

     以上是ES开发RestHighLevelClient使用过程中遇到问题及相关的代码整理。还不知道这里为什么要用LinkedList来缓存请求,希望能和大家共勉学习,找到更好的解决方案。

 

 

©️2020 CSDN 皮肤主题: 深蓝海洋 设计师:CSDN官方博客 返回首页