ES错误记录

最新推荐文章于 2022-08-28 12:32:00 发布

lishengping_max

最新推荐文章于 2022-08-28 12:32:00 发布

阅读量714

点赞数

分类专栏： ELK 文章标签： elasticsearch

本文链接：https://blog.csdn.net/qq_23146763/article/details/70863375

版权

ELK 专栏收录该内容

6 篇文章 0 订阅

订阅专栏

一、Too many open files
1.修改linux应用可打开文件句柄数，默认是4096
/etc/security/limits.conf 增加2行
soft nofile 100001
hard nofile 100002
2.如果步骤1没解决，可能是ssh限制，修改/etc/ssh/sshd_config允许ssh登录加载ulimit的限制

二、EsRejectedExecutionException

RemoteTransportException[[elasticsearch03][xxx:9300][indices:data/write/bulk[s][p]]]; nested: EsRejectedExecutionException[rejected execution of org.elasticsearch.transport.TransportService$7@2711e339 on EsThreadPoolExecutor[bulk, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@265e05f1[Running, pool size = 2, active threads = 2, queued tasks = 200, completed tasks = 2520965]]];

原因: 说明ES索引数据的速度已经跟不上client端发送bulk请求的速度，请求队列已满以致开始拒绝新的请求。这是ES集群的自我保护机制。可以适当睡眠一段时间或者将队列设置大点。

解决办法：打开 elasticsearch.yml 在末尾加上下面配置，thread_pool.bulk.size是核心线程数，最大为CPU核心+1 ,之后重启服务

thread_pool.bulk.size: 3
thread_pool.bulk.queue_size: 1000

三、Scroll API异常

Caused by: SearchContextMissingException[No search context found for id [2861]]
    at org.elasticsearch.search.SearchService.findContext(SearchService.java:613)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:403)
    at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryScrollTransportHandler.messageReceived(SearchServiceTransportAction.java:384)
    at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryScrollTransportHandler.messageReceived(SearchServiceTransportAction.java:381)
    at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33)
    at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
    at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

原因：滚动有效时间已过，搜索上下文(Search Context)会自动被清除，会报ScrollId无效
解决办法：
1.调大有效时间setScroll(new TimeValue(120000))
2.减少批次大小setSize(3000)