elasticsearch 异常

最新推荐文章于 2024-08-06 07:35:25 发布

ZJH2017b

最新推荐文章于 2024-08-06 07:35:25 发布

阅读量1k

点赞数

分类专栏： elasticsearch 文章标签： elasticsearch

本文链接：https://blog.csdn.net/ZJH2017b/article/details/121602780

版权

elasticsearch 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

max_open_scroll_context

异常日志：

{“shard”:0,“index”:"*_default_202111",“node”:“FjBZ-0PxQc-rvrTtYPmE1Q”,“reason”:{“type”:“exception”,“reason”:“Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting.”}}]},“status”:500}

解决思路：

日志出现在凌晨的统计功能上，该功能需要对es进行查询操作，且该操作为多线程执行
查询es的操作中，使用了 scroll 滚动机制，而从异常日志中可以判定应该是那一个瞬间的 scroll 超过es 的默认值500，但由于代码中已经设置了scroll的过期时间为1分钟，那可以尝试根据提示直接设置max_open_scroll_context 的值

/**
 * 查询
 *
 * @param queryBuilder 查询条件
 * @param sortBuilder  排序条件
 * @return 查询结果
 */
protected List<T> search(QueryBuilder queryBuilder, SortBuilder sortBuilder) {
    List<T> lstData = new ArrayList<>();

    try {
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(queryBuilder);
        sourceBuilder.size(10000);
        Optional.ofNullable(sortBuilder).ifPresent(sourceBuilder::sort);
        final Scroll scroll = new Scroll(TimeValue.timeValueMinutes(1L));
        SearchRequest searchRequest = new SearchRequest();
        searchRequest.indices(getIndexAlias());
        searchRequest.scroll(scroll);
        searchRequest.source(sourceBuilder);

        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        String scrollId = searchResponse.getScrollId();
        SearchHit[] searchHits = searchResponse.getHits().getHits();

        while (searchHits != null && searchHits.length > 0) {
            for (SearchHit hit : searchHits) {
                lstData.add(fromMap(hit.getSourceAsMap()));
            }

            SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
            scrollRequest.scroll(scroll);
            searchResponse = client.scroll(scrollRequest, RequestOptions.DEFAULT);
            scrollId = searchResponse.getScrollId();
            searchHits = searchResponse.getHits().getHits();
        }

        ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
        clearScrollRequest.addScrollId(scrollId);
        client.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
    } catch (Exception e) {
        throw new RuntimeException("elasticsearch search error:" + e.getMessage(), e);
    }
    return lstData;
}

将 max_open_scroll_context值设置为2000，后续跟踪日志后发现问题已经解决

PUT _cluster/settings
{
  "persistent": {
    "search.max_open_scroll_context": 2000
  },
  "transient": {
    "search.max_open_scroll_context": 2000
  }
}