ES版本升级后出现Trying to create too many scroll contexts. Must be less than or equal to: [500]异常

从一个异常说起：

[I/O dispatcher 79] WARN RestClient - request [POST http://xx.xx.xxx.xxx:8080/index/_search?scroll=600s] returned 1 warnings: [299 Elasticsearch-6.8.5-78990e9 "Trying to create more than 500 scroll contexts will not be allowed in the next major version by default. You can change the [search.max_open_scroll_context] setting to use a greater default value or lower the number of scrolls that you need to run in parallel."]

 

1.当前版本6.8.5，这个异常的描述是scroll快照太多，同时存在的context数量超过500导致异常。（首先肯定是使用的问题，但是之前的5.3.2版本为什么没异常？该如何解决？）

2.先看下5.3.2的源码，如果是查询是scroll，则进行如下操作：

发现5.3.2版本的默认值是Integer的最大值，如果大于500则一个warn，大于Integer最大值才抛异常

所以也就解释了为什么之前版本不出问题

3.再看一下6.8.5源码：

final SearchContext createAndPutContext(ShardSearchRequest request) throws IOException {
        if (request.scroll() != null && openScrollContexts.get() >= maxOpenScrollContext) {
            throw new ElasticsearchException(
                "Trying to create too many scroll contexts. Must be less than or equal to: [" +
                    maxOpenScrollContext + "]. " + "This limit can be set by changing the ["
                    + MAX_OPEN_SCROLL_CONTEXT.getKey() + "] setting.");
        }

        SearchContext context = createContext(request);
        boolean success = false;
        try {
            putContext(context);
            if (request.scroll() != null) {
                openScrollContexts.incrementAndGet();
                context.indexShard().getSearchOperationListener().onNewScrollContext(context);
            }
            context.indexShard().getSearchOperationListener().onNewContext(context);
            success = true;
            return context;
        } finally {
            if (!success) {
                freeContext(context.id());
            }
        }
    }

maxOpenScrollContext = MAX_OPEN_SCROLL_CONTEXT.get(settings);

public static final Setting<Integer> MAX_OPEN_SCROLL_CONTEXT =
    Setting.intSetting("search.max_open_scroll_context", 500, 0, Property.Dynamic, Property.NodeScope);

所以，maxOpenScrollContext=500，所以到500就报错了

 

 

今天又看到一个问题：

Trying to create too many scroll contexts. Must be less than or equal to: [500]

也是版本升级5->7之后就报错了

ES7版本，7版本每次scroll请求使用的scrollId是同一个（7版本一下，一次scroll中每次查都生成不同的scrollId），所以不是ID的问题

本质原因和以上分析一致