Elasticsearch索引的数据存储路径是如何确定的

Elasticsearch中，在node的配置中可以指定path.data用来作为节点数据的存储目录，而且我们可以指定多个值来作为数据存储的路径，那么Elasticsearch是如何判断应该存储到哪个路径下呢？今天我就记录一下这个问题。

Elasticsearch的索引创建过程

1. 集群master收到创建索引的请求后，经过创建索引的一些步骤，最终会将索引创建完成的请求提交到ClusterState
2. master将根据ClusterState分发给所有节点
3. 涉及创建shard的节点会读取本地可用的path.data，然后依据一定的规则获取路径。
4. 创建基本shard路径，保存基本的shard信息。

如何确定在哪个目录下

源码

主要调用的是ShardPath的selectNewPathForShard方法

   for (NodeEnvironment.NodePath nodePath : env.nodePaths()) {
                totFreeSpace = totFreeSpace.add(BigInteger.valueOf(nodePath.fileStore.getUsableSpace()));
            }

            // TODO: this is a hack!!  We should instead keep track of incoming (relocated) shards since we know
            // how large they will be once they're done copying, instead of a silly guess for such cases:

            // Very rough heuristic of how much disk space we expect the shard will use over its lifetime, the max of current average
            // shard size across the cluster and 5% of the total available free space on this node:
            BigInteger estShardSizeInBytes = BigInteger.valueOf(avgShardSizeInBytes).max(totFreeSpace.divide(BigInteger.valueOf(20)));

            // TODO - do we need something more extensible? Yet, this does the job for now...
            final NodeEnvironment.NodePath[] paths = env.nodePaths();

            // If no better path is chosen, use the one with the most space by default
            NodeEnvironment.NodePath bestPath = getPathWithMostFreeSpace(env);

            if (paths.length != 1) {
                Map<NodeEnvironment.NodePath, Long> pathToShardCount = env.shardCountPerPath(shardId.getIndex());

                // Compute how much space there is on each path
                final Map<NodeEnvironment.NodePath, BigInteger> pathsToSpace = new HashMap<>(paths.length);
                for (NodeEnvironment.NodePath nodePath : paths) {
                    FileStore fileStore = nodePath.fileStore;
                    BigInteger usableBytes = BigInteger.valueOf(fileStore.getUsableSpace());
                    pathsToSpace.put(nodePath, usableBytes);
                }

                bestPath = Arrays.stream(paths)
                        // Filter out paths that have enough space
                        .filter((path) -> pathsToSpace.get(path).subtract(estShardSizeInBytes).compareTo(BigInteger.ZERO) > 0)
                        // Sort by the number of shards for this index
                        .sorted((p1, p2) -> {
                                int cmp = Long.compare(pathToShardCount.getOrDefault(p1, 0L),
                                    pathToShardCount.getOrDefault(p2, 0L));
                                if (cmp == 0) {
                                    // if the number of shards is equal, tie-break with the number of total shards
                                    cmp = Integer.compare(dataPathToShardCount.getOrDefault(p1.path, 0),
                                            dataPathToShardCount.getOrDefault(p2.path, 0));
                                    if (cmp == 0) {
                                        // if the number of shards is equal, tie-break with the usable bytes
                                        cmp = pathsToSpace.get(p2).compareTo(pathsToSpace.get(p1));
                                    }
                                }
                                return cmp;
                            })
                        // Return the first result
                        .findFirst()
                        // Or the existing best path if there aren't any that fit the criteria
                        .orElse(bestPath);
            }

            statePath = bestPath.resolve(shardId);
            dataPath = statePath;
        }

过程分析

1. 首先判断是否自定义了path.data，没有自定义就在默认路径下创建
2. 自定义的情况下确保节点下最少有5%的空间可以使用
3. 获取所有的paths，
4. 然后设置默认最佳的path是当前拥有最多空间的path
5. 遍历所有的paths，首先过滤掉没有空间的path，如果最终没有符合的，就返回4步骤的path，否则继续6步骤
6. 按照规则对paths排序，首先判断每个path下该索引的shard数，优先返回含有本索引的shard数最少的path；
   当条件结果相同，对比每个path中包含有的shard总数（所有索引的），返回包含shard数最少的path；
   当2条件结果相同，对比可用空间，返回可用空间最大的path
7. 生成相应的路径，创建目录等信息。