ElasticSearch(二)-入门之集群搭建

环境准备

        1.三台Ubuntu(版本18.04)虚拟机IP分别为

                192.168.201.136 - node1

                192.168.201.137 - node2

                192.168.201.138 - node3

        2.准备elasticsearch-8.1.0-linux-x86_64.tar.gz压缩包

        3.关闭三台服务器防火墙(service firewalld stop)

        4.虚拟机做端口映射方便外部主机访问到虚拟机ES应用

        

        特别说明：

                1. elasticsearch版本为8.1.0,不同ES版本在配置elasticsearch.yml上语法有差异;初次搭

                    建建议使用8.1.0版本,如下相关配置均在8.1.0版本上测试过;

                 2.elasticsearch集群中节点实际上是通过“集群名称”来加入集群的。在elasticsearch.yml

                    中配置了discovery.seed_hosts(节点发现)这一项信息,在集群启动时会自动探测指定的

                    节点地址,探测到节点地址后判断是否为同一集群名称,若是则当前节点加入集群;

流程

1. 三台虚拟机分别执行解压缩和重命名操作

mkdir es-cluster # 创建存放解压后的elasticsearch文件目录
tar -xzvf elasticsearch.tar.gz # 解压缩
mv elasticsearch-8.1.0 elasticsearch # 重命名

2.因为安全问题ElasticSearch不允许root用户直接运行，所以需要在每个节点上创建新用户,

           并授权

useradd es # 新增es用户
passwd es # 为es用户设置密码
chown -R es:es /software/es-cluster # 将存放elasticsearch的目录授权给新建的es用户

补充：
# 如果创建错了可以删除该用户重新再添加
userdel -r es # 删除es用户

3.修改elasticsearch.yml文件配置(ES解压文件夹下config目录中elasticsearch.yml文件)

节点一(elasticsearch.yml配置)：

# 集群配置如下(节点一配置)
# 集群名称
 cluster.name: cluster-es
# 节点名称(每个节点名称不能重复)
 node.name: node-1
# IP地址,每个节点IP地址不能重复
 network.host: 192.168.201.136
# 是否有主节点资格;master使其有资格被选为控制集群的主节点;data此节点同时为数据节点
 node.roles: [master,data]
 http.port: 9200
# head插件需要打开这两个配置(开启跨域配置)
 http.cors.allow-origin: "*"
 http.cors.enabled: true
 http.max_content_length: 200mb
# ES7.x 之后新增的配置,初始化一个新的集群时需要此配置来选举master
 cluster.initial_master_nodes: ["192.168.201.136"]
# ES7.x 之后新增的配置,节点发现
 discovery.seed_hosts: ["192.168.201.136:9300","192.168.201.137:9300","192.168.201.138:9300"]
 network.tcp.keep_alive: true
 network.tcp.no_delay: true
# 集群内同时启动的数据任务个数,默认是2个
 cluster.routing.allocation.cluster_concurrent_rebalance: 16
# 添加或删除节点及负载均衡时并发恢复线程个数,默认4个
 cluster.routing.allocation.node_concurrent_recoveries: 16
# 初始化数据恢复时,并发恢复线程的个数,默认4个
 cluster.routing.allocation.node_initial_primaries_recoveries: 16
# 关闭安全性
 xpack.security.enabled: false

节点二(elasticsearch.yml配置)：

# 集群配置如下(节点二配置)
# 集群名称
 cluster.name: cluster-es
# 节点名称(每个节点名称不能重复)
 node.name: node-2
# IP地址,每个节点IP地址不能重复
 network.host: 192.168.201.137
# 是否有主节点资格;master使其有资格被选为控制集群的主节点;data此节点同时为数据节点
 node.roles: [master,data]
 http.port: 9200
# head插件需要打开这两个配置(开启跨域配置)
 http.cors.allow-origin: "*"
 http.cors.enabled: true
 http.max_content_length: 200mb
# ES7.x 之后新增的配置,初始化一个新的集群时需要此配置来选举master
 cluster.initial_master_nodes: ["192.168.201.136"]
# ES7.x 之后新增的配置,节点发现
 discovery.seed_hosts: ["192.168.201.136:9300","192.168.201.137:9300","192.168.201.138:9300"]
 network.tcp.keep_alive: true
 network.tcp.no_delay: true
# 集群内同时启动的数据任务个数,默认是2个
 cluster.routing.allocation.cluster_concurrent_rebalance: 16
# 添加或删除节点及负载均衡时并发恢复线程个数,默认4个
 cluster.routing.allocation.node_concurrent_recoveries: 16
# 初始化数据恢复时,并发恢复线程的个数,默认4个
 cluster.routing.allocation.node_initial_primaries_recoveries: 16
# 关闭安全性
 xpack.security.enabled: false

节点三(elasticsearch.yml配置)：

# 集群配置如下(节点三配置)
# 集群名称
 cluster.name: cluster-es
# 节点名称(每个节点名称不能重复)
 node.name: node-3
# IP地址,每个节点IP地址不能重复
 network.host: 192.168.201.138
# 是否有主节点资格;master使其有资格被选为控制集群的主节点;data此节点同时为数据节点
 node.roles: [master,data]
 http.port: 9200
# head插件需要打开这两个配置(开启跨域配置)
 http.cors.allow-origin: "*"
 http.cors.enabled: true
 http.max_content_length: 200mb
# ES7.x 之后新增的配置,初始化一个新的集群时需要此配置来选举master
 cluster.initial_master_nodes: ["192.168.201.136"]
# ES7.x 之后新增的配置,节点发现
 discovery.seed_hosts: ["192.168.201.136:9300","192.168.201.137:9300","192.168.201.138:9300"]
 network.tcp.keep_alive: true
 network.tcp.no_delay: true
# 集群内同时启动的数据任务个数,默认是2个
 cluster.routing.allocation.cluster_concurrent_rebalance: 16
# 添加或删除节点及负载均衡时并发恢复线程个数,默认4个
 cluster.routing.allocation.node_concurrent_recoveries: 16
# 初始化数据恢复时,并发恢复线程的个数,默认4个
 cluster.routing.allocation.node_initial_primaries_recoveries: 16
# 关闭安全性
 xpack.security.enabled: false

4.改动每个节点的系统配置(需改动如下三个配置文件)

vim /etc/security/limits.conf
# 末尾添加如下配置(ES可最大创建的线程)
es soft nofile 65536
es hard nofile 65536

vim /etc/security/limits.d/20-nproc.conf
# 末尾添加如下配置
es soft nofile 65536
es hard nofile 65536
* hard nproc 4096 # * 代表Linux所有用户名称

vim /etc/sysctl.conf # 限制虚拟内存
# 末尾添加如下配置
vm.max_map_count=655360

# 重新加载配置使其生效
sysctl -p

5.切换每个节点用户为第2步创建的es用户,并启动es

# 切换为es用户
su es

# 启动elasticsearch
./bin/elasticsearch

# 响应异常信息
Exception in thread "main" java.nio.file.AccessDeniedException: /software/es-cluster/elasticsearch/config/elasticsearch.keystore
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
	at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
	at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
	at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
	at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:152)
	at org.elasticsearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:244)
	at org.elasticsearch.cli.keystore.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:34)
	at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:81)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
	at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:95)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
	at org.elasticsearch.cli.Command.main(Command.java:77)
	at org.elasticsearch.cli.keystore.KeyStoreCli.main(KeyStoreCli.java:33)
Skipping security auto configuration because the node keystore file [/software/es-cluster/elasticsearch/config/elasticsearch.keystore] is not a readable regular file
Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /software/es-cluster/elasticsearch/config/elasticsearch.keystore
Likely root cause: java.nio.file.AccessDeniedException: /software/es-cluster/elasticsearch/config/elasticsearch.keystore
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
	at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
	at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
	at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
	at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:152)
	at org.elasticsearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:244)
	at org.elasticsearch.common.settings.KeyStoreWrapper.bootstrap(KeyStoreWrapper.java:209)
	at org.elasticsearch.bootstrap.BootstrapUtil.loadSecureSettings(BootstrapUtil.java:60)
	at org.elasticsearch.bootstrap.BootstrapUtil.loadSecureSettings(BootstrapUtil.java:55)
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:301)
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166)
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157)
	at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:81)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
	at org.elasticsearch.cli.Command.main(Command.java:77)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80)

 6.之所以第一次启动会出现异常信息是因为elasticsearch在第一次启动时会自动创建一些新的文件

    或目录，而这些新建的文件或目录并没有授权给es用户，所以我们需要再次为每个节点的es用

    户重新授权

chown -R es:es /software/es-cluster # 重新为es用户授权

7.再次启动所有节点,启动成功后在浏览器中访问如下地址

# 查询集群节点信息
http://192.168.201.136:9200/_cat/nodes

  

配置开机自启Elasticsearch服务

1.任意目录下编写ES开机自启脚本

        vim elasticsearchstartscript.sh

#!/bin/sh
#chkconfig: 2345 80 05
#description: elasticsearch
export JAVA_HOME=/home/software/es-cluster/elasticsearch/jdk
export JAVA_BIN=/home/software/es-cluster/elasticsearch/jdk/bin
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JAVA_HOME JAVA_BIN PATH CLASSPATH
export ES_PATH=/home/software/es-cluster/elasticsearch

case "$1" in
start)
    su es<<!
    cd $ES_PATH
    ./bin/elasticsearch -d
!
    echo "elasticsearch startup"
    ;;
stop)
    es_pid=`ps aux|grep elasticsearch | grep -v 'grep elasticsearch' | awk '{print $2}'`
    kill -9 $es_pid
    echo "elasticsearch stopped"
    ;;
restart)
    es_pid=`ps aux|grep elasticsearch | grep -v 'grep elasticsearch' | awk '{print $2}'`
    kill -9 $es_pid
    echo "elasticsearch stopped"
    su es<<!
    cd $ES_PATH
    ./bin/elasticsearch -d
!
    echo "elasticsearch startup"
    ;;
*)
    echo "start|stop|restart"
    ;;
esac

exit $?

脚本解释

2.为elasticsearchstartscript.sh脚本授权

sudo chmod +x elasticsearchstartscript.sh

3.在/etc/systemd/system/目录下新建elasticsearchstartup.service自启服务

        vim /etc/systemd/system/elasticsearchstartup.service

[Unit]
Description=es start script
# 脚本所在文件夹
ConditionPathExists=/home/software/es-cluster
[Service]
Type=forking
# 启动脚本命令
ExecStart=/home/software/es-cluster/elasticsearchstartscript.sh start
TimeoutSec=0
StandardOutput=tty
RemainAfterExit=yes
SysVStartPriority=99
[Install]
WantedBy=multi-user.target

4.启用elasticsearchstartup.service服务

sudo systemctl enable elasticsearchstartup.service

5.启动elasticsearchstartup.service服务

sudo systemctl start elasticsearchstartup.service

6.查看elasticsearchstartup.service服务状态

sudo systemctl status elasticsearchstartup.service

7.重启验证即可