ElasticSearch集群搭建及常见问题（三节点）

ElasticSearch集群搭建（三节点）

1.centos7安装了jdk1.8版本

2.下载elasticsearch6.8版本，下载地址为：https://www.elastic.co/cn/downloads/past-releases/elasticsearch-6-8-0

3.上传到三台虚拟机上，创建/etc/software,将安装文件解压到该文件夹下

4.进入安装目录，修改config文件夹下的elasticsearch.yml文件，修改内容如下（所用三台机器为192.168.16.40,192.168.16.41,192.168.16.42）

（1）40服务器参数配置：

# Use a descriptive name for the node:
#
cluster.name: test-master
node.name: test-node1
#
#设置充当master节点，默认为true
node.master: true
#
#设置不充当data节点，默认为true
node.data: false
network.host: 192.168.16.40
network.bind_host: 192.168.16.40
network.publish_host: 192.168.16.40
#
# Set a custom port for HTTP:
#
#http.port: 9200
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["192.168.16.41:9300","192.168.16.42:9300"]

（2）41服务器参数配置：

# Use a descriptive name for the node:
#
cluster.name: test-master
node.name: test-node2
#
#设置充当master节点，默认为true
node.master: false
#设置不充当data节点，默认为true
node.data: true
#network.host: 192.168.0.1
network.host: 192.168.16.41
network.bind_host: 192.168.16.41
network.publish_host: 192.168.16.41
#
# Set a custom port for HTTP:
#
#http.port: 9200
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["192.168.16.40:9300","192.168.16.42:9300"]

（3）42服务器参数配置：

#cluster.name: my-application
#
cluster.name: test-master
node.name: test-node3
#
#设置充当master节点，默认是true
node.master: false
#设置不充当data节点，默认为true
node.data: true
#network.host: 192.168.0.1
network.host: 192.168.16.42
network.bind_host: 192.168.16.42
network.publish_host: 192.168.16.42
#
# Set a custom port for HTTP:
#
#http.port: 9200
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["192.168.16.40:9300","192.168.16.41:9300"]

5.因为elasticsearch不支持root用户启动，所以创建一个普通用户来启动elasticsearch

Adduser elk
Passwd 123456

然后给elk用户授权：（在安装目录下）

chown -R elk:elk elasticsearch-6.8.0/

然后就可以切换elk用户启动elasticsearch

[root@slave01 bin]# su elk
[elk@slave01 bin]$ pwd
/etc/software/elasticsearch-6.8.0/bin
[elk@slave01 bin]$ ./elasticsearch

6.启动过程遇到的问题

1.问题一

[WARN ][o.e.b.JNANatives ] unable to install syscall filter:
Java.lang.UnsupportedOperationException: seccomp unavailable: requires kernel 3.5+ with CONFIG_SECCOMPandCONFIG_SECCOMP_FILTERcompiledinatorg.elasticsearch.bootstrap.Seccomp.linuxImpl(Seccomp.java:349) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.bootstrap.Seccomp.init(Seccomp.java:630) ~[elasticsearch-5.0.0.jar:5.0.0]

原因：只是一个警告，主要是因为Linux版本过低造成的。
解决方案：
1、重新安装新版本的Linux系统
2、警告不影响使用，可以忽略

2.问题二

ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
原因：无法创建本地文件问题,用户最大可创建文件数太小
解决方案：
切换到root用户，编辑limits.conf配置文件， 添加类似如下内容：

vi /etc/security/limits.conf

添加如下内容

*  soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096

备注：* 代表Linux所有用户名称（比如 hadoop）
保存、退出、重新登录才可生效

3.问题三

max number of threads [1024] for user [es] likely too low, increase to at least [2048]

原因：无法创建本地线程问题,用户最大可创建线程数太小
解决方案：切换到root用户，进入limits.d目录下，修改90-nproc.conf 配置文件

vi /etc/security/limits.d/90-nproc.conf

找到如下内容：

* soft nproc 1024

#修改为

* soft nproc 2048

4.问题四

max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]

原因：最大虚拟内存太小
解决方案：切换到root用户下，修改配置文件sysctl.conf

vi /etc/sysctl.conf

添加下面配置：

vm.max_map_count=655360

并执行命令：

sysctl -p

然后重新启动elasticsearch，即可启动成功。

5.问题五

ElasticSearch启动找不到主机或路由
原因：ElasticSearch 单播配置有问题
解决方案：
检查ElasticSearch中的配置文件

vi  config/elasticsearch.yml

找到如下配置：

discovery.zen.ping.unicast.hosts: ["172.16.31.220", "172.16.31.221","172.16.31.224"]  

一般情况下，是这里配置有问题，注意书写格式

6.问题六

org.elasticsearch.transport.RemoteTransportException: Failed to deserialize exception response from stream

原因:ElasticSearch节点之间的jdk版本不一致
解决方案：ElasticSearch集群统一jdk环境

7.问题七

Failed to send join request to master [{node-1}{WbcP0pC_T32jWpYvu5is1A}{2_LCVHx1QEaBZYZ7XQEkMg}{10.10.11.200}{10.10.11.200:9300}],
reason [RemoteTransportException[[node-1][10.10.11.200:9300][internal:discovery/zen/join]]; 
nested: IllegalArgumentException[can't add node {node-2}{WbcP0pC_T32jWpYvu5is1A}{p-HCgFLvSFaTynjKSeqXyA}{10.10.11.200}{10.10.11.200:9301}, 
found existing node {node-1}{WbcP0pC_T32jWpYvu5is1A}{2_LCVHx1QEaBZYZ7XQEkMg}{10.10.11.200}{10.10.11.200:9300} with the same id but is a different node instance]; ]

问题原因：要是部署的时候从一个节点复制elasticsearch文件夹，其他节点可能包含被复制节点的data文件数据，需要把data文件下的文件清空