(十三) ES7.4集群搭建+Kibana安全认证登录

本文为学习笔记，主要用于记录本人学习过程。部分内容为转载！！！！.

一、ES集群搭建

0.前期准备

三台linux服务器

1.下载相关软件包

到官网上下载最新版本ElasticSearch 7.4, https://www.elastic.co/cn/downloads/elasticsearch

2.修复系统配置

vi /etc/security/limits.conf
新增内容如下：
* hard nofile 65536
* soft nproc 2048
* hard nproc 4096
* soft memlock unlimited
* hard memlock unlimited

vi /etc/sysctl.conf
新增内容如下：
vm.max_map_count=655360
fs.file-max=655360
执行 sysctl -p 让其生效

3.安装软件

[book@BOOK-CENTOS01 soft]$ pwd
/home/book/soft
[book@BOOK-CENTOS01 soft]$ ls
elasticsearch-7.4.0  elasticsearch-7.4.0-linux-x86_64.tar.gz

4.修改配置文件

[book@BOOK-CENTOS01 soft]$ pwd
/home/book/soft
[book@BOOK-CENTOS01 soft]$ ls
elasticsearch-7.4.0  elasticsearch-7.4.0-linux-x86_64.tar.gz

备份 /home/book/soft/elasticsearch-7.4.0/config/elasticsearch.yml

主节点配置信息：

#集群名称
cluster.name: ES-Cluster
#节点名称
node.name: ES-node1
#是否是master节点
node.master: true
#是否允许该节点存储索引数据
node.data: true
#日志目录
path.logs: /var/log/elasticsearch
#绑定地址
network.host: 0.0.0.0
#http端口
http.port: 9200
#集群主机列表
discovery.seed_hosts: [“10.45.154.236”,“10.45.154.70”]
#启动全新的集群时需要此参数，再次重新启动时此参数可免
#cluster.initial_master_nodes: [“10.45.154.236”]
#集群内同时启动的数据任务个数，默认是2个
cluster.routing.allocation.cluster_concurrent_rebalance: 32
#添加或删除节点及负载均衡时并发恢复的线程个数，默认4个
cluster.routing.allocation.node_concurrent_recoveries: 32
#初始化数据恢复时，并发恢复线程的个数，默认4个
cluster.routing.allocation.node_initial_primaries_recoveries: 32
#存储位置
path.data: /data
#是否开启跨域访问
http.cors.enabled: true
#开启跨域访问后的地址限制，*表示无限制
http.cors.allow-origin: “*”



## 认证使用的
xpack.security.transport.ssl.enabled: true
xpack.security.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

第二个节点配置如下:

#集群名称
cluster.name: ES-Cluster
#节点名称
node.name: ES-node2
#是否是master节点
node.master: false
#是否允许该节点存储索引数据
node.data: true
#日志目录
path.logs: /var/log/elasticsearch
#绑定地址
network.host: 0.0.0.0
#http端口
http.port: 9200
#集群主机列表
discovery.seed_hosts: [“10.45.154.236”,“10.45.154.70”]
#启动全新的集群时需要此参数，再次重新启动时此参数可免
#cluster.initial_master_nodes: [“10.45.154.236”]
#集群内同时启动的数据任务个数，默认是2个
cluster.routing.allocation.cluster_concurrent_rebalance: 32
#添加或删除节点及负载均衡时并发恢复的线程个数，默认4个
cluster.routing.allocation.node_concurrent_recoveries: 32
#初始化数据恢复时，并发恢复线程的个数，默认4个
cluster.routing.allocation.node_initial_primaries_recoveries: 32
#存储位置
path.data: /data
#是否开启跨域访问
http.cors.enabled: true
#开启跨域访问后的地址限制，*表示无限制
http.cors.allow-origin: “*”


## 认证使用的
xpack.security.transport.ssl.enabled: true
xpack.security.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

5.创建存储目录

mkdir -p /data/nodes
chown elasticsearch:elasticsearch /data/nodes

6.开放防火墙端口

firewall-cmd --zone=public --add-port=9300/tcp --permanent
firewall-cmd --zone=public --add-port=9200/tcp --permanent
systemctl restart firewalld.service

7.启动ES

前台启动：/home/book/soft/elasticsearch-7.4.0/bin/elasticsearch 

后台启动：/home/book/soft/elasticsearch-7.4.0/bin/elasticsearch -d

停止 
lsof -i:9200
netstat -tunlp|grep 9200

kill - 9 xxxx

二.kibana配置

1 安装kibana

下载压缩包  解压

2 配置kibana

server.port: 5601
server.host: "192.168.1.95"
elasticsearch.hosts: ["http://192.168.1.96:9200","http://192.168.1.97:9200","http://192.168.1.98:9200"]
i18n.locale: "zh-CN"

3 启动kibana

前台启动 ./bin/kibana
后台启动 nohup ./bin/kibana &

三、设置用户名和密码

0 在集群上配置TLS(单台跳过)

1.生成CA证书 :

bin/elasticsearch-certutil ca

将产生新文件 elastic-stack-ca.p12。
该 elasticsearch-certutil 命令还会提示你输入密码以保护文件和密钥，请保留该文件的副本并记住其密码，此处我们设置为空

2.为集群中的每个节点生成证书和私钥 
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
mv elastic-certificates.p12 config/

将产生新文件 elastic-certificates.p12。系统还会提示你输入密码，你可以输入证书和密钥的密码，也可以按Enter键将密码留空。默认情况下 elasticsearch-certutil 生成没有主机名信息的证书，这意味着你可以将证书用于集群中的每个节点，另外要关闭主机名验证。
将 elastic-certificates.p12 文件复制到每个节点上Elasticsearch配置目录中
无需将 elastic-stack-ca.p12 文件复制到此目录。

3.配置集群中的每个节点以使用其签名证书标识自身并在传输层上启用TLS
启用TLS并指定访问节点证书所需的信息，将以下信息添加到每个节点的 elasticsearch.yml 文件中：

xpack.security.transport.ssl.enabled: true
xpack.security.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

1、修改elasticsearch.yml

http.cors.enabled: true
http.cors.allow-origin: "*"

xpack.security.transport.ssl.enabled: true
xpack.security.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

2、修改password

elasticsearch-setup-passwords interactive

Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y

Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

3、访问elasticsearch

4、Kibana  config\kibana.yml

elasticsearch.username: "elastic"
elasticsearch.password: "密码".

http://localhost:5601  登录用户为 elastic

四.遇见的错误

1 Cause: Cluster state has not been recovered yet, cannot write to the [null] index

[elastic@szyyelk01t elasticsearch-7.4.2]$ bin/elasticsearch-setup-passwords interactive

Failed to determine the health of the cluster running at http://10.10.18.90:19200
Unexpected response code [503] from calling GET http://10.10.18.90:19200/_cluster/health?pretty
Cause: master_not_discovered_exception

It is recommended that you resolve the issues with your cluster before running elasticsearch-setup-passwords.
It is very likely that the password changes will fail when run against an unhealthy cluster.

Do you want to continue with the password setup process [y/N]y

Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y               


Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [apm_system]: 
Reenter password for [apm_system]: 
Enter password for [kibana]: 
Reenter password for [kibana]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Enter password for [beats_system]: 
Reenter password for [beats_system]: 
Enter password for [remote_monitoring_user]: 
Reenter password for [remote_monitoring_user]: 

Unexpected response code [503] from calling PUT http://10.10.18.90:19200/_security/user/apm_system/_password?pretty
Cause: Cluster state has not been recovered yet, cannot write to the [null] index

Possible next steps:
* Try running this tool again.
* Try running with the --verbose parameter for additional messages.
* Check the elasticsearch logs for additional error details.
* Use the change password API manually. 

ERROR: Failed to set password for user [apm_system].

解决：集群中的三台服务器需要同时设置，并重新启动

http.cors.enabled: true
http.cors.allow-origin: "*"

xpack.security.transport.ssl.enabled: true
xpack.security.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

2  max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

解决：
切换到root用户
执行命令：
sysctl -w vm.max_map_count=262144
查看结果：
sysctl -a|grep vm.max_map_count
显示：
vm.max_map_count = 262144


上述方法修改之后，如果重启虚拟机将失效，所以：
解决办法：
在   /etc/sysctl.conf文件最后添加一行
vm.max_map_count=262144
即可永久修改

3  max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

每个进程最大同时打开文件数太小，可通过下面2个命令查看当前数量

ulimit -Hn
ulimit -Sn
　　
修改/etc/security/limits.conf文件，增加配置，用户退出后重新登录生效

*               soft    nofile          65536
*               hard    nofile          65536

4、max number of threads [3818] for user [es] is too low, increase to at least [4096]

问题同上，最大线程个数太低。修改配置文件/etc/security/limits.conf（和问题1是一个文件），增加配置

*               soft    nproc           4096
*               hard    nproc           4096
　　
可通过命令查看

ulimit -Hu
ulimit -Su

 

修改后的文件：

 

5、Exception in thread "main" java.nio.file.AccessDeniedException: /usr/local/elasticsearch/elasticsearch-6.2.2-1/config/jvm.options

elasticsearch用户没有该文件夹的权限，执行命令

chown -R es:es /usr/local/elasticsearch/

 

 

————————————————
原文链接：https://blog.csdn.net/goodsirlee/article/details/102589354
原文链接：https://blog.csdn.net/chengyuqiang/article/details/102935955

原文链接：https://www.cnblogs.com/yidiandhappy/p/7714489.html

原文链接：https://www.cnblogs.com/zhi-leaf/p/8484337.html