本文主要记录Elasticsearch的集群搭建和通过Kibana查看
环境
服务器:2核CPU、4GB内存
系统:CentOS 7.6
⚠️这个配置只能启动两台Elasticsearch节点同时启动Kibana就会报内存问题,所以自己测试完集群的话最好高于这配置。
服务器基础配置
修改文件描述符数目
- 设置环境变量 修改 /etc/profile文件
[root@VM-4-9-centos data]# vim /etc/profile
##打开文件/etc/profile文末添加
ulimit -n 65535
[root@VM-4-9-centos data]# source /etc/profile
- 修改limits.conf
[root@VM-4-9-centos data]# vim /etc/security/limits.conf
## 文件最后增加
* soft nofile 65536
* hard nofile 65536
验证是否成功
使用ulimit -a查看是否修改成功
[root@VM-4-9-centos ~]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 14690
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 14690
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
修改 最大映射数量 MMP
Elasticsearch 对各种文件混合使用了 NioFs( 非阻塞文件系统)和 MMapFs ( 内存映射文件系统)。
请确保你配置的最大映射数量,以便有足够的虚拟内存可用于 mmapped 文件。这可以暂时设置:临时方案
sysctl -w vm.max_map_count=262144
永久方案
可以在 /etc/sysctl.conf 通过修改 vm.max_map_count 永久设置它。
[root@VM-4-9-centos config]# vim /etc/sysctl.conf
## 文末增加
vm.max_map_count=262144
执行下sysctl -p生效
[root@VM-4-9-centos data]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
net.ipv4.conf.all.promote_secondaries = 1
net.ipv4.conf.default.promote_secondaries = 1
net.ipv6.neigh.default.gc_thresh3 = 4096
net.ipv4.neigh.default.gc_thresh3 = 4096
kernel.softlockup_panic = 1
kernel.sysrq = 1
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
kernel.numa_balancing = 0
kernel.shmmax = 68719476736
kernel.printk = 5
vm.max_map_count = 262144
如果没有修改会出现如下错误
ERROR: [1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.
bootstrap check failure [1] of [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
- 部署elasticsearch
elasticsearch不能以root账户启动,部署elasticsearch前需要先设置用户组和用户。
这里添加用户组和用户名均为 elasticsearch
# 添加用户租和用户 elasticsearch:elasticsearch
[root@VM-4-9-centos data]# groupadd elasticsearch
[root@VM-4-9-centos data]# useradd -g elasticsearch elasticsearch
[root@VM-4-9-centos data]# grep elasticsearch /etc/group
elasticsearch:x:1003:
解压下载的gz包
[root@VM-4-9-centos ~]# cd /data
[root@VM-4-9-centos data]# tar -vxcf elasticsearch-8.1.0-linux-x86_64.tar.gz
授权文件夹权限给elasticsearch 用户
chown -R elastic:elastic elasticsearch-8.1.0
切换用户
[root@VM-4-9-centos data] su elasticsearch
[elasticsearch@VM-4-9-centos data]$
修改配置文件/data/elasticsearch-8.1.0/conf/elasticsearch.yml
network.host : 0.0.0.0
启动
./bin/elasticsearch
修改配置文件 /data/elasticsearch-8.1.0/conf/jvm.options
-Xms1g
-Xmx1g
控制台输出内容需要保存下来,后面会用到
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✅ Elasticsearch security features have been automatically configured!
✅ Authentication is enabled and cluster connections are encrypted.
ℹ️ Password for the elastic user (reset with bin/elasticsearch-reset-password -u elastic):
hrSUSpjDURbtxg-hTGEp
ℹ️ HTTP CA certificate SHA-256 fingerprint:
a5ac5cc0e49da20f16cb32d4a717fa208def996b9df4d449b03910f6fd4acdaf
ℹ️ Configure Kibana to use this cluster:
• Run Kibana and click the configuration link in the terminal when Kibana starts.
• Copy the following enrollment token and paste it into Kibana in your browser (valid for the next 30 minutes):
eyJ2ZXIiOiI4LjEuMCIsImFkciI6WyIxNzIuMTYuNC43OjkyMDAiXSwiZmdyIjoiOGE0N2FiZjE4NTcyZGFjOTM3MDM0OGVhNDZiOTZhODQzOGI1NDkxYTg0MjhlMzExOWE0OWI1MTFmNDE1ODIzNSIsImtleSI6Ik1ldXNOWU1CdlltRk9KeFYzWWt4OlZvaTJWdVlxUmctRmJVdzZGWEtJbFEifQ==
ℹ️ Configure other nodes to join this cluster:
• Copy the following enrollment token and start new Elasticsearch nodes with bin/elasticsearch --enrollment-token <token> (valid for the next 30 minutes):
eyJ2ZXIiOiI4LjEuMCIsImFkciI6WyIxMC4wLjQuOTo5MjAyIl0sImZnciI6ImE1YWM1Y2MwZTQ5ZGEyMGYxNmNiMzJkNGE3MTdmYTIwOGRlZjk5NmI5ZGY0ZDQ0OWIwMzkxMGY2ZmQ0YWNkYWYiLCJrZXkiOiJnZTZVSFlNQmdqZmxZWkJ2WXpTcDpiV3pfTTlGQlFMS0lUZnNxU09Ba2N3In0=
If you're running in Docker, copy the enrollment token and run:
docker run -e "ENROLLMENT_TOKEN=<token>" docker.elastic.co/elasticsearch/elasticsearch:8.1.0
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
启动好后,就可以用浏览器访问了。
https://{你的服务器外网ip地址}:9200
记得防火墙开放9200端口
部署kibana
解压下载的gz包
[root@VM-4-9-centos ~]# cd /data
[root@VM-4-9-centos data]# tar -vxcf kibana-8.1.0-linux-x86_64.tar.gz
修改配置kibana.yml
[root@VM-4-9-centos ~]# cd /data/kibana-8.1.0//config/
[root@VM-4-9-centos kibana-8.1.0]# vim kibana.yml
server.host: "0.0.0.0"
启动kibana
[root@iZbp18g1ehm4ub2j97z6lxZ kibana-8.1.0]# nohup ./bin/kibana --allow-root </dev/null &>/dev/null &
注意控制台内容
。。。
Go to http://0.0.0.0:5601/?code=883322 to get started.
浏览器访问
http://{你的服务器外网ip地址:5601}/?code=475418 to get started.
(这里的475418每个人实际情况是不同的)
在弹框中输入
用户 elasticseearch
密码 hrSUSpjDURbtxg-hTGEp (这里每个人实际情况是不同的)
以上就搭建好了一个elasticsearch的单机环境。
Elasticsearch 双节点集群环境部署
把上一步启动的关掉, Contrl-C/Ctrl-C. 关掉elasticsearch和kibana窗口。
或者用命令来查看和结束进程
[root@VM-4-9-centos kibana-8.1.0]# ps ef|grep elasticseearch
[root@VM-4-9-centos kibana-8.1.0]# kill -9 {xxxid}
[root@VM-4-9-centos kibana-8.1.0]# ps ef|grep kibana
[root@VM-4-9-centos kibana-8.1.0]# kill -9 {xxxid}
多节点elasticsearch.yml
节点1
node.name: node1
network.host: 0.0.0.0
http.port: 9200
transport.port: 9300
discovery.seed_hosts: ["127.0.0.0:9300","127.0.0.0:9301"]
cluster.initial_master_nodes: ["127.0.0.0:9300","127.0.0.0:9301"]
节点2
node.name: node2
network.host: 0.0.0.0
http.port: 9201
transport.port: 9301
discovery.seed_hosts: ["127.0.0.0:9300","127.0.0.0:9301"]
cluster.initial_master_nodes: ["127.0.0.0:9300","127.0.0.0:9301"]
节点2直接cp节点1的文件来修改
[root@VM-4-9-centos kibana-8.1.0]# cp -r elasticsearch-8.1.0 elasticsearch-8.1.0_02
以此启动node1,node2 (记得开放9200 9201 5601端口)
访问 9200需要https
访问 5601需要http
访问地址 http://your IP address:5601 验证集群是否成功
访问地址 https://{your IP address}:9200/
访问地址 https://{your IP address}:9200/_cat/health?v