[Centos7]部署Elasticsearch-7.10.2集群

1 节点规划

节点	IP	node.master	node.data
node-1	172.16.78.18	true	true
node-2	172.16.78.20	true	true
node-3	172.16.78.19	true	true

2 下载解压

cd /alidata1
useradd hdfs
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.2-linux-x86_64.tar.gz
tar -zxvf elasticsearch-7.10.2-linux-x86_64.tar.gz
chown -R hdfs:hdfs elasticsearch-7.10.2

su - hdfs
cd /alidata1/elasticsearch-7.10.2
mkdir data

3 生成证书、秘钥

/alidata1/elasticsearch-7.10.2/bin/elasticsearch-certutil ca

两次回车。

设置完成后，会在elasticsearch的目录下看到新生成的证书elastic-stack-ca.p12

使用上面生成的ca证书"elastic-stack-ca.p12"生成p12密钥

/alidata1/elasticsearch-7.10.2/bin/elasticsearch-certutil cert --ca /alidata1/elasticsearch-7.10.2/elastic-stack-ca.p12

三次回车。

设置完成后，会在elasticsearch的目录下看到新生成的密钥elastic-certificates.p12

拷贝p12密钥到其它es节点

在config/目录下创建个certs目录，然后将p12证书移动到certs目录下，方便后面在配置文件里引用

cd /alidata1/elasticsearch-7.10.2/config/
mkdir certs
mv /alidata1/elasticsearch-7.10.2/elastic-certificates.p12 certs/
chmod -R +755 certs/

将p12证书移动到certs目录下后直接将certs目录拷贝到其他es节点

su - hdfs
cd /alidata1/elasticsearch-7.10.2/config/
mkdir certs
cp /tmp/elastic-certificates.p12 certs/
chmod -R +755 certs/

4 更换JDK

这里我们不使用elasticsearch自带的OpenJDK，我们手动部署Oracle JDK

先将elasticsearch自带的jdk备份

mv /alidata1/elasticsearch-7.10.2/jdk /alidata1/elasticsearch-7.10.2/jdk.default
tar -zxf /tmp/jdk-11.0.20_linux-x64_bin.tar.gz -C /alidata1/elasticsearch-7.10.2/
mv /alidata1/elasticsearch-7.10.2/jdk-11.0.20/ /alidata1/elasticsearch-7.10.2/jdk
---`这里解释下为什么要把oracle jdk放到elasticsearch目录下：因为后面会去
用systemd来管理elasticsearch，但是用systemd不会去用全局的系统变量，
就会依着elasticsearch去使用自带的OpenJDK，所以这一步相当于是直接把elasticsearch自带的环境变量替换成了Oracle JDK`---

配置jdk环境变量

vim /etc/profile.d/elasticsearch.sh

添加如下内容

#JAVA_HOME
export JAVA_HOME=/alidata1/elasticsearch-7.10.2/jdk
export PATH=$PATH:$JAVA_HOME/bin

让新的环境变量PATH生效

source /etc/profile
java --version

5 修改配置文件

两个配置文件需要修改

elasticsearch.yml
jvm.options

5.1 elasticsearch.yml

es001

cluster.name: driving-elite-cluster-prd
node.master: true
node.data: true
node.name: node-1
# Path to directory where to store the data (separate multiple locations by comma):
path.data: /alidata1/elasticsearch-7.10.2/data
# Path to log files:
path.logs: /alidata1/elasticsearch-7.10.2/logs
# Set the bind address to a specific IP (IPv4 or IPv6):
network.host: 172.16.78.18
# Set a custom port for HTTP:
http.port: 9200
# Pass an initial list of hosts to perform discovery when this node is started:
discovery.seed_hosts: ["172.16.78.18", "172.16.78.20", "172.16.78.19"]
# Bootstrap the cluster using an initial set of master-eligible nodes:
cluster.initial_master_nodes: ["node-1"]
# 故障检测请求超时时间，默认为30秒。
discovery.zen.fd.ping_timeout: 120s
# 故障检测超时后的重试次数，默认为3次。
discovery.zen.fd.ping_retries: 6
# 故障检测间隔周期，默认为1秒。
discovery.zen.fd.ping_interval: 60s
discovery.initial_state_timeout: 60s
# X-pack安全认证
xpack.security.enabled: true
# X-pack开启传输层认证
xpack.security.transport.ssl.enabled: true
# certificate：它验证所提供的证书是否由受信任的机构(CA)签名，但不执行任何主机名验证。
xpack.security.transport.ssl.verification_mode: certificate
# 下面是p12密钥文件的存储位置，建议使用绝对路径，当然相对路径也是可以的："certs/elastic-certificates.p12"
xpack.security.transport.ssl.keystore.path: /alidata1/elasticsearch-7.10.2/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /alidata1/elasticsearch-7.10.2/config/certs/elastic-certificates.p12

es002

cluster.name: driving-elite-cluster-prd
node.master: true
node.data: true
node.name: node-2
# Path to directory where to store the data (separate multiple locations by comma):
path.data: /alidata1/elasticsearch-7.10.2/data
# Path to log files:
path.logs: /alidata1/elasticsearch-7.10.2/logs
# Set the bind address to a specific IP (IPv4 or IPv6):
network.host: 172.16.78.20
# Set a custom port for HTTP:
http.port: 9200
# Pass an initial list of hosts to perform discovery when this node is started:
discovery.seed_hosts: ["172.16.78.18", "172.16.78.20", "172.16.78.19"]
# Bootstrap the cluster using an initial set of master-eligible nodes:
cluster.initial_master_nodes: ["node-1"]
# 故障检测请求超时时间，默认为30秒。
discovery.zen.fd.ping_timeout: 120s
# 故障检测超时后的重试次数，默认为3次。
discovery.zen.fd.ping_retries: 6
# 故障检测间隔周期，默认为1秒。
discovery.zen.fd.ping_interval: 60s
discovery.initial_state_timeout: 60s
# X-pack安全认证
xpack.security.enabled: true
# X-pack开启传输层认证
xpack.security.transport.ssl.enabled: true
# certificate：它验证所提供的证书是否由受信任的机构(CA)签名，但不执行任何主机名验证。
xpack.security.transport.ssl.verification_mode: certificate
# 下面是p12密钥文件的存储位置，建议使用绝对路径，当然相对路径也是可以的："certs/elastic-certificates.p12"
xpack.security.transport.ssl.keystore.path: /alidata1/elasticsearch-7.10.2/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /alidata1/elasticsearch-7.10.2/config/certs/elastic-certificates.p12

es003

cluster.name: driving-elite-cluster-prd
node.master: true
node.data: true
node.name: node-3
# Path to directory where to store the data (separate multiple locations by comma):
path.data: /alidata1/elasticsearch-7.10.2/data
# Path to log files:
path.logs: /alidata1/elasticsearch-7.10.2/logs
# Set the bind address to a specific IP (IPv4 or IPv6):
network.host: 172.16.78.19
# Set a custom port for HTTP:
http.port: 9200
# Pass an initial list of hosts to perform discovery when this node is started:
discovery.seed_hosts: ["172.16.78.18", "172.16.78.20", "172.16.78.19"]
# Bootstrap the cluster using an initial set of master-eligible nodes:
cluster.initial_master_nodes: ["node-1"]
# 故障检测请求超时时间，默认为30秒。
discovery.zen.fd.ping_timeout: 120s
# 故障检测超时后的重试次数，默认为3次。
discovery.zen.fd.ping_retries: 6
# 故障检测间隔周期，默认为1秒。
discovery.zen.fd.ping_interval: 60s
discovery.initial_state_timeout: 60s
# X-pack安全认证
xpack.security.enabled: true
# X-pack开启传输层认证
xpack.security.transport.ssl.enabled: true
# certificate：它验证所提供的证书是否由受信任的机构(CA)签名，但不执行任何主机名验证。
xpack.security.transport.ssl.verification_mode: certificate
# 下面是p12密钥文件的存储位置，建议使用绝对路径，当然相对路径也是可以的："certs/elastic-certificates.p12"
xpack.security.transport.ssl.keystore.path: /alidata1/elasticsearch-7.10.2/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /alidata1/elasticsearch-7.10.2/config/certs/elastic-certificates.p12

5.2 jvm.options

# 官方建议设置es内存，大小为物理内存的一半，剩下的一半留给luence，这是因为es的底层是luence(java语言研发的搜索引擎框架)，luence本身就是单独占用内存的，而且占用的还不少
-Xms48g
-Xmx48g

6 设置内核参数

修改打开文件的数量

cat > /etc/security/limits.d/elk.conf <<'EOF'
* soft nofile 65535
* hard nofile 131070
EOF

修改内核参数的内存映射信息

cat > /etc/sysctl.d/elk.conf <<'EOF'
vm.max_map_count = 262144
EOF

sysctl -p /etc/sysctl.d/elk.conf 

7 编写ES启动脚本

vim /usr/lib/systemd/system/elasticsearch.service

[Unit]
Description=elasticsearch-v7.10.2
After=network.target

[Service]
Restart=on-failure
ExecStart=/alidata1/elasticsearch-7.10.2/bin/elasticsearch
User=hdfs
Group=hdfs
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target

ES，启动！

systemctl daemon-reload
systemctl start elasticsearch
systemctl enable elasticsearch

查看日志

tail -fn100 /alidata1/elasticsearch-7.10.2/logs/driving-elite-cluster-prd.log

8 设置用户名密码

在集群状态正常的情况下设置用户名密码

自动随机生成

/alidata1/elasticsearch-7.10.2/bin/elasticsearch-setup-passwords auto

9 附：kibana

wget https://artifacts.elastic.co/downloads/kibana/kibana-7.10.2-linux-x86_64.tar.gz
tar -zxf kibana-7.10.2-linux-x86_64.tar.gz

vim /alidata1/kibana-7.10.2-linux-x86_64/config/kibana.yml

server.port: 5601
server.host: "0.0.0.0"
server.name: "bigdata-prd-elite-es001"
elasticsearch.hosts: ["http://172.16.78.18:9200", "http://172.16.78.20:9200", "http://172.16.78.19:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: ""
i18n.locale: "zh-CN"

编写kibana启动脚本

vim /etc/systemd/system/kibana.service 

[Unit]
Description=kibana-7.10.2
After=network.target  

[Service]
Type=simple
User=hdfs
Group=hdfs
ExecStart=/alidata1/kibana-7.10.2-linux-x86_64/bin/kibana
PrivateTmp=true

[Install]
WantedBy=multi-user.target

systemctl daemon-reload
systemctl start kibana
systemctl enable kibana

10 参考文献

https://blog.csdn.net/qq_44930876/article/details/128628749

https://blog.csdn.net/qq_44930876/article/details/128653811

https://www.elastic.co/guide/en/elasticsearch/reference/7.10/settings.html

https://blog.csdn.net/yi_qingjun/article/details/128660086

https://blog.csdn.net/weixin_43883625/article/details/129138770