ES 7.16.2 Linux环境集群搭建

elasticsearch-7.16.2 (logstash kibana 版本与ES相同)

3节点 ES集群配置

cluster.name: elasticsearch-cluster

node.name: es-node-01

network.host: 0.0.0.0

#network.publish_host: 127.0.0.1

http.port: 9201

transport.tcp.port: 9301

http.cors.enabled: true

http.cors.allow-origin: "*"

node.master: true

node.data: true

discovery.seed_hosts: ["10.206.65.132:9201","10.206.65.132:9202","10.206.65.132:9203"]

cluster.initial_master_nodes: ["es-node-01","es-node02","es-node03"]

正常启动后结果如下(注意：

cluster_uuid

)

{ "name" : "es-node-01", "cluster_name" : "elasticsearch-cluster", "cluster_uuid" : "N6Uj_nbKSM2FzLyfPIUdag", "version" : { "number" : "7.16.1", "build_flavor" : "default", "build_type" : "docker", "build_hash" : "5b38441b16b1ebb16a27c107a4c3865776e20c53", "build_date" : "2021-12-11T00:29:38.865893768Z", "build_snapshot" : false, "lucene_version" : "8.10.1", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" }

kibana.yml 配置如下

#kibana 端口

server.port: 5601

server.host: 0.0.0.0

#ES 节点地址

elasticsearch.hosts: ["http://10.206.65.132:9201", "http://10.206.65.132:9202", "http://10.206.65.132:9203"]

# 显示语言 默认为 en 英文

i18n.locale: "zh-CN"

logstash.conf 配置

# Sample Logstash configuration for creating a simple

# Beats -> Logstash -> Elasticsearch pipeline.

input {

tcp {

port => 5044

#codec => json

}

stdin { }

}

output {

elasticsearch {

hosts => ["http://10.206.65.132:9201", "http://10.206.65.132:9202", "http://10.206.65.132:9203"]

#protocol => "http"

index => "logstash-test-%{+YYYY.MM.dd}"

#action => "index"

#user => "elastic"

#password => "changeme"

}

}

logstash.yml配置如下

path.config: /usr/share/logstash/config/logstash.conf

注：windows 下启动是可以指定使用的配置文件,命令如下： logstash -f "D:\tmp\ES-Window\logstash-7.6.2\config\logstash.conf"

访问 http://localhost:9600 显示如下

{"host":"LAPTOP-0RAO5C7K","version":"7.16.2","http_address":"127.0.0.1:9600","id":"1118b56b-ea7c-483a-b3bd-553aa2538b8e","name":"LAPTOP-0RAO5C7K","ephemeral_id":"8a761cbf-d58a-47d2-9cc3-05ef0afbea7e","status":"green","snapshot":false,"pipeline":{"workers":12,"batch_size":125,"batch_delay":50},"monitoring":{"hosts":["https://127.0.0.1:9200"],"username":"logstash_system"},"build_date":"2021-12-18T19:42:46Z","build_sha":"f1d497fd30cdb16dccebf2de1a788aad1005be9a","build_snapshot":false}

ES 启用安全，生成安全证书

1.进入 ES bin 目录执行 elasticsearch-certutil ca 生成 elastic-stack-ca.p12 文件

2. 执行 elasticsearch-certutil cert --ca elastic-stack-ca.p12 生成 elastic-stack-ca.p12 文件elastic-certificates.p12(将elastic-certificates.p12文件复制到各个节点的对应位置,这里在 ES的config目录下新建 certs 文件夹，存放改文件)

3. 重启 ES

 

 

 

 注意：两个证书文件都不要设置密码，否则ES无法启动

4.在ES bin 目录下执行 elasticsearch-setup-passwords interactive，为所有内置用户设置密码

 

内置用户

用户名 作用

elastic 超级用户

kibana 用于负责Kibana连接Elasticsearch

logstash_system Logstash将监控信息存储在Elasticsearch中时使用

beats_system Beats在Elasticsearch中存储监视信息时使用

apm_system APM服务器在Elasticsearch中存储监视信息时使用

remote_monitoring_user Metricbeat用户在Elasticsearch中收集和存储监视信息时使用

为elastic用户设置密码后，引导密码将不再有效。并且再次执行elasticsearch-setup-passwords命令会抛出异常

所有内置用户 名称如下

Enter password for [elastic]:

Reenter password for [elastic]:

Enter password for [apm_system]:

Reenter password for [apm_system]:

Enter password for [kibana_system]:

Reenter password for [kibana_system]:

Enter password for [logstash_system]:

Reenter password for [logstash_system]:

Enter password for [beats_system]:

Reenter password for [beats_system]:

Enter password for [remote_monitoring_user]:

Reenter password for [remote_monitoring_user]:

Changed password for user [apm_system]

Changed password for user [kibana_system]

Changed password for user [kibana]

Changed password for user [logstash_system]

Changed password for user [beats_system]

Changed password for user [remote_monitoring_user]

Changed password for user [elastic]

修改ES配置文件elasticsearch.yml，增加如下配置

#开启权限校验, 生成认证证书

xpack.security.enabled: true

xpack.security.transport.ssl.enabled: true

xpack.security.transport.ssl.verification_mode: certificate

#以下两行配置证书位置

xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12

xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12

修改 kibana.yml 文件，增加如下两行

elasticsearch.username: "kibana_system"

elasticsearch.password: "123456"

修改 logstash.conf

input {

tcp {

port => 5044

#codec => json

}

stdin { }

}

output {

elasticsearch {

hosts => ["http://10.206.65.132:9201", "http://10.206.65.132:9202", "http://10.206.65.132:9203"]

#protocol => "http"

# ES 中索引名字

index => "logstash-test-%{+YYYY.MM.dd}"

#action => "index"

user => "elastic"

password => "123456"

}

}

修改 logstash.yml 增加如下配置

xpack.monitoring.enabled: true

xpack.monitoring.elasticsearch.username: "logstash_system"

xpack.monitoring.elasticsearch.password: "123456"

xpack.monitoring.elasticsearch.hosts: ["http://10.206.65.132:9201", "http://10.206.65.132:9202", "http://10.206.65.132:9203"]

 

 

SpringBoot 配置 logstash

 

monitoring.ui.elasticsearch.username 的值为ES的用户，用户操作命令如下

1.查询现有用户信息

GET /_security/user

2.新增用户并为其赋权(设置角色)

POST /_security/user/remote_monitor_admin{  "password" : "123456",  "roles" : [ "remote_monitoring_collector", "remote_monitoring_agent", "kibana_system" ]}

3.删除用户

DELETE /_xpack/security/user/remote_monitor_admin

Linux环境部署基本一致

firewall-cmd --zone=public --add-port=9200/tcp --permanent
firewall-cmd --reload

firewall-cmd --query-port=9200/tcp

老版本Linux系统使用iptables

docker 部署举例

docker pull canal/canal-admin:v1.1.5

docker run -it -d --name ES -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --network=host --privileged=true --restart=always -v /home/data/elasticsearch/config:/usr/share/elasticsearch/config -v /home/data/elasticsearch/data:/usr/share/elasticsearch/data -v /home/data/elasticsearch/logs:/usr/share/elasticsearch/logs -v /home/data/elasticsearch/plugins:/usr/share/elasticsearch/plugins elasticsearch:7.16.2

docker run -it -d --name kibana -p 5601:5601 --network=host --privileged=true --restart=always -v /home/data/kibana/config:/usr/share/kibana/config -v /home/data/kibana/logs:/usr/share/kibana/logs kibana:7.16.2