docker-compose部署elasticsearch集群
一、部署思路
模拟部署了六个节点,其中两个master节点,不做数据存储,四个data节点,用来存储数据
注意:
-
本文使用docker-compose进行部署,没有安装的先自行安装
-
本文的elasticsearch集群使用docker进行部署,开发学习环境中可以这么使用,在生产环境中还是不要这样部署,熟悉elasticsearch原理的就会清楚,单机部署elasticsearch集群基本无法起到什么作用。
二、配置文件
1、目录结构
docker-compose.yml
data
es00
es01
es02
es03
es04
es05
config
es00
elasticsearch.yml
es01
elasticsearch.yml
es02
elasticsearch.yml
es03
elasticsearch.yml
es04
elasticsearch.yml
es05
elasticsearch.yml
logs
es00
es01
es02
es03
es04
es05
2、docker-compose.yml
version: '2'
services:
cerebro:
image: lmenezes/cerebro:0.8.4
container_name: cerebro
ports:
- "9000:9000"
command:
- -Dhosts.0.host=http://es01:9201
networks:
- esnet
kibana:
image: kibana:7.3.0
container_name: kibana
environment:
- ELASTICSEARCH_HOSTS=http://es01:9201
# 需要将Kibana配置文件中的小写转换成大写,然后这个才能用于变量,才能被设置到
- I18N_LOCALE=zh-CN
- xpack.monitoring.ui.container.elasticsearch.enabled=false
ports:
- 5601:5601
networks:
- esnet
es00:
image: docker.elastic.co/elasticsearch/elasticsearch:7.3.0
container_name: es00
restart: always
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "ES_JAVA_OPTS=-Des.insecure.allow.root=true"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/es00:/usr/share/elasticsearch/data
- ./config/es00/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./logs/es00:/usr/share/elasticsearch/logs
ports:
- 9200:9200
- 9300:9300
networks:
- esnet
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.3.0
container_name: es01
restart: always
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "ES_JAVA_OPTS=-Des.insecure.allow.root=true"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/es01:/usr/share/elasticsearch/data
- ./config/es01/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./logs/es01:/usr/share/elasticsearch/logs
ports:
- 9201:9201
- 9301:9301
networks:
- esnet
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.3.0
container_name: es02
restart: always
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/es02:/usr/share/elasticsearch/data
- ./config/es02/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./logs/es02:/usr/share/elasticsearch/logs
depends_on:
- es01
networks:
- esnet
ports:
- 9202:9202
- 9302:9302
es03:
image: docker.elastic.co/elasticsearch/elasticsearch:7.3.0
container_name: es03
restart: always
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/es03:/usr/share/elasticsearch/data
- ./config/es03/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./logs/es03:/usr/share/elasticsearch/logs
depends_on:
- es01
networks:
- esnet
ports:
- 9203:9203
- 9303:9303
es04:
image: docker.elastic.co/elasticsearch/elasticsearch:7.3.0
container_name: es04
restart: always
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/es04:/usr/share/elasticsearch/data
- ./config/es04/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./logs/es04:/usr/share/elasticsearch/logs
depends_on:
- es01
networks:
- esnet
ports:
- 9204:9204
- 9304:9304
es05:
image: docker.elastic.co/elasticsearch/elasticsearch:7.3.0
container_name: es05
restart: always
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/es05:/usr/share/elasticsearch/data
- ./config/es05/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./logs/es05:/usr/share/elasticsearch/logs
depends_on:
- es01
networks:
- esnet
ports:
- 9205:9205
- 9305:9305
networks:
esnet:
3、master 节点
1、es00
cluster.name: es-cluster
node.name: es00
node.master: true
node.data: false
path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/logs
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
# transport.tcp.port: [9300,9302,9303,9304,9305]
# discovery.zen.ping.unicast.hosts: ["es01","es02","es03","es04","es05"]
cluster.initial_master_nodes: ["es00","es01"]
discovery.seed_hosts: ["es00","es01"]
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: false
xpack.security.transport.ssl.enabled: false
2、es01
cluster.name: es-cluster
node.name: es01
node.master: true
node.data: false
path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/logs
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9201
# transport.tcp.port: [9300,9302,9303,9304,9305]
# discovery.zen.ping.unicast.hosts: ["es01","es02","es03","es04","es05"]
cluster.initial_master_nodes: ["es00","es01"]
discovery.seed_hosts: ["es00","es01"]
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: false
xpack.security.transport.ssl.enabled: false
4、data节点
1、es02
cluster.name: es-cluster
node.name: es02
node.master: false
node.data: true
path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/logs
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9202
# transport.tcp.port: [9300,9302,9303,9304,9305]
# discovery.zen.ping.unicast.hosts: ["es01","es02","es03","es04","es05"]
cluster.initial_master_nodes: ["es00","es01"]
discovery.seed_hosts: ["es00","es01"]
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: false
xpack.security.transport.ssl.enabled: false
2、es03
cluster.name: es-cluster
node.name: es03
node.master: false
node.data: true
path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/logs
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9203
# transport.tcp.port: [9300,9302,9303,9304,9305]
# discovery.zen.ping.unicast.hosts: ["es01","es02","es03","es04","es05"]
cluster.initial_master_nodes: ["es00","es01"]
discovery.seed_hosts: ["es00","es01"]
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: false
xpack.security.transport.ssl.enabled: false
1、es04
cluster.name: es-cluster
node.name: es04
node.master: false
node.data: true
path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/logs
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9204
# transport.tcp.port: [9300,9302,9303,9304,9305]
# discovery.zen.ping.unicast.hosts: ["es01","es02","es03","es04","es05"]
cluster.initial_master_nodes: ["es00","es01"]
discovery.seed_hosts: ["es00","es01"]
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: false
xpack.security.transport.ssl.enabled: false
1、es05
cluster.name: es-cluster
node.name: es05
node.master: false
node.data: true
path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/logs
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9205
# transport.tcp.port: [9300,9302,9303,9304,9305]
# discovery.zen.ping.unicast.hosts: ["es01","es02","es03","es04","es05"]
cluster.initial_master_nodes: ["es00","es01"]
discovery.seed_hosts: ["es00","es01"]
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: false
xpack.security.transport.ssl.enabled: false
cluster.name: es-cluster
node.name: es02
node.master: false
node.data: true
path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/logs
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9202
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["es01","es02","es03"]
cluster.initial_master_nodes: ["es01"]
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: false
xpack.security.transport.ssl.enabled: false
三、集群启动
1、启动
在docker-compose.yml文件目录下
docker-compose up -d
2、停止
docker-compose stop
3、如果启动后无法访问,查看日志排查问题(会有使用root用户无法启动es的问题,需要排查)
docker logs -f -t --tail=500 es01
无法启动的原因:
1、会有使用root用户无法启动es的问题,需要排查(新建系统用户启动es集群)
2、需要修改文件夹权限 chmod 777 -R [文件夹] (data、log、config 几个文件夹改权限)
三、集群安装分词器
6.1 进入docker容器
docker exec -it es01/bin/bash
6.2 下载安装分词器
./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.3.0/elasticsearch-analysis-ik-7.3.0.zip
文件地址:https://download.csdn.net/download/iampatrick_star/87131975?spm=1001.2014.3001.5503
参考文章
https://blog.csdn.net/epitomizelu/article/details/105592906