docker 部署 ES集群
一. 安装环境说明
Ubuntu 20.04.2 LTS
elasticsearch 7.10.1
二. 从docker镜像仓库拉取es镜像
docker pull elasticsearch:7.10.1
若镜像拉取不到可以使用腾讯云的docker镜像源https://mirror.ccs.tencentyun.com
三. 创建文件映射目录
# 在当前用户下创建es集群文件夹
mkdir ./elasticsearch
# 该集群共创建三个节点 分别是es-master、es-node01、es-node02
# 为这三个节点分别创建数据和插件映射文件夹
cd elasticsearch
mkdir ./es-{master,node01,node02} ./es-{master,node01,node02}/data ./es-{master,node01,node02}/plugins
# 授予文件夹访问权限
chmod 777 ./es-{master,node01,node02}/data ./es-{master,node01,node02}/plugins
四. 运行docker容器
#master
docker run -d \
--name=es-master \
--restart=always \
-e "http.host=0.0.0.0" \
-e "ES_JAVA_OPTS=-Xms4g -Xmx4g" \
-e "cluster.name=es-cluster" \
-e "cluster.initial_master_nodes=es-master" \
-v /etc/localtime:/etc/localtime \
-v /home/ubuntu/elasticsearch/es-master/data:/usr/share/elasticsearch/data \
-v /home/ubuntu/elasticsearch/es-master/plugins:/usr/share/elasticsearch/plugins \
-p 9200:9200 \
-p 9300:9300 \
elasticsearch:7.10.1
#01
docker run -d \
--name=es-node01 \
--restart=always \
-e "http.host=0.0.0.0" \
-e "ES_JAVA_OPTS=-Xms2g -Xmx2g" \
-e "cluster.name=es-cluster" \
-e "cluster.initial_master_nodes=es-master" \
-v /etc/localtime:/etc/localtime \
-v /home/ubuntu/elasticsearch/es-node01/data:/usr/share/elasticsearch/data \
-v /home/ubuntu/elasticsearch/es-node01/plugins:/usr/share/elasticsearch/plugins \
-p 9201:9201 \
-p 9301:9301 \
elasticsearch:7.10.1
#02
docker run -d \
--name=es-node02 \
--restart=always \
-e "http.host=0.0.0.0" \
-e "ES_JAVA_OPTS=-Xms2g -Xmx2g" \
-e "cluster.name=es-cluster" \
-e "cluster.initial_master_nodes=es-master" \
-v /etc/localtime:/etc/localtime \
-v /home/ubuntu/elasticsearch/es-node02/data:/usr/share/elasticsearch/data \
-v /home/ubuntu/elasticsearch/es-node02/plugins:/usr/share/elasticsearch/plugins \
-p 9202:9202 \
-p 9302:9302 \
elasticsearch:7.10.1
docker ps 查看启动状态
Elasticsearch 的 9200 端口和 9300 端口分别承担着不同的职责:
9200 端口
- 用途:9200 端口主要用于 HTTP 协议的 RESTful 接口,允许客户端通过 HTTP 协议与 Elasticsearch 进行交互。
- 功能:
- 提供了一个 RESTful API,用于执行 CRUD(创建、读取、更新、删除)操作。
- 支持查询、索引管理和集群管理等功能。
- 通常用于客户端应用、Web 界面(如 Kibana)或任何希望与 Elasticsearch 交互的应用程序。
9300 端口
- 用途:9300 端口主要用于节点间的 TCP 通信,是 Elasticsearch 集群内部通信的基础。
- 功能:
- 用于 Elasticsearch 节点之间的通信,包括数据传输、心跳检测等。
- 支持集群发现和节点加入集群的过程。
- 通常用于集群内部节点之间的通信,而不是客户端直接使用。
启动出现的问题及解决方案
-
AccessDeniedException[/usr/share/elasticsearch/data/nodes]
映射文件夹没有权限,通过chmod授予文件夹权限即可
-
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
虚拟内存限制:vm.max_map_count
的值太低,需要增加到至少 262144。1). 修改系统参数:
-
在宿主机上编辑
/etc/sysctl.conf
文件,添加以下行:vm.max_map_count=262144
2). 使更改生效:
-
运行以下命令使更改立即生效:
sysctl -p
3). 验证设置:
-
查看当前的
vm.max_map_count
设置:cat /proc/sys/vm/max_map_count
-
-
the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
发现设置:默认的发现设置不适合生产使用,需要配置至少一个
discovery.seed_hosts
、discovery.seed_providers
或cluster.initial_master_nodes
。配置
cluster.initial_master_nodes
:- 为
es-master
节点配置cluster.initial_master_nodes
,使其知道哪些节点可以成为主节点。
- 为
五. 生成证书
# 进入master容器
docker exec -it es-master bash
# 进入bin目录
cd bin
# 执行生成证书命令并一路回车
elasticsearch-certutil cert
# 生成的证书 elastic-certificates.p12 默认会放在当前目录下 即/usr/share/elasticsearch
# 将证书拷贝到config文件夹下
mv elastic-certificates.p12 ./config
# 修改证书所有者
chown elasticsearch:elasticsearch elastic-certificates.p12
将证书拷贝到另外两个服务的容器中
# 先将证书从当前容器中拷贝出来
docker cp es-master:/usr/share/elasticsearch/config/elastic-certificates.p12 ./
# 将证书拷贝到目标容器中并修改所有者(需进入容器修改,命令略)
docker cp ./elastic-certificates.p12 es-node01:/usr/share/elasticsearch/config
docker cp ./elastic-certificates.p12 es-node02:/usr/share/elasticsearch/config
六. 修改 elasticsearch.yml文件
进入es容器,编辑elasticsearch.yml文件
docker exec -it es-master bash
vi /usr/share/elasticsearch/config/elasticsearch.yml
三个容器的配置分别为(根据实际情况修改ip地址):
-
master
cluster.name: "es-cluster" network.host: 0.0.0.0 network.publish_host: 127.0.0.1 http.port: 9200 transport.tcp.port: 9300 http.cors.enabled: true http.cors.allow-origin: "*" node.name: es-master node.master: true node.data: false node.ingest: false indices.queries.cache.size: 5% indices.fielddata.cache.size: 5% indices.breaker.fielddata.limit: 70% indices.breaker.request.limit: 60% indices.breaker.total.limit: 90% http.max_content_length: 200m discovery.zen.ping_timeout: 10s discovery.zen.fd.ping_timeout: 10000s discovery.zen.fd.ping_retries: 10 discovery.zen.minimum_master_nodes: 1 discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300","127.0.0.1:9302","127.0.0.1:9303"] cluster.initial_master_nodes: ["es-master"] # 添加xpack证书配置 xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.client_authentication: required xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
-
node01
cluster.name: "es-cluster" network.host: 0.0.0.0 network.publish_host: 127.0.0.1 http.port: 9201 transport.tcp.port: 9301 http.cors.enabled: true http.cors.allow-origin: "*" node.name: es-node01 node.master: false node.data: true node.ingest: true indices.queries.cache.size: 5% indices.fielddata.cache.size: 5% indices.breaker.fielddata.limit: 70% indices.breaker.request.limit: 60% indices.breaker.total.limit: 90% http.max_content_length: 200m discovery.zen.ping_timeout: 10s discovery.zen.fd.ping_timeout: 10000s discovery.zen.fd.ping_retries: 10 discovery.zen.minimum_master_nodes: 1 discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300","127.0.0.1:9301","127.0.0.1:9302"] cluster.initial_master_nodes: ["es-master"] # 添加xpack证书配置 xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.client_authentication: required xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
-
node02
cluster.name: "es-cluster" network.host: 0.0.0.0 network.publish_host: 127.0.0.1 http.port: 9202 transport.tcp.port: 9302 http.cors.enabled: true http.cors.allow-origin: "*" node.name: es-node02 node.master: false node.data: true node.ingest: true indices.queries.cache.size: 5% indices.fielddata.cache.size: 5% indices.breaker.fielddata.limit: 70% indices.breaker.request.limit: 60% indices.breaker.total.limit: 90% http.max_content_length: 200m discovery.zen.ping_timeout: 10s discovery.zen.fd.ping_timeout: 10000s discovery.zen.fd.ping_retries: 10 discovery.zen.minimum_master_nodes: 1 discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300","127.0.0.1:9301","127.0.0.1:9302"] cluster.initial_master_nodes: ["es-master"] # 添加xpack证书配置 xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.client_authentication: required xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
七. 重启服务
docker restart es-master
docker restart es-node01
docker restart es-node02
八. 修改默认密码
# 进入es-master容器
docker exec -it es-master bash
# 执行修改密码命令并一次输入密码和确认密码
./bin/elasticsearch-setup-passwords interactive
九. 查看集群状态
访问http://127.0.0.1:9200/_cluster/health?pretty 或者 http://127.0.0.1:9200/_cluster/state?pretty 查看集群状态
{
"cluster_name" : "es-cluster",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 2,
"active_primary_shards" : 1,
"active_shards" : 2,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}