使用docker快速部署Elasticsearch ssl 环境
一.前言
由于生产环境出于安全考虑,启用了Elasticsearch ssl 安全功能.为了开发测试最大程度与生产环境一致,并可以快速搭建.故有此文档帮助快速搭建Elasticsearch ssl 测试环境.
二.需要技术
- docker基本命令操作
- docker-compose基本命令操作
- Elasticsearch简单操作
三.部署es集群容器
1.准备yaml文件elasticsearch-cloud-ssl.yml
version: "3.7"
services:
esnode-master-01:
image: elasticsearch:7.1.0
container_name: esnode-master-01
hostname: esnode-master-01
volumes:
- type: bind
source: ./elasticsearch/config/elasticsearch.yml
target: /usr/share/elasticsearch/config/elasticsearch.yml
read_only: true
- type: volume
source: esnode-certs
target: /usr/share/elasticsearch/config/certs
- type: volume
source: esnode-master-data-01
target: /usr/share/elasticsearch/data
- type: volume
source: esnode-master-logs-01
target: /usr/share/elasticsearch/logs
environment:
- node.name=esnode-master-01
- network.host=172.18.101.51
- node.master=true
- node.data=false
- node.ingest=true
- node.attr.box_type=master
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
extra_hosts: # 设置容器 hosts
- "esnode-master-01:172.18.101.51"
- "esnode-hot-01:172.18.101.52"
- "esnode-warm-01:172.18.101.53"
- "esnode-master-02:172.18.101.54"
- "esnode-hot-02:172.18.101.55"
- "esnode-warm-02:172.18.101.56"
- "esnode-hot-03:172.18.101.57"
- "esnode-hot-04:172.18.101.58"
- "esnode-warm-03:172.18.101.59"
networks:
qzt-network:
ipv4_address: 172.18.101.51
esnode-hot-01:
image: elasticsearch:7.1.0
container_name: esnode-hot-01
hostname: esnode-hot-01
volumes:
- type: bind
source: ./elasticsearch/config/elasticsearch.yml
target: /usr/share/elasticsearch/config/elasticsearch.yml
read_only: true
- type: volume
source: esnode-certs
target: /usr/share/elasticsearch/config/certs
- type: volume
source: esnode-hot-data-01
target: /usr/share/elasticsearch/data
- type: volume
source: esnode-hot-logs-01
target: /usr/share/elasticsearch/logs
ports:
- 9200:9200
environment:
- node.name=esnode-hot-01
- network.host=172.18.101.52
- node.master=false
- node.data=true
- node.ingest=true
- network.bind_host=172.18.101.52
- node.attr.box_type=master
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
extra_hosts: # 设置容器 hosts
- "esnode-master-01:172.18.101.51"
- "esnode-hot-01:172.18.101.52"
- "esnode-warm-01:172.18.101.53"
- "esnode-master-02:172.18.101.54"
- "esnode-hot-02:172.18.101.55"
- "esnode-warm-02:172.18.101.56"
- "esnode-hot-03:172.18.101.57"
- "esnode-hot-04:172.18.101.58"
- "esnode-warm-03:172.18.101.59"
networks:
qzt-network:
ipv4_address: 172.18.101.52
esnode-warm-01:
image: elasticsearch:7.1.0
container_name: esnode-warm-01
hostname: esnode-warm-01
volumes:
- type: bind
source: ./elasticsearch/config/elasticsearch.yml
target: /usr/share/elasticsearch/config/elasticsearch.yml
read_only: true
- type: volume
source: esnode-certs
target: /usr/share/elasticsearch/config/certs
- type: volume
source: esnode-warm-data-01
target: /usr/share/elasticsearch/data
- type: volume
source: esnode-warm-logs-01
target: /usr/share/elasticsearch/logs
environment:
- node.name=esnode-warm-01
- network.host=172.18.101.53
- node.master=false
- node.data=true
- node.ingest=true
- node.attr.box_type=master
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
extra_hosts: # 设置容器 hosts
- "esnode-master-01:172.18.101.51"
- "esnode-hot-01:172.18.101.52"
- "esnode-warm-01:172.18.101.53"
- "esnode-master-02:172.18.101.54"
- "esnode-hot-02:172.18.101.55"
- "esnode-warm-02:172.18.101.56"
- "esnode-hot-03:172.18.101.57"
- "esnode-hot-04:172.18.101.58"
- "esnode-warm-03:172.18.101.59"
networks:
qzt-network:
ipv4_address: 172.18.101.53
# ==已有网络qzt-network使用===
# networks:
# qzt-network:
# external: true
# ==没有网络qzt-network使用===
networks:
qzt-network:
ipam:
config:
- subnet: 172.18.101.0/24
gateway: 172.18.101.1
volumes:
esnode-master-data-01:
driver: local
driver_opts:
type: none
o: bind
device: /home/qzt-docker/elasticsearch/data/esnode-master-01
esnode-hot-data-01:
driver: local
driver_opts:
type: none
o: bind
device: /home/qzt-docker/elasticsearch/data/esnode-hot-01
esnode-warm-data-01:
driver: local
driver_opts:
type: none
o: bind
device: /home/qzt-docker/elasticsearch/data/esnode-warm-01
esnode-master-logs-01:
driver: local
driver_opts:
type: none
o: bind
device: /home/qzt-docker/elasticsearch/logs/esnode-master-01
esnode-hot-logs-01:
driver: local
driver_opts:
type: none
o: bind
device: /home/qzt-docker/elasticsearch/logs/esnode-hot-01
esnode-warm-logs-01:
driver: local
driver_opts:
type: none
o: bind
device: /home/qzt-docker/elasticsearch/logs/esnode-warm-01
esnode-certs:
driver: local
driver_opts:
type: none
o: bind
device: /home/qzt-docker/elasticsearch/config/certs
简单说明:
- esnode-master-0*:服务名称,作为标识用
- image: 使用的镜像
- hostname:设置容器的主机名称
- volumes:挂载本地目录到容器中,本配置主要配置文件,数据目录,日志目录,证书目录
- environment:设置容器内配置变量
- extra_hosts:设置容器的hosts
- networks:设置网络,本配置容器使用固定ip
- ports:把本地端口映射到容器端口
更多请参考docker-compose v3版本命令详解参考
2.启动集群容器
准备挂载目录:
mkdir -p /home/qzt-docker/elasticsearch/{data,logs}/{esnode-master-01,esnode-hot-01,esnode-warm-01}
mkdir /home/qzt-docker/elasticsearch/config/certs
准备通用配置文件
elasticsearch-cloud-ssl.yml文件所在目录下执行vim ./elasticsearch/config/elasticsearch.yml
elasticsearch.yml文件内容:
cluster.name: elastic-hot-cold-cluster
# 指定数据存放目录,多目录使用逗号分隔
path.data: /usr/share/elasticsearch/data
# 指定日志存放目录
path.logs: /usr/share/elasticsearch/logs
# 设置为true锁住内存,当服务混合部署了多个组件及服务时,应开启此操作,允许es占用足够多的内存
bootstrap.memory_lock: true
network.host: 172.18.101.51
# 设置单个request请求的内存熔断限制,默认是jvm堆的60%(es7.0引入了新的内存熔断机制,会智能判断,规避OOM)
indices.breaker.request.limit: 10%
# 设置segment合并时占用的线程数,配置线程数越多对磁盘io消耗就越大(SSD忽略)
# #index.merge.scheduler.max_thread_count: 1 #新版ES里面,官方不建议在配置文件中写入index相关的配置
# # query请求可使用的jvm内存限制,默认是10%
indices.queries.cache.size: 20%
# # 查询request请求的DSL语句缓存,被缓存的DSL语句下次请求时不会被二次解析,可提升检索性能,默认值是1%
indices.requests.cache.size: 2%
# # 设置字段缓存的最大值,默认无限制
indices.fielddata.cache.size: 30%
# 指定http协议端口,多实例部署时,要注意修改为不同的端口
http.port: 9200
# # 指定TCP协议端口,多实例部署时,要注意修改为不同的端口
transport.tcp.port: 9300
discovery.seed_hosts: ["esnode-master-01","esnode-hot-01","esnode-warm-01"]
# 指定主节点列表,需要在每个节点上配置该参数
cluster.initial_master_nodes: ["172.18.101.51:9300"]
部署容器
docker-compose -f ~/elasticsearch-cloud-ssl.yml up --build -d
3.验证es集群
宿主机中操作:
curl 172.18.101.52:9200
非宿主机操作(假设宿主机ip为192.168.0.2):
curl 192.168.0.2:9200
正常响应信息:
{
"name": "esnode-hot-01",
"cluster_name": "elastic-hot-cold-cluster",
"cluster_uuid": "xrpdMcUITbCK511TUyRW9g",
"version": {
"number": "7.1.0",
"build_flavor": "default",
"build_type": "docker",
"build_hash": "606a173",
"build_date": "2019-05-16T00:43:15.323135Z",
"build_snapshot": false,
"lucene_version": "8.0.0",
"minimum_wire_compatibility_version": "6.8.0",
"minimum_index_compatibility_version": "6.0.0-beta1"
},
"tagline": "You Know, for Search"
}
自此,一个简单的es集群已经部署好.如果没有其他要求的可以使用了.
四.配置transport-ssl
1.启用es的安全功能:
curl -X POST "localhost:9200/_xpack/license/start_trial?acknowledge=true"
2.创建证书
为了简单,直接使用内置的证书创建工具创建
进入其中的一个容器:docker exec -it d40d4b3fd0e4 /bin/bash
容器中执行:
#创建ca证书
bin/elasticsearch-certutil ca
>>输入证书口令
#使用ca证书创建p12证书
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
>>输入证书口令
#复制证书到指定位置
mkdir /tmp/certs
cp elastic-* /tmp/certs/
从容器中复制证书出来到宿主机: docker cp d40d4b3fd0e4:/tmp/certs/ ./elasticsearch/config/
3.设置各节点容器的密钥库中密钥的密码
进入各节点容器执行:
bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
因为是docker部署,我们不直接在外部执行容器内命令:
docker exec -it d40d4b3fd0e4 /bin/bash -c 'bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password'
docker exec -it d40d4b3fd0e4 /bin/bash -c 'bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password'
docker exec -it a1b06729a414 /bin/bash -c 'bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password'
docker exec -it a1b06729a414 /bin/bash -c 'bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password'
docker exec -it 1cc30ad9e738 /bin/bash -c 'bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password'
docker exec -it 1cc30ad9e738 /bin/bash -c 'bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password'
这里最好不要一起复制执行,一条条执行比较好.
4.elasticsearch.yml中添加transport-ssl配置项
vim ./elasticsearch/config/elasticsearch.yml
添加内容为:
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
5.重启es集群
docker-compose 方式:
docker-compose -f ~/elasticsearch-cloud-ssl.yml restart
6.设置密码
集群正确启动后配置安全密码:
配置内置用户密码:(注意保存好)
docker exec -it d40d4b3fd0e4 /bin/bash -c './bin/elasticsearch-setup-passwords auto'
再次访问curl 172.18.101.52:9200
要求出示用户和密码信息
自此,数据传输安全配置完成.更多请参考官网:Encrypting communications in Elasticsearch
五.配置http-ssl
证书直接使用transport-ssl的证书即可
1.设置各节点容器的密钥库中密钥的密码
bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password
因为是docker部署,我们不直接在外部执行容器内命令:
docker exec -it d40d4b3fd0e4 /bin/bash -c '/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password'
docker exec -it d40d4b3fd0e4 /bin/bash -c '/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password'
docker exec -it a1b06729a414 /bin/bash -c '/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password'
docker exec -it a1b06729a414 /bin/bash -c '/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password'
docker exec -it 1cc30ad9e738 /bin/bash -c '/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password'
docker exec -it 1cc30ad9e738 /bin/bash -c '/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password'
2.elasticsearch.yml中添加http-ssl配置项
vim ./elasticsearch/config/elasticsearch.yml
添加内容为:
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12
xpack.security.http.ssl.client_authentication: optional
- client_authentication:optional 表示客户端可以不用必须出示证书
3.重启es集群
docker-compose 方式:
docker-compose -f ~/elasticsearch-cloud-ssl.yml restart
4.验证
使用https方式:
curl 'https://127.0.0.1:9200' -u elastic:4DTLkUuF9JYbemSmzORJ -k -v
自此,es-ssl配置完成.更多配置项请参考官网:Elasticsearch中的安全设置
附完整elasticsearch.yml内容
cluster.name: elastic-hot-cold-cluster
# 指定数据存放目录,多目录使用逗号分隔
path.data: /usr/share/elasticsearch/data
# 指定日志存放目录
path.logs: /usr/share/elasticsearch/logs
# 设置为true锁住内存,当服务混合部署了多个组件及服务时,应开启此操作,允许es占用足够多的内存
bootstrap.memory_lock: true
network.host: 172.18.101.51
# 设置单个request请求的内存熔断限制,默认是jvm堆的60%(es7.0引入了新的内存熔断机制,会智能判断,规避OOM)
indices.breaker.request.limit: 10%
# 设置segment合并时占用的线程数,配置线程数越多对磁盘io消耗就越大(SSD忽略)
# #index.merge.scheduler.max_thread_count: 1 #新版ES里面,官方不建议在配置文件中写入index相关的配置
# # query请求可使用的jvm内存限制,默认是10%
indices.queries.cache.size: 20%
# # 查询request请求的DSL语句缓存,被缓存的DSL语句下次请求时不会被二次解析,可提升检索性能,默认值是1%
indices.requests.cache.size: 2%
# # 设置字段缓存的最大值,默认无限制
indices.fielddata.cache.size: 30%
# 指定http协议端口,多实例部署时,要注意修改为不同的端口
http.port: 9200
# # 指定TCP协议端口,多实例部署时,要注意修改为不同的端口
transport.tcp.port: 9300
discovery.seed_hosts: ["esnode-master-01","esnode-hot-01","esnode-warm-01"]
# 指定主节点列表,需要在每个节点上配置该参数
cluster.initial_master_nodes: ["172.18.101.51:9300"]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12
xpack.security.http.ssl.client_authentication: optional