EFK搭建并开启TLS通信
搭建Elasticsearch & Kibana
安装docker
需要安装docker 和docker compose , 这部分的安装,不再赘述, 可以参考我之前的文章。
创建配置文件
创建instances.yml , 这里给es和kibana生成证书,也可以添加一个client,给filebeat用,不过我这里filebeat使用es的证书, 所以就没有创建client 的证书
instances:
- name: es01
dns:
- es01
- localhost
ip:
- 127.0.0.1
- name: kib01
dns:
- kib01
- localhost
配置 .env
COMPOSE_PROJECT_NAME=es
CERTS_DIR=/usr/share/elasticsearch/config/certificates
VERSION=7.8.0
创建 create-certs.yml , 准备制造证书
version: '2.2'
services:
create_certs:
image: elasticsearch:${VERSION}
container_name: create_certs
command: >
bash -c '
yum install -y -q -e 0 unzip;
if [[ ! -f /certs/bundle.zip ]]; then
bin/elasticsearch-certutil cert --silent --pem --in config/certificates/instances.yml -out /certs/bundle.zip;
unzip /certs/bundle.zip -d /certs;
fi;
chown -R 1000:0 /certs
'
working_dir: /usr/share/elasticsearch
volumes:
- certs:/certs
- .:/usr/share/elasticsearch/config/certificates
networks:
- elastic
volumes:
certs:
driver: local
networks:
elastic:
driver: bridge
接下来, 创建elastic-docker-tls.yml, 给es和kibana配置相关信息
version: '2.2'
services:
es01:
image: elasticsearch:${VERSION}
container_name: es01
environment:
- node.name=es01
- cluster.name=es-docker
- discovery.type=single-node
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- xpack.license.self_generated.type=trial
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=$CERTS_DIR/es01/es01.key
- xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
- xpack.security.http.ssl.certificate=$CERTS_DIR/es01/es01.crt
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
- xpack.security.transport.ssl.certificate=$CERTS_DIR/es01/es01.crt
- xpack.security.transport.ssl.key=$CERTS_DIR/es01/es01.key
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data01:/usr/share/elasticsearch/data
- certs:$CERTS_DIR
ports:
- 9200:9200
networks:
- elastic
healthcheck:
test: curl --cacert $CERTS_DIR/ca/ca.crt -s https://localhost:9200 >/dev/null; if [[ $$? == 52 ]]; then echo 0; else echo 1; fi
interval: 30s
timeout: 10s
retries: 5
kib01:
image: kibana:${VERSION}
container_name: kib01
depends_on: {"es01": {"condition": "service_healthy"}}
ports:
- 5601:5601
environment:
SERVERNAME: localhost
ELASTICSEARCH_URL: https://es01:9200
ELASTICSEARCH_HOSTS: https://es01:9200
ELASTICSEARCH_USERNAME: kibana_system
ELASTICSEARCH_PASSWORD: CHANGEME
ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES: $CERTS_DIR/ca/ca.crt
SERVER_SSL_ENABLED: "true"
SERVER_SSL_KEY: $CERTS_DIR/kib01/kib01.key
SERVER_SSL_CERTIFICATE: $CERTS_DIR/kib01/kib01.crt
volumes:
- certs:$CERTS_DIR
networks:
- elastic
volumes:
data01:
driver: local
certs:
driver: local
networks:
elastic:
driver: bridge
执行步骤
-
先执行如下命令, 注意只执行1次就好
docker-compose -f create-certs.yml run --rm create_certs -
执行完毕后, 可以检查如下目录是否生成证书相关信息
[root@es-jira volumes]# pwd
/var/lib/docker/volumes
[root@es-jira volumes]# ll
total 24
brw------- 1 root root 253, 1 Nov 17 21:45 backingFsBlockDev
drwxr-xr-x 3 root root 19 Nov 17 22:23 es_certs #####注意是这个目录里面
drwx-----x 3 root root 19 Nov 17 22:28 es_data01
-rw------- 1 root root 32768 Nov 17 22:28 metadata.db[root@es-jira _data]# ls -al /var/lib/docker/volumes/es_certs/_data
total 8
drwxr-xr-x 5 1000 root 59 Nov 17 22:27 .
drwxr-xr-x 3 root root 19 Nov 17 22:23 …
-rw------- 1 1000 root 6182 Nov 17 22:27 bundle.zip
drwxrwxr-x 2 1000 root 20 Nov 17 22:27 ca
drwxrwxr-x 2 1000 root 38 Nov 17 22:27 es01
drwxrwxr-x 2 1000 root 40 Nov 17 22:27 kib01 -
检查es和kib是否正确启动
[root@es-jira ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
51fcc32b62f4 kibana:7.8.0 “/usr/local/bin/dumb…” 2 days ago Up 2 days 0.0.0.0:5601->5601/tcp kib01
26bac6de7fd4 elasticsearch:7.8.0 “/tini – /usr/local…” 2 days ago Up 2 days (healthy) 0.0.0.0:9200->9200/tcp, 9300/tcp es01 -
集群启动之后, 生成默认用户的密码
docker exec es01 /bin/bash -c “bin/elasticsearch-setup-passwords
auto --batch --url https://es01:9200”
此处注意保留生成的密码, 我们还要调整下yml配置文件
-
修改elastic-docker-tls.yml文件中 kibana_system 的密码
kib01:
image: kibana:${VERSION}
container_name: kib01
depends_on: {“es01”: {“condition”: “service_healthy”}}
ports:
- 5601:5601
environment:
SERVERNAME: localhost
ELASTICSEARCH_URL: https://es01:9200
ELASTICSEARCH_HOSTS: https://es01:9200
ELASTICSEARCH_USERNAME: kibana_system
ELASTICSEARCH_PASSWORD: CHANGEME >>>>>>>>>>>>>>>>>>>>>>>>>>>此处
ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES: $CERTS_DIR/ca/ca.crt
SERVER_SSL_ENABLED: “true”
SERVER_SSL_KEY: $CERTS_DIR/kib01/kib01.key
SERVER_SSL_CERTIFICATE: $CERTS_DIR/kib01/kib01.crt -
重启es 和kib
docker-compose -f elastic-docker-tls.yml stop
docker-compose -f elastic-docker-tls.yml up -d -
验证登录kib
https://HOSTIP:5601
Filebeat配置
Filebeat的安装, 可以采用多种方式, 二进制或者docker方式都可以,这里我选用tar包安装方式。
重点是filebeat的配置,需要开启ssl及配置证书相关,截取部分配置如下:
output.elasticsearch:
hosts: ["https://127.0.0.1:9200"] ### 注意这里, 是127.0.0.1 ,如果你是其他地址, 在创建证书的时候就要考虑
protocol: "https"
username: elastic
password: rMesfHfEETESJEliJSIv
ssl.certificate_authorities: /root/filebeat/ca.crt
ssl.certificate: /root/filebeat/es01.crt
ssl.key: /root/filebeat/es01.key
index: "tiktok-%{+yyyy.MM.dd}"