Elasticsearch (一) 基于Docker-compose 搭建集群

最新推荐文章于 2024-03-24 21:38:39 发布

m0_67392931

最新推荐文章于 2024-03-24 21:38:39 发布

阅读量1.1k

点赞数

分类专栏： java 文章标签： docker elasticsearch 容器后端 apache

本文链接：https://blog.csdn.net/m0_67392931/article/details/126358456

版权

java 专栏收录该内容

272 篇文章 6 订阅

订阅专栏

基于Docker-compose 搭建Elasticsearch集群

1.前言

1.前言

基于docker-compose 安装 elasticsearch 集群，kibana可视化组件，通过cerebro工具监控集群信息

Elasticsearch 官网：https://www.elastic.co/cn/

Docker

安装

yum install -y docker

启动

systemctl start docker

测试

docker --version

Compose

Docker Compose是一个用来定义和运行复杂应用的Docker工具。一个使用Docker容器的应用，通常由多个容器组成。使用Docker Compose不再需要使用shell脚本来启动容器。
Compose 通过一个配置文件来管理多个Docker容器，在配置文件中，所有的容器通过services来定义，然后使用docker-compose脚本来启动，停止和重启应用，和应用中的服务以及所有依赖服务的容器，非常适合组合使用多个容器进行开发的场景。

安装

 sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

对二进制文件应用可执行权限：

sudo chmod +x /usr/local/bin/docker-compose

测试

docker-compose --version

Compose和Docker兼容性

compose文件格式版本 docker版本
3.4 17.09.0+
3.3 17.06.0+
3.2 17.04.0+
3.1 1.13.1+
3.0 1.13.0+
2.3 17.06.0+
2.2 1.13.0+
2.1 1.12.0+
2.0 1.10.0+
1.0 1.9.1.+

配置文件

官方文档

创建 docker-compose-elasticsearch.yaml

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.15.2
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./elasticsearch/config/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - elastic
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.15.2
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./elasticsearch/config/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
      - data02:/usr/share/elasticsearch/data
    networks:
      - elastic
  es03:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.15.2
    container_name: es03
    environment:
      - node.name=es03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./elasticsearch/config/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
      - data03:/usr/share/elasticsearch/data
    networks:
      - elastic
  kibana:
    image: docker.elastic.co/kibana/kibana:7.15.2
    container_name: kibana
    volumes:
      - ./kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml
    ports:
      - 5601:5601
    environment:
      ELASTICSEARCH_URL: http://es01:9200
      ELASTICSEARCH_HOSTS: http://es01:9200
    depends_on:
      - es01
    networks:
      - elastic
volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local

networks:
  elastic:
    driver: bridge

elasticsearch/config/elasticsearch.yml

network.host: 0.0.0.0
http.port: 9200
# 开启es跨域
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization
# 开启安全控制
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/elastic-certificates.p12

kibana/config/kibana.yml

server.name: kibana
server.host: "0.0.0.0"
xpack.monitoring.ui.container.elasticsearch.enabled: true
elasticsearch.username: "elastic"  # es账号
elasticsearch.password: "*******"   # es密码 进入容器设置的密码
i18n.locale: zh-CN # 中文

上传并运行

上传文件 docker-compose-elasticsearch.yaml 到自己创建的目录
在这里插入图片描述

elasticsearch 安全策略

生成证书
依次执行命令
1.创建临时容器
2.进入容器
3.创建ca [直接回车不用输入密码]
4.创建证书 [直接回车不用输入密码]
5.退出容器并将容器中的证书拷贝出来
6.删除这个临时容器

docker run -d  docker.elastic.co/elasticsearch/elasticsearch:7.15.2 --name=es
docker exec -it es /bin/bash
./bin/elasticsearch-certutil ca
./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
docker cp es:/usr/share/elasticsearch/elastic-certificates.p12 .
docker rm -f es

如果创建证书的时候输入了密码需要在容器内执行否则这个证书在集群启动的时候会认证不通过。
./bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
./bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

启动所有容器

docker-compose -f docker-compose-elasticsearch.yaml up -d

进入容器修改密码

docker exec -it es01 /bin/bash

[root@localhost elasticsearch-cluster]# docker exec -it es01 /bin/bash
bash-4.4# ./bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [apm_system]: 
Reenter password for [apm_system]: 
Enter password for [kibana_system]: 
Reenter password for [kibana_system]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Enter password for [beats_system]: 
Reenter password for [beats_system]: 
Enter password for [remote_monitoring_user]: 
Reenter password for [remote_monitoring_user]: 
Changed password for user [apm_system]
Changed password for user [kibana_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

通过命令 docker-compose -f docker-compose-elasticsearch.yaml up -d 运行

```bash
[root@localhost elasticsearch-cluster]# docker-compose -f docker-compose-elasticsearch.yaml up -d
Creating network "elasticsearch-cluster_elastic" with driver "bridge"
Creating volume "elasticsearch-cluster_data01" with local driver
Creating volume "elasticsearch-cluster_data02" with local driver
Creating volume "elasticsearch-cluster_data03" with local driver
Pulling es01 (docker.elastic.co/elasticsearch/elasticsearch:7.15.2)...
Trying to pull repository docker.elastic.co/elasticsearch/elasticsearch ... 
7.15.2: Pulling from docker.elastic.co/elasticsearch/elasticsearch
009c11f4ddee: Pull complete
8772b99d888d: Pull complete
bd8b744bf3bf: Pull complete
2a41be2c565a: Pull complete
e7e9200dd33e: Pull complete
Digest: sha256:a1dce08d504b22e87adc849c94dcae53f6a0bd12648a4d99d7f9fc07bb2e8a3e
Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.15.2
Creating es02 ... done
Creating es01 ... done
Creating es03 ... done

通过 docker ps 命令查看运行中的容器

[root@localhost elasticsearch-cluster]# docker ps
CONTAINER ID        IMAGE                                                  COMMAND                  CREATED             STATUS              PORTS                              NAMES
f43b017dd23a        docker.elastic.co/elasticsearch/elasticsearch:7.15.2   "/bin/tini -- /usr..."   17 seconds ago      Up 12 seconds       9200/tcp, 9300/tcp                 es03
7ed565d7eb4e        docker.elastic.co/elasticsearch/elasticsearch:7.15.2   "/bin/tini -- /usr..."   17 seconds ago      Up 12 seconds       0.0.0.0:9200->9200/tcp, 9300/tcp   es01
fb89e106eea2        docker.elastic.co/elasticsearch/elasticsearch:7.15.2   "/bin/tini -- /usr..."   17 seconds ago      Up 12 seconds       9200/tcp, 9300/tcp                 es02

通过 docker logs -f es01 查看容器的运行日志

[root@localhost elasticsearch-cluster]# docker logs -f es01
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.elasticsearch.bootstrap.Elasticsearch (file:/usr/share/elasticsearch/lib/elasticsearch-7.15.2.jar)
WARNING: Please consider reporting this to the maintainers of org.elasticsearch.bootstrap.Elasticsearch
WARNING: System::setSecurityManager will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.elasticsearch.bootstrap.Security (file:/usr/share/elasticsearch/lib/elasticsearch-7.15.2.jar)
WARNING: Please consider reporting this to the maintainers of org.elasticsearch.bootstrap.Security
WARNING: System::setSecurityManager will be removed in a future release
{"type": "server", "timestamp": "2021-12-30T04:49:03,608Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "es-docker-cluster", "node.name": "es01", "message": "version[7.15.2], pid[6], build[default/docker/93d5a7f6192e8a1a12e154a2b81bf6fa7309da0c/2021-11-04T14:04:42.515624022Z], OS[Linux/3.10.0-1160.el7.x86_64/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/17.0.1/17.0.1+12]" }
..........
..........
..........

开放防火墙端口：9200，9300

[root@localhost elasticsearch-cluster]# firewall-cmd --zone=public --add-port=9200/tcp --add-port=9300/tcp --permanent&&firewall-cmd --reload
success
success
[root@localhost elasticsearch-cluster]# firewall-cmd --list-ports
6379/tcp 9200/tcp 9300/tcp
[root@localhost elasticsearch-cluster]#

访问Kibana
在这里插入图片描述

查看节点

[root@localhost elasticsearch-cluster]# curl -X GET "localhost:9200/_cat/nodes?v=true&pretty"
ip         heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
172.18.0.4           64          89   6    0.11    0.18     0.19 cdfhilmrstw *      es02
172.18.0.3           68          89   6    0.11    0.18     0.19 cdfhilmrstw -      es01
172.18.0.2           53          89   6    0.11    0.18     0.19 cdfhilmrstw -      es03
[root@localhost elasticsearch-cluster]#