docker搭建es集群+skywalking

​

docker搭建es集群+skywalking

1.利用docker镜像在一台服务器上搭建es集群

我的整个过程都是在/usr/local/packages中进行，所以最好先切换到该目录，没有则创建

mkdir -p /usr/local/packages

cd /usr/local/packages

修改linux的一些参数

vim /etc/sysctl.conf

vm.max_map_count=262144

#不重启， 直接生效当前的命令

sysctl -w vm.max_map_count=262144

启动skywalking，将配置文件拷出，便于后期挂在配置

docker run --name oap -d apache/skywalking-oap-server:8.5.0-es7

docker cp oap:/skywalking/config /usr/local/packages/skywalking/

#删除skywalking-oap

docker rm -f oap

编写elasticsearch.yml

network.host: 0.0.0.0

xpack.security.enabled: true

xpack.security.transport.ssl.enabled: true

xpack.security.transport.ssl.keystore.type: PKCS12

xpack.security.transport.ssl.verification_mode: certificate

xpack.security.transport.ssl.keystore.path: elastic-certificates.p12

xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

xpack.security.transport.ssl.truststore.type: PKCS12

xpack.security.audit.enabled: true

#network.host 设置允许其他ip访问，解除ip绑定

#xpack.security 则是安全相关配置，其中ssl的证书需要自己生成

关于证书elastic-certificates.p12

es提供了生成证书的工具elasticsearch-certutil，我们可以在docker实例中生成它，然后复制出来，后面统一使用。

首先运行es实例

docker run -dit --name=es docker.elastic.co/elasticsearch/elasticsearch:7.12.1 /bin/bash

#进入实例内部

docker exec -it es /bin/bash

#第一步

./bin/elasticsearch-certutil ca

Please enter the desired output file [elastic-stack-ca.p12]: #这里直接回车即可

Enter password for elastic-stack-ca.p12 : #这里直接回车即可，不要设置密码

设置完毕后，可以看到新生成的文件

#第二步

./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

下面三项直接回车即可：

Enter password for CA (elastic-stack-ca.p12) :

Please enter the desired output file [elastic-certificates.p12]:

Enter password for elastic-certificates.p12 : #这里直接回车即可，不要设置密码，否则后面ES会启动不了

Certificates written to /usr/local/elasticsearch-7.12.1/elastic-certificates.p12

设置完毕后，会在/usr/local/elasticsearch-7.12.1下看到新生成的文件：

elastic-certificates.p12

这个生成elastic-certificates.p12 就是我们需要使用的。

复制出证书, ctrl+d退出容器内部

docker cp es:/usr/share/elasticsearch/elastic-certificates.p12 .

#设置权限，不然es启动报错

chmod 777 elastic-certificates.p12

创建子目录用于挂在数据和日志

mkdir -p /usr/local/packages/es01/data

mkdir -p /usr/local/packages/es01/logs

mkdir -p /usr/local/packages/es02/data

mkdir -p /usr/local/packages/es02/logs

mkdir -p /usr/local/packages/es03/data

mkdir -p /usr/local/packages/es03/logs

chmod 777 es* -R

编写docker-compose.yml文件

version: '2.2'

services:

es01:

image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1

container_name: es01

environment:

- node.name=es01

- cluster.name=es-docker-cluster

- discovery.seed_hosts=es02,es03

- cluster.initial_master_nodes=es01,es02,es03

- bootstrap.memory_lock=true

- "ES_JAVA_OPTS=-Xms2g -Xmx2g"

ulimits:

memlock:

soft: -1

hard: -1

nofile:

soft: 65536

hard: 65536

volumes:

- ./es01/data:/usr/share/elasticsearch/data

- ./es01/logs:/usr/share/elasticsearch/logs

- ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml

- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12

ports:

- 9200:9200

networks:

- elastic

es02:

image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1

container_name: es02

environment:

- node.name=es02

- cluster.name=es-docker-cluster

- discovery.seed_hosts=es01,es03

- cluster.initial_master_nodes=es01,es02,es03

- bootstrap.memory_lock=true

- "ES_JAVA_OPTS=-Xms2g -Xmx2g"

ulimits:

memlock:

soft: -1

hard: -1

nofile:

soft: 65536

hard: 65536

volumes:

- ./es02/data:/usr/share/elasticsearch/data

- ./es02/logs:/usr/share/elasticsearch/logs

- ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml

- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12

networks:

- elastic

es03:

image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1

container_name: es03

environment:

- node.name=es03

- cluster.name=es-docker-cluster

- discovery.seed_hosts=es01,es02

- cluster.initial_master_nodes=es01,es02,es03

- bootstrap.memory_lock=true

- "ES_JAVA_OPTS=-Xms2g -Xmx2g"

ulimits:

memlock:

soft: -1

hard: -1

nofile:

soft: 65536

hard: 65536

volumes:

- ./es03/data:/usr/share/elasticsearch/data

- ./es03/logs:/usr/share/elasticsearch/logs

- ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml

- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12

networks:

- elastic

kibana:

image: kibana:7.12.1

container_name: kibana

environment:

- SERVER_NAME=kibana

- ELASTICSEARCH_HOSTS=http://es01:9200

- XPACK_MONITORING_ENABLED=true

ports:

- 5601:5601

networks:

- elastic

depends_on:

- es01

skywalking-oap:

image: apache/skywalking-oap-server:8.5.0-es7

container_name: skywalking-oap

restart: always

depends_on:

- es01

ports:

- 11800:11800

- 12800:12800

environment:

- TZ=Asia/Shanghai

- SW_STORAGE=elasticsearch7

- SW_STORAGE_ES_CLUSTER_NODES=es01:9200

volumes:

- ./skywalking/config:/skywalking/config

networks:

- elastic

skywalking-ui:

image: apache/skywalking-ui:8.5.0

container_name: skywalking-ui

restart: always

depends_on:

- skywalking-oap

ports:

- 1300:8080

environment:

- TZ=Asia/Shanghai

- SW_OAP_ADDRESS= skywalking-oap:12800

networks:

- elastic

networks:

elastic:

driver: bridge

用docker-compose启动命名

docker-compose up -d

进入其中一台

docker exec -it es01 /bin/bash

设置密码

#我这里用的auto，也可以自己定义

./bin/elasticsearch-setup-passwords -h

#控制台返回信息

Sets the passwords for reserved users

Commands

--------

auto - Uses randomly generated passwords

interactive - Uses passwords entered by a user

Non-option arguments:

command

Option Description

------ -----------

-E <KeyValuePair> Configure a setting

-h, --help Show help

-s, --silent Show minimal output

-v, --verbose Show verbose output

#然后运行自动生成密码

./bin/elasticsearch-setup-passwords auto

#控制台返回密码

Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.

The passwords will be randomly generated and printed to the console.

Please confirm that you would like to continue [y/N]y

Changed password for user apm_system

PASSWORD apm_system = YxVzeT9B2jEDUjYp66Ws

Changed password for user kibana

PASSWORD kibana = 8NnThbj0N02iDaTGhidU

Changed password for user logstash_system

PASSWORD logstash_system = 9nIDGe7KSV8SQidSk8Dj

Changed password for user beats_system

PASSWORD beats_system = qeuVaf1VEALpJHfEUOjJ

Changed password for user remote_monitoring_user

PASSWORD remote_monitoring_user = DtZCrCkVTZsinRn3tW3D

Changed password for user elastic

PASSWORD elastic = K9fPmfki6XKGc693nLv5

重启es集群

docker restart es01 es02 es03

修改skywalking-oap的

skywalking此时已经停止，无法进入容器内部，修改挂在外面的配置文件就行

#修改skywalking/config下的application.yml

vim skywalking/config/application.yml

#找到elasticsearch7下的

user: ${SW_ES_USER:elastic}

password: ${SW_ES_PASSWORD:K9fPmfki6XKGc693nLv5}

重启skywalking-oap

docker restart skywalking-oap

修改es密码

docker exec -it es01 /bin/bash

#创建一个临时的超级用户ryan

./bin/elasticsearch-users useradd ryan -r superuser

Enter new password:

ERROR: Invalid password...passwords must be at least [6] characters long

[root@cfeeab4bb0eb elasticsearch]# ./bin/elasticsearch-users useradd ryan -r superuser

Enter new password:

Retype new password:

#用这个用户去修改密码

curl -XPUT -u ryan:ryan123 http://localhost:9200/_xpack/security/user/elastic/_password -H "Content-Type: application/json" -d '

{

"password": "q5f2qNfUJQyvZPIz57MZ"

}'

2.安装docker-compose

curl -L https://get.daocloud.io/docker/compose/releases/download/1.25.4/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

#添加这个可以使用docker-compose命令

chmod +x /usr/local/bin/docker-compose

ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

#查看docker-compose版本

docker-compose -version

#用docker-compose启动命名

docker-compose up -d

#开放所需的端口

firewall-cmd --list-all

firewall-cmd --add-port=7000-8200/tcp --permanent

firewall-cmd --reload

3.agent.config要写入的

这里的agent.config是指在将服务打包时，需要将agent的包一起打包进docker镜像中，修改这里的agent.config

Dockerfile中编写

FROM openjdk:8-jdk

WORKDIR /app

ADD bas-service-1.0-SNAPSHOT.jar bas.jar

EXPOSE 7061

COPY agent8.5 /usr/local/agent

ENV PARAM=""

ENTRYPOINT ["sh","-c","java $PARAM -jar bas.jar"]

修改agent8.5/config/agent.config

ip写到你的skywalking安装地址

plugin.toolkit.log.grpc.reporter.server_host=${SW_GRPC_LOG_SERVER_HOST:192.168.x.xx}

plugin.toolkit.log.grpc.reporter.server_port=${SW_GRPC_LOG_SERVER_PORT:11800}

plugin.toolkit.log.grpc.reporter.max_message_size=${SW_GRPC_LOG_MAX_MESSAGE_SIZE:10485760}

plugin.toolkit.log.grpc.reporter.upstream_timeout=${SW_GRPC_LOG_GRPC_UPSTREAM_TIMEOUT:30}

打包镜像

docker build -t bas-sky:1.0 .

4.创建docker容器

docker run -d --name bas --restart always --net host -e PARAM="-Dserver.port=7061 -javaagent:/usr/local/agent/skywalking-agent.jar -Dskywalking.agent.service_name=bas -Dskywalking.collector.backend_service=192.168.5.24:11800" bas-sky:1.0

5.docker可视化界面

docker pull portainer/portainer

docker volume create portainer_data

docker run -d -p 9000:9000 --name portainer --restart always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

docker run -d -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock --name portainer --restart=always portainer/portainer

6.定时删除7天前日志

1.创建脚本文件

vim es_index_delete.sh

2.写入

LAST_DATA=`date -d "-7 days" "+%Y-%m-%d"`

curl -XDELETE http://192.168.5.24:9200/*-${LAST_DATA}

3.修改脚本权限

chmod 777 es_index_delete.sh

4.设定定时任务

crontab -e

5.写入

0 1 * * * /usr/local/packages/es_index_del.sh

​