活动地址:毕业季·进击的技术er
docker swarm 布署minio集群
环境:ubuntu18.04服务器 4台,docker(docker-compose,docker swarm)
分为主节点(1个)和从节点(3个):
192.168.44.134 root1
192.168.44.135 root2
192.168.44.136 root3
192.168.44.137 root4
一、初始化主节点
1.主节点下执行以下命令
docker swarm init --advertise-addr 192.168.44.134
如果出现:Error response from daemon: --live-restore daemon configuration is incompatible with swarm mode
可尝试此解决方法:
vim /etc/docker/daemon.json --- "live-restore": true 修改true为false "live-restore": f --- # 重启Docker systemctl restart docker.service
后面若出现这个错也是一样
2.查看节点信息
docker node ls
二、将子节点添加到主节点swarm
1.在子节点运行docker swarm join ...
在
docker swarm init
中得到的输出docker swarm join --token SWMTKN-1-25i33klsk74p0oarwrbsyge4ggmrvto33o42iinm8oho1geksu-6vqqb5s56mdjrlpausbt2hhah 10.0.0.151:2377
在各个子节点运行若没有则在主节点执行
docker swarm join-token worker
,也可获得
2.在主节点查看节点信息
docker node ls
# MANAGER 显示为Leader是主节点,其余的为工作节点
三、在主节点为minio生成密钥
1.输入命令(也是设置账号和密码
,后面yaml文件里面使用)
echo "admin" | docker secret create access_key -
echo "admin123456" | docker secret create secret_key -
# 分布式Minio里所有的节点需要有同样的access秘钥和secret秘钥,这样这些节点才能建立联接。为了实现这个,你需要在执行minio server命令之前,先将access秘钥和secret秘钥export成环境变量。
四、给节点打标签,主节点执行
# 节点打标签,绑定容器与节点之间一一对应的关系
docker node update --label-add minio1=true 7czsx0pl2u50bpjllg37dmvu1
docker node update --label-add minio2=true zx76va5pcadgblj8hnxms1x0z
docker node update --label-add minio3=true 64odnhwvebsjhw1tox721kyvx
docker node update --label-add minio4=true dz7idun0rtjcqx416y0jbwxvp
五、部署分布式minio服务
# 运行命令,部署minio,服务名:minio_stack,docker-compose-secrets.yaml文件见最后
docker stack deploy --compose-file=docker-compose-secrets.yaml minio_stack
部署成功后通过浏览器访问:http://IP:9001 来访问minio server客户端
查看正在运行的服务列表:
docker service ls
查看哪些节点正在运行该服务:
docker stack ps minio_stack
删除集群
docker stack rm minio_stack
六、在各个节点中都会运行一个容器
查看其中一个子节点:
docker ps
查看日志可以查看minio客户端地址(这里这个地址是容器内地址,可以映射出来)
docker logs --tail 100 -f <CONTAINED>
七、minio数据迁移
docker启动mc实例,并进入容器内部
docker run -it --entrypoint=/bin/sh minio/mc
通过mc命令连接两个minio服务
mc alias set minio1 http://127.0.0.1:9000 adminminio adminminio
mc alias set 名称 服务地址 用户名 密码
迁移数据
#全量迁移,重名文件不覆盖,bucket不存在会自动创建
mc mirror minio1 minio2
#只是迁移某个bucket,以test为例,目标的bucket需要提前建好
mc mirror minio1/test minio2/test
#覆盖重名文件,加--overwrite
mc mirror --overwrite minio1/test minio2/test
docker-compose-secrets.yaml
version: '3.3'
services:
minio1:
image: minio/minio:RELEASE.2022-04-16T04-26-02Z
hostname: minio1
volumes:
- /home/liu/minio/data:/export
ports:
- "9000:9000"
- "9001:9001"
networks:
- minio_distributed
deploy:
restart_policy:
delay: 10s
max_attempts: 10
window: 60s
placement:
constraints:
- node.labels.minio1==true
command: server --address ':9000' --console-address ':9001' http://minio{1...4}/export
secrets:
- secret_key
- access_key
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
minio2:
image: minio/minio:RELEASE.2022-04-16T04-26-02Z
hostname: minio2
volumes:
- /home/liu/minio/data:/export
ports:
- "9002:9000"
- "9003:9001"
networks:
- minio_distributed
deploy:
restart_policy:
delay: 10s
max_attempts: 10
window: 60s
placement:
constraints:
- node.labels.minio2==true
command: server --address ':9000' --console-address ':9001' http://minio{1...4}/export
secrets:
- secret_key
- access_key
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
minio3:
image: minio/minio:RELEASE.2022-04-16T04-26-02Z
hostname: minio3
volumes:
- /home/liu/minio/data:/export
ports:
- "9004:9000"
- "9005:9001"
networks:
- minio_distributed
deploy:
restart_policy:
delay: 10s
max_attempts: 10
window: 60s
placement:
constraints:
- node.labels.minio3==true
command: server --address ':9000' --console-address ':9001' http://minio{1...4}/export
secrets:
- secret_key
- access_key
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
minio4:
image: minio/minio:RELEASE.2022-04-16T04-26-02Z
hostname: minio4
volumes:
- /home/liu/minio/data:/export
ports:
- "9006:9000"
- "9007:9001"
networks:
- minio_distributed
deploy:
restart_policy:
delay: 10s
max_attempts: 10
window: 60s
placement:
constraints:
- node.labels.minio4==true
command: server --address ':9000' --console-address ':9001' http://minio{1...4}/export
secrets:
- secret_key
- access_key
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
networks:
minio_distributed:
driver: overlay
secrets:
secret_key:
external: true
access_key:
external: true