Minio集群docker-compose部署Ngnix均衡负载(多机分布式)

目录

概述

部署流程

软件准备：

部署环境：

---

 

概述

        先动手部署，部署成功后再补充，加深理解。

 

部署流程

软件准备：

虚拟机 + Ubuntu18.04 + docker

部署环境：

1、整体思路：

        分别在3台服务器上创建minio服务，这里可以使用docker或者docker-compose的方式比较方便，相当于独立的本地服务；然后在其中1台服务器上进行ngnix的部署以实现多节点的反向代理与均衡负载。已在公司服务器成功部署过，这里再拿虚拟机走一遍。

2、docker-compose安装：docker-compose文件下载

         可以在这个步骤完成之后克隆虚拟机

# 下载上方的docker-compose-linux-x86_64文件
# 拷贝docker-compose-linux-x86_64文件为
sudo cp mv docker-compose-linux-x86_64 /usr/local/bin/docker-compose
# 给权限
chmod +x /usr/local/bin/docker-compose
# 验证
docker-compose version

如图所示 我这里建了3个虚拟机

3、创建挂载路径（3台服务器）

sudo mkdir -p /data/application/oosEnv/minio/minio/data1
sudo mkdir -p /data/application/oosEnv/minio/minio/data1
sudo mkdir -p /data/application/oosEnv/nginx

4、创建docker-compose.yml（3台服务器）

cd /data/application/oosEnv/minio
sudo vim docker-compose.yml
# 粘贴下方代码框yaml文件，保存推出

version: '3.7'
services:
  minio1:  # 注意3台服务器分别为minio1,minio2,minio3
    image: minio/minio
    volumes:
      - /data/application/oosEnv/minio/minio/data1:/data1
      - /data/application/oosEnv/minio/minio/data2:/data2
    environment:
      MINIO_ROOT_USER: admin
      MINIO_ROOT_PASSWORD: admin123456
    network_mode: host
    command: server --console-address ":9001" http://minio{1...3}/data{1...2}
    extra_hosts:
      - "minio1:192.168.xx.131"
      - "minio2:192.168.xx.132"
      - "minio3:192.168.xx.133"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

5、创建nginx的docker-compose.yml （3台服务器）

cd /data/application/oosEnv/nginx
sudo vim docker-compose.yml
# 粘贴下方代码框yaml文件，保存推出

version: '3.7'
services:
  nginx:
    container_name: nginx
    image: nginx:1.19.2-alpine
    volumes:
      - /data/application/oosEnv/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "90:9000"
      - "91:9001"

cd /data/application/oosEnv/nginx
sudo vim nginx.conf
# 粘贴下方代码框conf文件，保存推出

user  nginx;
worker_processes  auto;
 
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
 
events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;

    sendfile        on
    keepalive_timeout  65;

    upstream minio {
        server 192.168.xx.131:9000;
        server 192.168.xx.132:9000;
        server 192.168.xx.133:9000;
    }

    upstream console {
        ip_hash;
        server 192.168.xx.131:9001;
        server 192.168.xx.132:9001;
        server 192.168.xx.133:9001;
    }

    server {
        listen       9000;
        listen  [::]:9000;
        server_name  localhost;

        ignore_invalid_headers off;
        client_max_body_size 0;
        proxy_buffering off;

        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            proxy_connect_timeout 300;
            # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            chunked_transfer_encoding off;

            proxy_pass http://minio;
        }
    }

    server {
        listen       9091;
        listen  [::]:9091;
        server_name  localhost;
        ignore_invalid_headers off;
        client_max_body_size 0;
        proxy_buffering off;
        proxy_request_buffering off;
 
        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-NginX-Proxy true;
            real_ip_header X-Real-IP;
            proxy_connect_timeout 300;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            chunked_transfer_encoding off;
            proxy_pass http://console;
        }
    }
}

 文件结构如下：

 

6、启动minio服务（3台服务器的服务分别起来）

sudo docker-compose pull
sudo docker-compose up -d
sudo docker-compose ps  # 查看容器状态

 随便打开其中1台服务器端口可以正常连接

 

7、启动nginx服务（可以指定其中一台）

sudo docker-compose pull
sudo docker-compose up -d

如图在131的服务器上创建bucket另外两个成功同步。