Linux 搭建部署Minio分布式集群

操作系统：Centos 8.5
安装介质：Minio RELEASE.2023-01-31T02-24-19Z
说明：本文仅供参考。

---

不多说直接开整

前期准备

 各节点信息

        确保服务器池中的所有节点都使用具有相同容量（例如TB）的相同类型（NVMe、SSD 或 HDD）的驱动器。MinIO 不区分驱动器类型，也不受益于混合存储类型。此外。MinIO 将每个驱动器使用的大小限制为部署中的最小驱动器。例如，如果部署有 15 个 10TB 驱动器和 1 个 1TB 驱动器，MinIO 将每个驱动器的容量限制为 1TB。

IP	用途	磁盘挂载点	数据盘大小
192.168.0.61	节点1	/mnt/data1、/mnt/data2、/mnt/data3、/mnt/data4	200G * 4
192.168.0.62	节点2	/mnt/data1、/mnt/data2、/mnt/data3、/mnt/data4	200G * 4
192.168.0.63	节点3	/mnt/data1、/mnt/data2、/mnt/data3、/mnt/data4	200G * 4
192.168.0.64	节点4	/mnt/data1、/mnt/data2、/mnt/data3、/mnt/data4	200G * 4
192.168.0.100	负载均衡

分布式Minio使用的磁盘里必须是干净的，里面没有数据。

时间同步

节点1-节点4

yum -y install ntp 
systemctl enable ntpd 
systemctl start ntpd 

timedatectl set-ntp yes ntpdate -u 192.168.0.200  ##使用自己的时间服务器或者替换国内时间服务器： cn.pool.ntp.org 
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

分布式Minio里的节点时间差不能超过3秒，你可以使用NTP 来保证时间一致。 

创建运行环境

节点1-节点4

mkdir -vp /opt/minio-server/{bin,log}

将 Minio RELEASE.2023-01-31T02-24-19Z 上传至/opt/minio-server/bin目录下

启动&停止

节点1

vi /opt/minio-server/bin/start.sh

export MINIO_ROOT_USER=admin 
export MINIO_ROOT_PASSWORD=12345678 
MINIO_HOME=/opt/minio-server 

nohup  ${MINIO_HOME}/bin/minio server --address ":9000" --console-address ":8000" \
http://192.168.0.61:9000/mnt/data1 http://192.168.0.61:9000/mnt/data2 \
http://192.168.0.61:9000/mnt/data3 http://192.168.0.61:9000/mnt/data4 \
http://192.168.0.62:9000/mnt/data1 http://192.168.0.62:9000/mnt/data2 \
http://192.168.0.62:9000/mnt/data3 http://192.168.0.62:9000/mnt/data4 \
http://192.168.0.63:9000/mnt/data1 http://192.168.0.63:9000/mnt/data2 \
http://192.168.0.63:9000/mnt/data3 http://192.168.0.63:9000/mnt/data4 \
http://192.168.0.64:9000/mnt/data1 http://192.168.0.64:9000/mnt/data2 \
http://192.168.0.64:9000/mnt/data3 http://192.168.0.64:9000/mnt/data4 > ${MINIO_HOME}/log/minio-9000.log 2>&1 &

请仔细看脚本中的启动顺序，结合自己服务器去修改，每个服务器4个磁盘分区

Nginx负载

负载均衡

vi  /home/nginx/conf/nginx.conf

#user  nobody;
worker_processes  1;
#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
#pid        logs/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    #添加配置
    include  /home/nginx/nginx/conf/conf.d/*.conf;
    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';
    #access_log  logs/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    #keepalive_timeout  0;
    keepalive_timeout  65;
    #gzip  on;
}

nginx.conf 中默认配置不动，只添加include 即可

vi /home/nginx/nginx/conf/conf.d/minio.conf

upstream minio {
     server 192.168.0.61:9000;
     server 192.168.0.62:9000;
     server 192.168.0.63:9000;
     server 192.168.0.64:9000;
}
upstream console {
     ip_hash;
     server 192.168.0.61:8000;
     server 192.168.0.62:8000;
     server 192.168.0.63:8000;
     server 192.168.0.64:8000;
}

server {
     listen 9000;
     server_name localhost;
     ignore_invalid_headers off;
     client_max_body_size 0;
     proxy_buffering off;
location / {
     proxy_set_header Host $http_host;
     proxy_set_header X-Real-IP $remote_addr;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header X-Forwarded-Proto $scheme;
     proxy_set_header Upgrade $http_upgrade;
     proxy_set_header X-NginX-Proxy true;
     proxy_set_header Connection "upgrade";
     proxy_next_upstream     http_500 http_502 http_503 http_504 error timeout invalid_header;
     proxy_connect_timeout 300;
     proxy_http_version 1.1;
     proxy_set_header Connection "";
     chunked_transfer_encoding off;
     proxy_pass http://minio;
     }
}

server {
     listen 8000;
     server_name localhost;
     ignore_invalid_headers off;
     client_max_body_size 0;
     proxy_buffering off;
location / {
     proxy_set_header Host $http_host;
     proxy_set_header X-Real-IP $remote_addr;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header X-Forwarded-Proto $scheme;
     proxy_set_header X-NginX-Proxy true;
     proxy_set_header Upgrade $http_upgrade;
     proxy_set_header Connection "upgrade";
     proxy_next_upstream     http_500 http_502 http_503 http_504 error timeout invalid_header;
     proxy_connect_timeout 300;
     proxy_http_version 1.1;
     proxy_set_header Connection "";
     chunked_transfer_encoding off; 
     proxy_pass http://console;
     }
}

借鉴了各路大神的代理方式，发现访问不了桶上传后的文件，结合自己实际情况加上了相关参数

访问

浏览器访问  http://192.168.0.100:8000/login 密码start.sh、stop.sh 中已经配置好了

admin/12345678