minio集群

1 集群部署

  minio集群的搭建并不复杂，别人也有很多的例子，这里只是先把自己的集群搭建记录下来，重点是后面的章节，遇到问题如何解决。

1.1 修改主机名

hostnamectl set-hostname minio1
hostnamectl set-hostname minio2
hostnamectl set-hostname minio3
hostnamectl set-hostname minio4

1.2 修改host

cat >> /etc/hosts <<EOF
10.101.12.123 minio1
10.101.12.124 minio2
10.101.12.125 minio3
10.101.12.126 minio4
EOF

1.3 开放端口

systemctl start firewalld
systemctl enable firewalld
systemctl status firewalld
# 开放端口
firewall-cmd --zone=public --add-port=9000/tcp --permanent
firewall-cmd --zone=public --add-port=19000/tcp --permanent
# 生效
firewall-cmd --reload
firewall-cmd --list-ports

1.4 上传minio

mkdir -p /appdata/minio
mkdir -p /appdata/minio_data
# 上传minio、run.sh到/appdata/minio

1.5 加入systemctl管理

cat > /usr/lib/systemd/system/minio.service <<EOF
[Unit]
Description=Minio service
Documentation=https://docs.minio.io/
 
[Service]
WorkingDirectory=/appdata/minio/
ExecStart=/appdata/minio/run.sh
Restart=on-failure
RestartSec=5
 
[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable minio

1.6 run.sh

#!/bin/bash
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
/appdata/minio/minio server --console-address ':19000' --config-dir /etc/minio http://minio1/appdata/minio_data http://minio2/appdata/minio_data http://minio3/appdata/minio_data http://minio4/appdata/minio_data

1.7 启动minio

systemctl start minio

1.8 安装mc 

cd /appdata
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
cp -a mc /usr/bin

配置accessKey和secretKey，其中myminio为minio的别名,下面minio是定义的accessKey,而minio123是secretKey

mc config host add eayc_test http://10.101.12.123:9000 minio minio123

2 问题处理

2.1  Error: Read failed. Insufficient number of disks online (*errors.errorString)

因为19000端口无法访问，因此执行journalctl -u minio.service -f，查看minio服务的日志，错误信息如下：

Sep 10 10:08:06 minio1 run.sh[5729]: 5: cmd/prepare-storage.go:268:cmd.connectLoadInitFormats()
Sep 10 10:08:06 minio1 run.sh[5729]: 4: cmd/prepare-storage.go:317:cmd.waitForFormatErasure()
Sep 10 10:08:06 minio1 run.sh[5729]: 3: cmd/erasure-server-pool.go:90:cmd.newErasureServerPools()
Sep 10 10:08:06 minio1 run.sh[5729]: 2: cmd/server-main.go:614:cmd.newObjectLayer()
Sep 10 10:08:06 minio1 run.sh[5729]: 1: cmd/server-main.go:531:cmd.serverMain()
Sep 10 10:08:06 minio1 run.sh[5729]: Waiting for a minimum of 2 disks to come online (elapsed 1m53s)
Sep 10 10:08:07 minio1 run.sh[5729]: Unable to read 'format.json' from http://minio2:9000/appdata/minio_data: Server expects 'storage' API version 'v39', instead found 'v39' - *rolling upgrade is not allowed* - please make sure all servers are running the same MinIO version (RELEASE.2021-09-03T03-56-13Z)
Sep 10 10:08:07 minio1 run.sh[5729]: Unable to read 'format.json' from http://minio3:9000/appdata/minio_data: Server expects 'storage' API version 'v39', instead found 'v39' - *rolling upgrade is not allowed* - please make sure all servers are running the same MinIO version (RELEASE.2021-09-03T03-56-13Z)
Sep 10 10:08:07 minio1 run.sh[5729]: Unable to read 'format.json' from http://minio4:9000/appdata/minio_data: Server expects 'storage' API version 'v39', instead found 'v39' - *rolling upgrade is not allowed* - please make sure all servers are running the same MinIO version (RELEASE.2021-09-03T03-56-13Z)
Sep 10 10:08:07 minio1 run.sh[5729]: Disk `/appdata/minio_data` the same as the system root disk.
Sep 10 10:08:07 minio1 run.sh[5729]: Disk will not be used. Please supply a separate disk and restart the server.
Sep 10 10:08:07 minio1 run.sh[5729]: API: SYSTEM()
Sep 10 10:08:07 minio1 run.sh[5729]: Time: 10:08:07 CST 09/10/2024
Sep 10 10:08:07 minio1 run.sh[5729]: Error: Read failed. Insufficient number of disks online (*errors.errorString)

 查看9000端口，看似没什么问题，为什么会出现上面的问题呢？

我这里出现的一种情况不是出自磁盘，而是因为虚拟机重启，防火墙的问题，启动端口，重启防火墙就好了。