中间件——MinIO:部署篇
文章目录
前言
在部署MinIO集群时,踩了许多坑,故在此做下记录
一、部署MinIO的限制条件
单机Minio服务存在单点故障,如果是一个有N块硬盘的分布式Minio,只要有N/2硬盘在线,数据就是安全的、可读的。不过需要至少有N/2+1个硬盘在线,才能创建新的对象。
二、部署步骤
1.前置环境
1.1 同步系统时间
timedatectl status # 查看时间
timedatectl set-ntp yes # 设置NTP时间同步
1.2 修改本地主机记录
sudo vim /etc/hosts
## 添加三台主机IP,后面部署MinIO指令时有用
节点1的IP node1
节点2的IP node2
节点3的IP node-nginx
2.下载MinIO
在 ~ 目录下操作
# 创建目录
mkdir /home/用户名/minio/{app, config,data1,data2,data3,data4 logs } -p
cd /home/用户名/minio
# 获取文件
wget https://dl.min.io/server/minio/release/linux-amd64/minio
# 修改文件权限
chmod a+x ./app/minio
3. 挂载数据盘
在 ~/minio 目录下操作
## 创建格式化磁盘并挂载的脚本
vim fdisk-minio.sh
## ---------------脚本内容如下---------------
#!/bin/bash
cnt=1;
for i in b c d e ## 注意这个名称列表,根据挂载的磁盘名称而决定
do
sudo echo "current operating disk "$i"...........";
sudo fdisk "/dev/sd$i";
sudo mkfs -t ext4 "/dev/sd"$i"1";
sudo mount -t ext4 "/dev/sd"$i"1" "/home/用户名/minio/data"$cnt"";
cnt=`expr $cnt + 1`;
sudo echo "after plus, cnt = $cnt";
done
sudo echo "finished operating..... see result \n";
sudo df -h;
## 修改脚本的权限
sudo chmod a+x ./fdisk-minio.sh
## 同时查看下data{1...4}是否归属于普通用户组,切记不能属于ROOT组否则将会出问题
## 执行脚本,输四遍 n->p->1->回车->回车->w, 即可
./fdisk-minio.sh
## 查看挂载是否成功
4. 运行MinIO
在 ~/minio 目录下操作
## 创建脚本
vim run-minio.sh
## -------脚本内容--------
#!/bin/bash
## 配置MinIO登录界面的账号密码
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=admin@minio123
## 启动脚本,挂载两个节点下的四个数据盘,设置配置文件的目录,登录窗口的端口,已经重定向日志到minio.logs文件
./app/minio server --config-dir /home/用户名/minio/config --console-address ":9001" \
http://node{1...2}/home/用户名/minio/data{1...4} \
/home/用户名/minio/logs/minio.logs 2>&1 &
## 修改脚本的权限
chmod a+x ./run-minio.sh
## 执行脚本
./run-minio.sh
## 查看日志
cat ./logs/minio.log
5. 测试MinIO集群
登录Dashboard页面,点击Support–》Performance
6. 配置访问权限用户
- 创建三个用户——下载用户、上传用户、管理员用户
- 每个用户分别创建一个ServiceAccount,用于API访问,记得要记录下Access_Key和Secret_Key(仅显示一次)
7. 配置基于Nginx的负载均衡
7.1 下载Nginx
## 安装前置环境
sudo apt install curl gnupg2 ca-certificates lsb-release debian-archive-keyring
## 导入签名
curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor \
| sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
## 检验签名
gpg --dry-run --quiet --import --import-options import-show /usr/share/keyrings/nginx-archive-keyring.gpg
## 添加Nginx源
echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \
http://nginx.org/packages/debian `lsb_release -cs` nginx" \
| sudo tee /etc/apt/sources.list.d/nginx.list
echo -e "Package: *\nPin: origin nginx.org\nPin: release o=nginx\nPin-Priority: 900\n" \
| sudo tee /etc/apt/preferences.d/99nginx
## 安装
sudo apt update
sudo apt install nginx
7.2 配置负载均衡
## 修改配置文件
# client_max_body_size 用来修改允许客户端上传文件的大小。
# 默认为1m,如果设置为0,表示上传文件大小不受限制。
# 可以在以下模块设置: http, server, location
client_max_body_size 10m;
## 在http模块下设置
upstream minio {
# 默认所有节点等权重负载均衡,可自行设置
server node1:9000 weight=1;
server node2:9000 weight=1;
}
upstream console {
ip_hash;
server node1:9001;
server node2:9001;
}
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
server {
listen 9001;
listen [::]:9001;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://console;
}
}
7.3 运行Nginx
## 运行nginx
nginx -c /etc/nginx/nginx.conf
通过浏览器访问 http://Nginx服务IP:9000,如果能够进入console控制台,说明配置成功。