glusterfs 集群文件系统配置笔记

nmcli connection add type bond con-name bond1 ifname bond1 ipv4.addresses 172.20.0.3/24 ipv4.method manual
nmcli connection add type bond-slave con-name bond1-ens4f0 ifname ens4f0 master bond1
nmcli connection add type bond-slave con-name bond1-ens5f0 ifname ens5f0 master bond1
nmcli conn up bond1
# 多网口做bond绑定，如果没有特殊要求，这步可不做

制作本地yum源
yum -y install createrepo
createrepo ./
createrepo --update ./

gfs相关
yum search glusterfs #查看gfs相关包

wget http://mirrors.aliyun.com/repo/Centos-7.repo #阿里源
https://blog.csdn.net/phn_csdn/article/details/75153913?utm_source=debugrun&utm_medium=referral #gfs链接

cd /etc/yum.repos.d/
mkdir bak
mv * bak/

填写yum配置文件，该repodata源是自己手动制作。
[c7-media]
name=CentOS-$releasever - Media
baseurl=file:///root/repodata
gpgcheck=0
enabled=1

yum install -y centos-release-gluster
yum install -y glusterfs glusterfs-server glusterfs-fuse glusterfs-rdma

systemctl restart glusterd.service
systemctl enable glusterd.service
systemctl restart glusterfsd
systemctl enable glusterfsd
systemctl status glusterd && systemctl status glusterfsd

启动
systemctl start glusterd.service
systemctl start glusterfsd.service

停止
systemctl stop glusterd.service&&systemctl stop glusterfsd.service

重启
systemctl restart glusterfsd
systemctl restart glusterd.service

添加hosts文件解析

gluster peer probe backup-14-202 ##加入池
gluster peer detach backup-14-202 ##删除池
gluster peer delete backup-14-202 ##删除整个池

gluster volume create data1 backup-14-201:/data backup-14-202:/data force ##创建卷

gluster volume create nvme-volume s1:/nvme_gfs s2:/nvme_gfs s3:/nvme_gfs force

gluster volume start backupdata1 ##启动卷
gluster volume start nvme-volume ##启动卷

gluster volume stop backupdata1 ##停止卷
gluster volume delete data1 ##删除卷

mount -t glusterfs backup-14-201:data1 /vdpp1 ##挂载

mount -t glusterfs s1:nvme-volume /nvme-volume

客户端添加卷（扩容 先做成文件系统，创建目录，在加入池）
添加brick
gluster volume add-brick data HN-IDC-NDP-node09:/data force

gluster volume add-brick data HN-IDC-NDP-node05:/data force

gluster volume add-brick data HN-IDC-NDP-node17:/hn-vol force

umount -l /vdpp1
mount -t glusterfs HN-IDC-NDP-node01:/data /vdpp1

删除brick
gluster volume remove-brick data HN-IDC-NDP-node09:/data force
gluster volume remove-brick data HN-IDC-NDP-node15:/hn-vol force

gluster volume remove-brick data HN-IDC-NDP-node02:/vdpp1 force
gluster volume add-brick data HN-IDC-NDP-node02:/vdpp1 force

gluster volume remove-brick nvme-volume hostname:/nvme_gfs

挂载卷(后面的data为gfs的volume的名称，而不是文件夹名称)
mount -t glusterfs backup-14-202:data /vdpp1

gluster volume info #查看所有gfs （卷状态）
gluster volume status
glusterfs volume status #查看所有gfs状态
glusterfs -V #查看版本
gluster peer status #查看集群状态
gluster volume list #查看所有的volume
gluster volume rebalance data start

如果需要清除glusterfs配置，将工作目录删除后重启服务即可
rm -rf /var/lib/glusterd/

umount -l /vdpp1
mount -t glusterfs HN-IDC-NDP-node03:/data /vdpp1
mount -t glusterfs HN-IDC-NDP-node04:/data /vdpp1
mount -t glusterfs HN-IDC-NDP-node05:/data /vdpp1
mount -t glusterfs HN-IDC-NDP-node06:/data /vdpp1

卸载服务
yum remove -y glusterfs-server

yum install -y glusterfs-server

mkdir -p /vdpp1

mount -t glusterfs HN-IDC-NDP-node03:/data /vdpp1

gluster负载均衡

gluster volume rebalance [fix-layout|migrate-data] {start|stop|status}

gluster volume rebalance data fix-layout start

gluster volume rebalance data start

gluster volume rebalance data status 查看rebalance执行情况