RH236配置IP故障转移--配置NFS Ganesha

IT民工金鱼哥

已于 2022-03-18 22:46:11 修改

阅读量2.4k

点赞数 1

分类专栏： RHCA 回忆录文章标签：运维 linux 服务器 GlusterFS RHCA-236

于 2022-03-10 23:32:50 首次发布

本文链接：https://blog.csdn.net/qq_41765918/article/details/123414215

版权

RHCA 回忆录专栏收录该内容

158 篇文章 80 订阅

订阅专栏

RH236配置IP故障转移–配置NFS Ganesha

本章节学习配置NFS Ganesha解决高可用问题。

RHCA专栏地址：https://blog.csdn.net/qq_41765918/category_11532281.html

文章目录

NFS-Ganesha特征

NFS-ganesha是NFS的用户模式文件服务器。它支持NFSv3、NFSv4、NFSv4.1和pNFS(作为技术预览)。使用Corosync和Pacemaker提供的集群基础设施，NFS-Ganesha可以实现高可用性。

Red Hat Gluster存储的内置NFS服务器只支持NFSv3。如果需要NFSv4、Kerberos身份验证或加密，或者IP故障转移，管理员应该使用NFS-Ganesha。

重要：NFS-Ganesha不能与内置的NFSv3服务器同时运行。应该在所有将运行NFS- ganesha的节点上禁用NFS。

可参考官网进行导出的学习：

https://www.gluster.org/glusterfs-and-nfs-ganesha-integration/

https://docs.gluster.org/en/latest/Administrator-Guide/NFS-Ganesha-GlusterFS-Integration/

课本练习（以练习来进行了解）

[root@workstation ~]# lab ganesha setup

1. 在servera和serverb上安装所需的包。

# systemctl stop glusterd
# killall glusterfs
# killall glusterfsd
# yum -y install glusterfs-ganesha

2. 更新servera和serverb上的防火墙。

以允许pacemaker/corosync、NFS、portmapper以及mountd。

# firewall-cmd --permanent --add-service=high-availability --add-service=nfs --add-service=rpc-bind --add-service=mountd 
success
# firewall-cmd --reload
success

3. 按要求修改配置文件。

[root@servera ~]# cp /etc/ganesha/ganesha-ha.conf.sample /etc/ganesha/ganesha-ha.conf
[root@servera ~]# vim /etc/ganesha/ganesha-ha.conf
[root@servera ~]# egrep -v ^# /etc/ganesha/ganesha-ha.conf
HA_NAME="gls-ganesha"
HA_VOL_SERVER="servera"
HA_CLUSTER_NODES="servera.lab.example.com,serverb.lab.example.com"
VIP_servera_lab_example_com="172.25.250.16"
VIP_serverb_lab_example_com="172.25.250.17"

[root@servera ~]# scp /etc/ganesha/ganesha-ha.conf serverb:/etc/ganesha/

4. 按要求设置集群。

通过启用正确的服务、设置集群用户密码和彼此进行身份验证，准备好servera和serverb作为集群成员。

# systemctl enable pacemaker.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/pacemaker.service to /usr/lib/systemd/system/pacemaker.service.
# systemctl enable pcsd.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/pcsd.service to /usr/lib/systemd/system/pcsd.service.
# systemctl start pcsd.service

# echo redhat | passwd --stdin hacluster 
Changing password for user hacluster.
passwd: all authentication tokens updated successfully.

从servera系统，验证所有节点之间的pc通信。
[root@servera ~]# pcs cluster auth -u hacluster -p redhat servera.lab.example.com serverb.lab.example.com
servera.lab.example.com: Authorized
serverb.lab.example.com: Authorized

5. 创建一个SSH密钥对，以支持nfs-ganesha的免密登录。

[root@servera ~]# ssh-keygen -f /var/lib/glusterd/nfs/secret.pem -t rsa -N ''
Generating public/private rsa key pair.
Your identification has been saved in /var/lib/glusterd/nfs/secret.pem.
Your public key has been saved in /var/lib/glusterd/nfs/secret.pem.pub.
The key fingerprint is:
a4:bd:d2:9d:b8:13:1a:4f:0a:21:a4:2c:b8:85:d0:d1 root@servera.lab.example.com
The key's randomart image is:
+--[ RSA 2048]----+
| ..o             |
|. o E            |
|++      .        |
|=.o .  +         |
|.o . .. S        |
|.   . ..o+ .     |
|     ..*+.o      |
|      o.o.       |
|        ..       |
+-----------------+
[root@servera ~]# scp /var/lib/glusterd/nfs/secret.pem* serverb:/var/lib/glusterd/nfs/
root@serverb's password: 
secret.pem                                             100% 1675     1.6KB/s   00:00    
secret.pem.pub                                         100%  410     0.4KB/s   00:00
[root@servera ~]# ssh-copy-id -i /var/lib/glusterd/nfs/secret.pem.pub root@servera
[root@servera ~]# ssh-copy-id -i /var/lib/glusterd/nfs/secret.pem.pub root@serverb

6. 在两个节点上启动glusterd，并为glusterd启用共享存储。

在两个节点上启动glusterd
# systemctl start glusterd.service

[root@servera ~]# gluster volume set all cluster.enable-shared-storage enable

7. 在两个节点上配置nfs-ganesha，以便为挂载进程使用默认端口(20048/tcp/20048/UDP)。

# tail -f -n 10 /etc/ganesha/ganesha.conf 
NFS_Core_Param {
        #Use supplied name other tha IP In NSM operations
        NSM_Use_Caller_Name = true;
        #Copy lock states into "/var/lib/nfs/ganesha" dir
        Clustered = false;
        #Use a non-privileged port for RQuota
        Rquota_Port = 4501;
        MNT_Port=20048;
}

8. 启用集群服务，配置对应的挂载。

[root@servera ~]# gluster nfs-ganesha enable
Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the trusted pool. Do you still want to continue?
 (y/n) y
This will take a few minutes to complete. Please wait ..
nfs-ganesha : success 
You have mail in /var/spool/mail/root

[root@servera ~]# gluster volume set custdata ganesha.enable on
volume set: success

9. 设置永久挂载。

[root@servera ~]# showmount -e
Export list for servera.lab.example.com:
/custdata (everyone)
[root@servera ~]# showmount -e 172.25.250.16
Export list for 172.25.250.16:
/custdata (everyone)
[root@servera ~]# showmount -e 172.25.250.17
Export list for 172.25.250.17:
/custdata (everyone)

[root@workstation ~]# mkdir /mnt/nfs
[root@workstation ~]# echo "172.25.250.16:/custdata /mnt/nfs nfs rw,vers=4 0 0" >> /etc/fstab 
[root@workstation ~]# mount -a
[root@workstation ~]# df -Th
Filesystem              Type      Size  Used Avail Use% Mounted on
/dev/vda1               xfs        10G  3.0G  7.1G  30% /
devtmpfs                devtmpfs  902M     0  902M   0% /dev
tmpfs                   tmpfs     920M   84K  920M   1% /dev/shm
tmpfs                   tmpfs     920M   17M  904M   2% /run
tmpfs                   tmpfs     920M     0  920M   0% /sys/fs/cgroup
tmpfs                   tmpfs     184M   16K  184M   1% /run/user/42
tmpfs                   tmpfs     184M     0  184M   0% /run/user/0
172.25.250.16:/custdata nfs4      2.0G   33M  2.0G   2% /mnt/nfs

10. 脚本评分。

[root@workstation ~]# lab ganesha grade

章节实验

[root@workstation ~]# lab ipfailover setup

1. 在serverc和serverd上安装所需的软件，并在这些机器上的防火墙上打开此设置所需的任何端口。

# yum -y install samba ctdb
# firewall-cmd --permanent --add-service=samba
# firewall-cmd --permanent --add-port=4379/tcp
# firewall-cmd --reload

2. 停止ctdbmeta卷，然后在serverc和serverd上设置相关的启动和停止触发程序，以便为CTDB使用ctdbmeta卷。还为两个节点上的Samba启用集群。

[root@serverc ~]# gluster volume stop ctdbmeta 
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
volume stop: ctdbmeta: success

# vim /var/lib/glusterd/hooks/1/start/post/S29CTDBsetup.sh
# $META is the volume that will be used by CTDB as a shared filesystem.
# It is not desirable to use this volume for storing 'data' as well.
# META is set to 'all' (viz. a keyword and hence not a legal volume name)
# to prevent the script from running for volumes it was not intended.
# User needs to set META to the volume that serves CTDB lockfile.
META=ctdbmeta

# vim /var/lib/glusterd/hooks/1/stop/pre/S29CTDB-teardown.sh
# $META is the volume that will be used by CTDB as a shared filesystem.
# It is not desirable to use this volume for storing 'data' as well.
# META is set to 'all' (viz. a keyword and hence not a legal volume name)
# to prevent the script from running for volumes it was not intended.
# User needs to set META to the volume that serves CTDB lockfile.
META=ctdbmeta

添加clustering = yes 配置
# grep clustering -C 2 /etc/samba/smb.conf 

[global]
clustering=yes
#------------------------ AIO Settings ------------------------
#

3. 启动ctdbmeta卷，然后配置 CTDB使用您的serverc和serverd系统进行iP故障转移，使用172.25.250.18/24作为浮动iP地址。

[root@serverc ~]# gluster volume start ctdbmeta 
volume start: ctdbmeta: success

# vim /etc/ctdb/nodes
172.25.250.12
172.25.250.13

# vim /etc/ctdb/public_addresses	
# cat /etc/ctdb/public_addresses 
172.25.250.18/24 eth0

# systemctl enable ctdb
# systemctl start ctdb

4. 确保使用Samba导出custdata卷。记住为smbuser用户设置一个redhat Samba密码。

将smbuser的samba密码设置为redhat。因为ctdb将此更改传播到所有节点，所以此步骤只需要在单个主机上执行。

# smbpasswd -a smbuser
New SMB password: redhat
Retype new SMB password: redhat
Added user smbuser.

# gluster volume set labdata stat-prefetch off
# gluster volume set labdata server.allow-insecure on
# gluster volume set labdata storage.batch-fsync-delay-usec 0

serverc和serverd都需要更改：
# vim /etc/glusterfs/glusterd.vol
    option rpc-auth-allow-insecure on
# systemctl restart glusterd

[root@serverc ~]# gluster volume stop labdata
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
volume stop: custdata: success
[root@serverc ~]# gluster volume start labdata
volume start: custdata: success

5. 在您的workstation系统上，使用Samba通过浮动IP地址在/mnt/custdata上持久地挂载custdata卷。

[root@workstation ~]# mkdir /mnt/labdata
[root@workstation ~]# echo "//172.25.250.18/gluster-labdata /mnt/labdata cifs user=smbuser,pass=redhat 0 0" >> /etc/fstab
[root@workstation ~]# mount -a
[root@workstation ~]# df -Th
Filesystem                      Type      Size  Used Avail Use% Mounted on
/dev/vda1                       xfs        10G  3.1G  7.0G  31% /
devtmpfs                        devtmpfs  902M     0  902M   0% /dev
tmpfs                           tmpfs     920M   84K  920M   1% /dev/shm
tmpfs                           tmpfs     920M   17M  904M   2% /run
tmpfs                           tmpfs     920M     0  920M   0% /sys/fs/cgroup
tmpfs                           tmpfs     184M   16K  184M   1% /run/user/42
tmpfs                           tmpfs     184M     0  184M   0% /run/user/0
//172.25.250.18/gluster-labdata cifs      2.0G   33M  2.0G   2% /mnt/labdata