CentOS 7 下 moosefs 集群安装

机器配置说明

主机名	IP	CPU	内存	磁盘	部署
node1	10.10.200.211	2核	4GB	50GB	mfsmaster mfscgiserv mfschunkserver
node2	10.10.200.212	2核	4GB	50GB	mfsmetalogger mfschunkserver
node3	10.10.200.213	2核	4GB	50GB	mfschunkserver

安装包准备

moosefs 安装包准备

[root@node1 soft]# wget https://github.com/moosefs/moosefs/archive/v3.0.114.zip

fuse 安装包准备

[root@node1 fuse-2.9.2-11]# wget http://www.rpmfind.net/linux/centos/7.8.2003/os/x86_64/Packages/fuse-2.9.2-11.el7.x86_64.rpm
[root@node1 fuse-2.9.2-11]# http://www.rpmfind.net/linux/centos/7.8.2003/os/x86_64/Packages/fuse-libs-2.9.2-11.el7.x86_64.rpm
[root@node1 fuse-2.9.2-11]# wget http://www.rpmfind.net/linux/centos/7.8.2003/os/x86_64/Packages/fuse-devel-2.9.2-11.el7.x86_64.rpm

fuse 安装

[root@node1 fuse-2.9.2-11]# rpm -ivh fuse-2.9.2-11.el7.x86_64.rpm 
[root@node1 fuse-2.9.2-11]# rpm -ivh fuse-libs-2.9.2-11.el7.x86_64.rpm
[root@node1 fuse-2.9.2-11]# rpm -ivh fuse-devel-2.9.2-11.el7.x86_64.rpm --force --nodeps

moosefs 安装

moosefs 用户创建

[root@node1 ~]# groupadd moosefs && useradd -s /sbin/nologin -g moosefs -M moosefs

moosefs 依赖包安装

[root@node1 ~]# yum install gcc make libpcap-devel zlib-devel fuse-devel fuse3-devel pkgconfig

解压安装包

[root@node1 soft]# unzip v3.0.114.zip

预编译

[root@node1 soft]# cd moosefs-3.0.114
[root@node1 moosefs-3.0.114]# ./configure --prefix=/usr/local/moosefs --with-default-user=moosefs --with-default-group=moosefs --enable-mfsmount

编译 && 安装

[root@node1 moosefs-3.0.114]# make -j `grep processor /proc/cpuinfo | wc -l`
[root@node1 moosefs-3.0.114]# make install -j `grep processor /proc/cpuinfo | wc -l`

配置文件配置

[root@node1 ~]# cd /usr/local/moosefs/etc/mfs
[root@node1 mfs]# for i in `ls | grep '.sample'`;do mv $i `echo $i | awk -F '.sample' '{print $1}'`; done

[root@node1 mfs]# cd /usr/local/moosefs/var/mfs
[root@node1 mfs]# mv metadata.mfs.empty metadata.mfs

启动 mfsmaster

[root@node1 mfs]# /usr/local/moosefs/sbin/mfsmaster start
open files limit has been set to: 16384
working directory: /usr/local/moosefs/var/mfs
lockfile created and locked
initializing mfsmaster modules ...
exports file has been loaded
topology file has been loaded
loading metadata ...
metadata file has been loaded
no charts data file - initializing empty charts
master <-> metaloggers module: listen on *:9419
master <-> chunkservers module: listen on *:9420
main master server module: listen on *:9421
mfsmaster daemon initialized properly

[root@node1 mfs]# /usr/local/moosefs/sbin/mfscgiserv
lockfile created and locked
starting simple cgi server (host: any , port: 9425 , rootpath: /usr/local/moosefs/share/mfscgi)

启动 mfsmetalogger

• 配置文件配置

[root@node2 mfs]# cat /usr/local/moosefs/etc/mfs/mfsmetalogger.cfg | grep MASTER_HOST
MASTER_HOST = node1

• 启动服务

[root@node2 mfs]# /usr/local/moosefs/sbin/mfsmetalogger start
open files limit has been set to: 4096
working directory: /usr/local/moosefs/var/mfs
lockfile created and locked
initializing mfsmetalogger modules ...
mfsmetalogger daemon initialized properly

启动 mfschunkserver

• 配置文件配置

[root@node1 moosefs]# cat /usr/local/moosefs/etc/mfs/mfschunkserver.cfg | grep MASTER_HOST
MASTER_HOST = node1

[root@node1 moosefs]# cat /usr/local/moosefs/etc/mfs/mfshdd.cfg | grep '/moosefs/data'
/usr/local/moosefs/data

• 对应目录创建 && 权限配置

[root@node1 moosefs]# mkdir /usr/local/moosefs/data
[root@node1 moosefs]# chown -R moosefs:moosefs /usr/local/moosefs

• 启动服务

[root@node1 moosefs]# /usr/local/moosefs/sbin/mfschunkserver start
open files limit has been set to: 16384
working directory: /usr/local/moosefs/var/mfs
lockfile created and locked
setting glibc malloc arena max to 4
setting glibc malloc arena test to 4
initializing mfschunkserver modules ...
hdd space manager: path to scan: /usr/local/moosefs/data/
hdd space manager: start background hdd scanning (searching for available chunks)
main server module: listen on *:9422
no charts data file - initializing empty charts
mfschunkserver daemon initialized properly

测试挂载

• 目录创建

[root@node1 ~]# mkdir /mfs_mount_dir

• 测试挂载

[root@node1 ~]# /usr/local/moosefs/bin/mfsmount /mfs_mount_dir -H node1
mfsmaster accepted connection with parameters: read-write,restricted_ip,admin ; root mapped to root:root

• 验证挂载

[root@node1 ~]# df -hT
Filesystem          Type      Size  Used Avail Use% Mounted on
/dev/mapper/cl-root xfs        34G  5.0G   29G  15% /
devtmpfs            devtmpfs  902M     0  902M   0% /dev
tmpfs               tmpfs     912M     0  912M   0% /dev/shm
tmpfs               tmpfs     912M  8.6M  904M   1% /run
tmpfs               tmpfs     912M     0  912M   0% /sys/fs/cgroup
/dev/sda2           xfs       197M  117M   81M  60% /boot
tmpfs               tmpfs     183M     0  183M   0% /run/user/0
node1:9421          fuse.mfs  102G   16G   86G  16% /mfs_mount_dir

• mfsmaster_ip:9425

• Can’t find masters (resolve given name) !!! 异常

更改 DNS master name 为 "node1"

参考

• moosefs-github
• MFS 搭建
• MFS 安装