nfs-ganesha(用户态NFS)编译、安装和使用

前面的文章介绍了基于社区或自己项目编译rpm包搭建Ceph分布式存储集群的方法。本篇介绍使用nfs-ganesha源码编译、安装，客户端使用用户态nfs协议挂载使用Ceph文件存储和对象存储的方法。

<一> 编译与安装nfs-ganesha

一、 获取源码

源码可直接从github上git clone，也可以使用自己库维护的代码：
github：

git clone git@github.com:nfs-ganesha/nfs-ganesha.git
git checkout V3.4

自己库：

git clone XXXXX

二、 安装依赖

安装所需的依赖:

yum install gcc git cmake autoconf libtool bison flex
yum install libgssglue-devel openssl-devel nfs-utils-lib-devel doxygen redhat-lsb gcc-c++
yum -y install libcephfs-devel.x86_64 librgw-devel.x86_64 libuuid-devel userspace-rcu-devel

新建build目录并进入：

mkdir build && cd build

三、 构建cmake编译环境

cmake -DUSE_FSAL_RGW=ON -DUSE_FSAL_CEPH=ON ../src/

四、 编译并安装

make && make install

五、 配置systemctl启动

将源码目录：nfs-ganesha/src/scripts/systemd下，nfs-ganesha-lock.service.el7,nfs-ganesha.service.el7拷贝到/usr/lib/systemd/system/下，并去掉后缀.el7：

cd nfs-ganesha/src/scripts/systemd
cp –a nfs-ganesha-lock.service.el7 nfs-ganesha.service.el7 /usr/lib/systemd/system/
cd /usr/lib/systemd/system/
mv nfs-ganesha-lock.service.el7 nfs-ganesha-lock.service
mv nfs-ganesha.service.el7 nfs-ganesha.service

并重置systemctl：

systemctl daemon-reload

<二> 使用nfs-ganesha

一、 配置/etc/ganesha/ganesha.conf

文件存储
如果是文件存储，建议配置如下：

###################################################
#
# Ganesha Config Example
#
# This is a commented example configuration file for Ganesha.  It is not
# complete, but only has some common configuration options.  See the man pages
# for complete documentation.
#
###################################################

## These are core parameters that affect Ganesha as a whole.
NFS_CORE_PARAM {
	## Allow NFSv3 to mount paths with the Pseudo path, the same as NFSv4,
	## instead of using the physical paths.
	mount_path_pseudo = true;

	## Configure the protocols that Ganesha will listen for.  This is a hard
	## limit, as this list determines which sockets are opened.  This list
	## can be restricted per export, but cannot be expanded.
	Protocols = 3,4;
}

## These are defaults for exports.  They can be overridden per-export.
EXPORT_DEFAULTS {
	## Access type for clients.  Default is None, so some access must be
	## given either here or in the export itself.
	Access_Type = RW;
}

## Configure settings for the object handle cache
#MDCACHE {
	## The point at which object cache entries will start being reused.
	#Entries_HWMark = 100000;
#}

## Configure an export for some file tree
EXPORT {
	## Export Id (mandatory, each EXPORT must have a unique Export_Id)
	Export_Id = 12345;

	## Exported path (mandatory)
	Path = /;

	## Pseudo Path (required for NFSv4 or if mount_path_pseudo = true)
	Pseudo = /mnt/cephfs;

	## Restrict the protocols that may use this export.  This cannot allow
	## access that is denied in NFS_CORE_PARAM.
	Protocols = 3,4;

	## Access type for clients.  Default is None, so some access must be
	## given. It can be here, in the EXPORT_DEFAULTS, or in a CLIENT block
	Access_Type = RW;

	## Whether to squash various users.
	Squash = No_Root_Squash;

	## Allowed security types for this export
	#Sectype = sys,krb5,krb5i,krb5p;

	## Exporting FSAL
	FSAL { 
		Name = CEPH;
	}
}

# Config block for FSAL_CEPH
CEPH {
 	# Path to a ceph.conf file for this ceph cluster.
 	Ceph_Conf = /etc/ceph/ceph.conf;

 	# User file-creation mask. These bits will be masked off from the unix
 	# permissions on newly-created inodes.
 	umask = 0;
}

## Configure logging.  Default is to log to Syslog.  Basic logging can also be
## configured from the command line
LOG {
	## Default log level for all components
	Default_Log_Level = WARN;

	## Configure per-component log levels.
	#Components {
		#FSAL = INFO;
		#NFS4 = EVENT;
	#}

	## Where to log
	Facility {
		name = FILE;
		destination = "/var/log/ganesha.log";
		enable = active;
	}
}

对象存储
如果是对象存储，建议配置如下：

###################################################
#
# Ganesha Config Example
#
# This is a commented example configuration file for Ganesha.  It is not
# complete, but only has some common configuration options.  See the man pages
# for complete documentation.
#
###################################################

## These are core parameters that affect Ganesha as a whole.
NFS_CORE_PARAM {
        ## Allow NFSv3 to mount paths with the Pseudo path, the same as NFSv4,
        ## instead of using the physical paths.
        mount_path_pseudo = true;

        ## Configure the protocols that Ganesha will listen for.  This is a hard
        ## limit, as this list determines which sockets are opened.  This list
        ## can be restricted per export, but cannot be expanded.
        Protocols = 3,4;
}

## These are defaults for exports.  They can be overridden per-export.
EXPORT_DEFAULTS {
        ## Access type for clients.  Default is None, so some access must be
        ## given either here or in the export itself.
        Access_Type = RW;
}

## Configure settings for the object handle cache
MDCACHE {
        ## The point at which object cache entries will start being reused.
        #Entries_HWMark = 100000;
        Dir_Chunk = 512;
}

## Configure an export for some file tree
EXPORT {
        ## Export Id (mandatory, each EXPORT must have a unique Export_Id)
        Export_Id = 1234;

        ## Exported path (mandatory)
        Path = /;

        ## Pseudo Path (required for NFSv4 or if mount_path_pseudo = true)
        Pseudo = /mnt/cephfs;

        ## Restrict the protocols that may use this export.  This cannot allow
        ## access that is denied in NFS_CORE_PARAM.
        Protocols = 3,4;

        ## Access type for clients.  Default is None, so some access must be
        ## given. It can be here, in the EXPORT_DEFAULTS, or in a CLIENT block
        Access_Type = RW;

        ## Whether to squash various users.
        Squash = No_Root_Squash;
        Transports = TCP;

        ## Allowed security types for this export
        #Sectype = sys,krb5,krb5i,krb5p;

        ## Exporting FSAL
        FSAL {
                Name = RGW;
                User_id = "User1";
                Access_Key_Id = "XXXXXXXXXX";
                Secret_Access_Key = "XXXXXXXXXXXXX";
        }
}

RGW {
        ceph_conf = /etc/ceph/ceph.conf;
        cluster = ceph;
        name = client.rgw.ceph2.rgw1;
        #init_args = "-d --debug-rgw=16";
}

## Configure logging.  Default is to log to Syslog.  Basic logging can also be
## configured from the command line
LOG {
        ## Default log level for all components
        Default_Log_Level = WARN;

        ## Configure per-component log levels.
        #Components {
                #FSAL = INFO;
                #NFS4 = EVENT;
        #}

        ## Where to log
        Facility {
                name = FILE;
                destination = "/var/log/ganesha.log";
                enable = active;
        }
}

二、 启动nfs-ganesha服务

systemctl start nfs-ganesha
systemctl enable nfs-ganesha

三、客户端挂载

客户端可通过showmount –e XX.XX.XX.XX查看启用ganesha服务的节点提供的挂载目录，然后通过：

mount –t nfs4 XX.XX.XX.XX:/mnt/XX /home/XXX

挂载到客户端某个目录下，便可开始文件/对象存储的访问。