glusterfs创建分布式复制卷（Distributed + Replication）

作者：吴业亮

博客：http://blog.csdn.net/wylfengyujiancheng

一、环境信息

node1  192.168.8.11 两块磁盘
node2  192.168.8.12 两块磁盘
node3  192.168.8.13 两块磁盘
node4  192.168.8.14 两块磁盘
node5  192.168.8.15 两块磁盘
node6  192.168.8.16 两块磁盘

二、通用配置（各个节点）

1、修改主机名

hostnamectl  set-hostname node1

2、修改hosts文件

# cat <<"EOF">/etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.8.11 node1
192.168.8.12 node2
192.168.8.13 node3
192.168.8.14 node4
192.168.8.15 node5
192.168.8.16 node6
EOF

3、配置源

# cat <<END >/etc/yum.repos.d/repo.repo
[repo]
name=repo
baseurl = https://mirrors.tuna.tsinghua.edu.cn/centos/7/storage/x86_64/gluster-6/
enabled=1
gpgcheck=0
END

4、安装NTP（各个节点）

#  yum -y install ntp

修改配置文件/etc/ntp.conf

server <NTP-server>

**注意： **
NTP-server修改为自己的NTP服务器，如果局域网内无NTP服务器，此处可以用默认配置，采用centos官方ntp服务器

5、启动ntpd服务并设置开机启动

# systemctl start ntpd
# systemctl enable ntpd

查看ntp状态

# ntpq -p

6、配置防火墙

# firewall-cmd --add-service=ntp --permanent 
success
# firewall-cmd --reload 
success

7、安装软件包

# yum  -y install glusterfs-server

8、启动服务并设置开机启动

# systemctl start glusterd

# systemctl enable glusterd

9、查看版本

# gluster --version 
glusterfs 6.4
Repository revision: git://git.gluster.org/glusterfs.git
Copyright (c) 2006-2016 Red Hat, Inc. <https://www.gluster.org/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.

10、配置防火墙

# firewall-cmd --add-service=glusterfs --permanent 
success
# firewall-cmd --reload

三、配置存储

1、创建lvm卷

# vgcreate data  /dev/vdb
# lvcreate  --name  lv  -l +100%FREE data

2、格式化lvm逻辑卷

# mkfs.xfs /dev/data/lv  -f

3、创建目录

# mkdir -p /glusterfs/distributed

4、修改/etc/fstab文件

/dev/mapper/data-lv  /glusterfs/data  xfs  defaults        0 0

5、挂载目录

mount  -a

6、检查

# df -h
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/centos-root   77G  1.2G   76G   2% /
devtmpfs                 3.9G     0  3.9G   0% /dev
tmpfs                    3.9G     0  3.9G   0% /dev/shm
tmpfs                    3.9G  8.5M  3.9G   1% /run
tmpfs                    3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/vda1               1014M  142M  873M  14% /boot
/dev/mapper/data-lv      200G  181G   20G  91% /glusterfs/data
tmpfs                    783M     0  783M   0% /run/user/0

四、创建和配置集群（在node1上执行）

1、配置集群

# gluster peer probe node2
# gluster peer probe node3
# gluster peer probe node4
# gluster peer probe node5
# gluster peer probe node6

2、查看状态

# gluster peer status

3、创建卷（Arbiter以避免脑裂）

# gluster volume create vol_dist-replica replica 3 arbiter 1 transport tcp \
node01:/glusterfs/data \
node02:/glusterfs/data \
node03:/glusterfs/data\
node04:/glusterfs/data \
node05:/glusterfs/data \
node06:/glusterfs/data 
volume create: vol_dist-replica: success: please start the volume to access data

4、启动卷

# gluster volume start vol_dist-replica 
volume start: vol_dist-replica : success

5、查看详细信息

# gluster volume info 

Volume Name: vol_dist-replica
Type: Replicate
Volume ID: 0649d480-ad8f-4170-b45f-fdb161bfa203
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x (2 + 1) = 6
Transport-type: tcp
Bricks:
Brick1: node1:/glusterfs/data
Brick2: node2:/glusterfs/data
Brick3: node3:/glusterfs/data
Brick4: node4:/glusterfs/data
Brick5: node5:/glusterfs/data
Brick6: node6:/glusterfs/data
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off