Glusterfs 分布式存储系统配置

在服务器集群中,为了同步更新APP内容,利用Glusterfs分布式存储功达到相同的APP只更新一次,其他机器同步更新

试验环境:Vmware Fusion3.1.1 CentOS 5.5

1.安装
#yum -y install glusterfs-cluster glusterfs-server

2.设置两台机器的别名分别为svr1,svr2

3.两台机器的服务端配置文件和客户端文件如下:
服务端配置文件glusterfsd.vol
# cat /etc/glusterfs/glusterfsd.vol
### file: server-volume.vol.sample

#####################################
### GlusterFS Server Volume File ##
#####################################

#### CONFIG FILE RULES:
### "#" is comment character.
### - Config file is case sensitive
### - Options within a volume block can be in any order.
### - Spaces or tabs are used as delimitter within a line.
### - Multiple values to options will be : delimitted.
### - Each option should end within a line.
### - Missing or commented fields will assume default values.
### - Blank/commented lines are allowed.
### - Sub-volumes should already be defined above before referring.

### Export volume "brick" with the contents of "/home/export" directory.
volume posix
type storage/posix # POSIX FS translator
option directory /opt/export # Export this directory
end-volume

volume locks
type features/locks
# option mandatory-locks on
subvolumes posix
end-volume

volume brick
type performance/io-threads
option thread-count 8
subvolumes locks
end-volume


### Add network serving capability to above brick.
volume server
type protocol/server
option transport-type tcp
# option transport-type unix
# option transport-type ib-sdp
# option transport.socket.bind-address 192.168.1.10 # Default is to listen on all interfaces
# option transport.socket.listen-port 6996 # Default is 6996

# option transport-type ib-verbs
# option transport.ib-verbs.bind-address 192.168.1.10 # Default is to listen on all interfaces
# option transport.ib-verbs.listen-port 6996 # Default is 6996
# option transport.ib-verbs.work-request-send-size 131072
# option transport.ib-verbs.work-request-send-count 64
# option transport.ib-verbs.work-request-recv-size 131072
# option transport.ib-verbs.work-request-recv-count 64

# option client-volume-filename /etc/glusterfs/glusterfs-client.vol
subvolumes brick
# NOTE: Access to any volume through protocol/server is denied by
# default. You need to explicitly grant access through # "auth"
# option.
option auth.addr.brick.allow 127.0.0.1,192.168.* # Allow access to "brick" volume
end-volume

客户端配置文件glusterfs.vol
# cat /etc/glusterfs/glusterfs.vol
### file: client-volume.vol.sample

#####################################
### GlusterFS Client Volume File ##
#####################################

#### CONFIG FILE RULES:
### "#" is comment character.
### - Config file is case sensitive
### - Options within a volume block can be in any order.
### - Spaces or tabs are used as delimitter within a line.
### - Each option should end within a line.
### - Missing or commented fields will assume default values.
### - Blank/commented lines are allowed.
### - Sub-volumes should already be defined above before referring.

### Add client feature and attach to remote subvolume
volume svr1
type protocol/client
option transport-type tcp
option remote-host svr1
option remote-subvolume brick #name of the remote volume
end-volume

volume svr2
type protocol/client
option transport-type tcp
option remote-host svr2
option remote-subvolume brick
end-volume

volume replicate1
type cluster/replicate
subvolumes svr1
end-volume

volume replicate2
type cluster/replicate
subvolumes svr2
end-volume

volume distribute
type cluster/distribute
subvolumes replicate1 replicate2
# subvolumes replicate1
end-volume

volume writebehind
type performance/write-behind
option cache-size 1MB
subvolumes distribute
end-volume

volume cache
type performance/io-cache
option cache-size 64MB
subvolumes writebehind
end-volume

4.接下来启动服务端
# glusterfsd -f /etc/glusterfs/glusterfsd.vol
执行ps fax|grep gluseterfs查看是否有启动

使用以上命令操作svr2

5.启动客户端
启动客户端之前先挂起fuse
# modprobe fuse

启动客户端
# glusterfs -l /var/log/glusterfs/opt-glusterfs.log -f /etc/glusterfs/glusterfs.vol /opt/clusterfs

使用df -h 查看挂载是否成功
# df -h
文件系统 容量 已用 可用 已用% 挂载点
/dev/hda2 5.7G 4.6G 858M 85% /
/dev/hda3 2.9G 69M 2.7G 3% /home
/dev/hda1 99M 12M 82M 13% /boot
tmpfs 502M 0 502M 0% /dev/shm
glusterfs#/etc/glusterfs/glusterfs.vol
12G 9.0G 1.9G 84% /opt/clusterfs

如果没有如上显示，则说明挂载没有成功，可以查看/var/log/glusterfs/opt-glusterfs.log下建立的日志进行排错

最后使用以上命令进行svr2的操作
注:假如重启到glusterfs挂连接不上,可以使用命令取消挂起
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_webserver2-lv_root
50G 2.0G 45G 5% /
tmpfs 1.5G 0 1.5G 0% /dev/shm
/dev/sda1 485M 37M 423M 8% /boot
/dev/mapper/vg_webserver2-lv_home
46G 180M 44G 1% /home
df: `/opt/clusterfs': Transport endpoint is not connected
#umount /opt/clusterfs
root@svr2 ~]#umount /opt/clusterfs
================================
最后的效果是,不管在哪台机器上的/opt/glusterfs下新建更新文件,都会同步更新到另一台机器上,当你相同的APP在N台机器上时,这是多么轻松的事啊