1 安装
1.1 配置要求
1 硬件要求
GlusterFS支持大多数的符合商业标准的x86-64bit硬件平台。存储可以是直接附加的存储，RAID盘，以SATA/SAS/FC盘做后端的FC/Infiniband/iSCSI SAN盘。
2 网路要求
GlusterFS支持千兆网，百兆网和Infiniband（光纤）。
3 操作系统要求
GlusterFS可以支持很多POSIX兼容的系统，如GNU/Linux, FreeBSD, OpenSolaris, Solaris, 和Mac OS X。Gluster建议采用Ext3, Ext4或者ZFS作为子系统磁盘的格式。Gluster对GNU/Linux, Mac OS X, FreeBSD和OpenSolaris有内嵌的支持。
1.2 Ubuntu下安装
Ubuntu 14: install software-properties-common:
sudo apt-get install software-properties-common
Then add the community GlusterFS PPA:
sudo add-apt-repository ppa:gluster/glusterfs-3.8
sudo apt-get update
Finally, install the packages:
sudo apt-get install glusterfs-server
1.3 gluster安装包下载
https://download.gluster.org/pub/gluster/glusterfs/
1.4 源码安装
1.4.1 安装
./configure
make
make install1.4.2 问题
# ./configure
configure: error: no acceptable C compiler found in $PATH解决：apt-get install build-essential
configure: error: Flex or lex required to build glusterfs.configure: error: GNU Bison required to build glusterfs.# ./configure
configure: error: OpenSSL crypto library is required to build glusterfs解决：apt-get install libssl-dev
error: Support for POSIX ACLs is required解决：apt-get install libacl1 libacl1-dev
checking for SQLITE... configure: error: pass --disable-tiering to build without sqliteconfigure: error: liburcu-bp not found# /usr/local/sbin/gluster
/usr/local/sbin/gluster: error while loading shared libraries: libglusterfs.so.0: cannot open shared object file: No such file or directory1.4.3 卸载
执行make命令：
make uninstall-recursive
make uninstall
make mostlyclean-recursive
make clean-recursive
make distclean-recursive# for i in `find / -name "gluster*"`; do rm -rf $i; done2 安装
2.1 部署拓扑
3台存储服务器: 192.168.4.132、192.168.4.133、192.168.4.134root@block2:/glusterfs# cat /etc/hosts
127.0.0.1 localhost
192.168.4.130 controller
192.168.4.131 network
192.168.4.132 compute
192.168.4.133 block1
192.168.4.134 block22.2 安装
在所有存储服务器上执行安装。
2.2.1 如果出现错误：
W: Failed to fetch http://ppa.launchpad.net/gluster/glusterfs-3.8/ubuntu/dists/trusty/main/binary-amd64/Packages Hash Sum mismatch
W: Failed to fetch http://ppa.launchpad.net/gluster/glusterfs-3.8/ubuntu/dists/trusty/main/binary-i386/Packages Hash Sum mismatch
W: Failed to fetch http://ppa.launchpad.net/gluster/glusterfs-3.8/ubuntu/dists/trusty/main/i18n/Translation-en Hash Sum mismatch
E: Some index files failed to download. They have been ignored, or old ones used instead.root@block1:~# ls /var/lib/apt/lists/ppa*
/var/lib/apt/lists/ppa.launchpad.net_gluster_glusterfs-3.8_ubuntu_dists_trusty_InRelease
/var/lib/apt/lists/ppa.launchpad.net_gluster_glusterfs-3.8_ubuntu_dists_trusty_main_binary-amd64_Packages
/var/lib/apt/lists/ppa.launchpad.net_gluster_glusterfs-3.8_ubuntu_dists_trusty_main_binary-i386_Packages
/var/lib/apt/lists/ppa.launchpad.net_gluster_glusterfs-3.8_ubuntu_dists_trusty_main_i18n_Translation-en 解决方法：
将/var/lib/apt/lists/partial/下的所有文件删除，但是这可以解决大部分问题，个别源不能解决，
注：经过长时间的实践，这个解决方法不是万能的。
2.2.2 安装glusterfs-server时，会安装glusterfs-client
root@block2:/glusterfs# apt-get install glusterfs-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
attr glusterfs-client glusterfs-common liblvm2app2.2 liburcu1
The following NEW packages will be installed:
attr glusterfs-client glusterfs-common glusterfs-server liblvm2app2.2
liburcu1
0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.
Need to get 3,389 kB of archives.
After this operation, 16.0 MB of additional disk space will be used.
Do you want to continue? [Y/n] 安装成功后，gluster会运行。查看版本：
root@block2:/glusterfs# glusterfs --version
glusterfs 3.8.10root@block2:/glusterfs# ps -ef | grep gluster
root 17920 1 0 17:58 ? 00:00:00 /usr/sbin/glusterd -p /var/run/glusterd.pid
2.3 查看运行端口
root@compute: # netstat -pan | grep glus
tcp 0 0 0.0.0.0:24007 0.0.0.0:* LISTEN 5323/glusterd
tcp 0 0 192.168.4.132:24007 192.168.4.133:1023 ESTABLISHED 5323/glusterd
tcp 0 0 192.168.4.132:1023 192.168.4.134:24007 ESTABLISHED 5323/glusterd
tcp 0 0 192.168.4.132:24007 192.168.4.134:1020 ESTABLISHED 5323/glusterd
tcp 0 0 192.168.4.132:1021 192.168.4.133:24007 ESTABLISHED 5323/glusterd 2.4 服务启停
使用命令：service glusterfs-server start/stop
root@compute:~# ps -ef | grep gluster
root 10929 1 0 Mar16 ? 00:00:00 /usr/sbin/glusterd -p /var/run/glusterd.pid
root 12215 1650 0 10:22 pts/0 00:00:00 grep --color=auto gluster
root@compute:~# ls /etc/init.d/glusterfs*
/etc/init.d/glusterfs-server
root@compute:~# service glusterfs-server stop
glusterfs-server stop/waiting
root@compute:~# ps -ef | grep gluster
root 12225 1650 0 10:23 pts/0 00:00:00 grep --color=auto gluster
root@compute:~# service glusterfs-server start
glusterfs-server start/running, process 12233
root@compute:~# ps -ef | grep gluster
root 12233 1 0 10:23 ? 00:00:00 /usr/sbin/glusterd -p /var/run/glusterd.pid
root 12467 1650 0 10:23 pts/0 00:00:00 grep --color=auto gluster3 集群关联
3.1创建GlusterFS集群
$ gluster peer probe SERVER SERVER表示存储服务器的hostname。在服务器192.168.4.132上执行：
root@compute:~# gluster peer probe
Usage: peer probe <HOSTNAME>
root@compute:~# gluster peer probe block1
peer probe: success
root@compute:~# gluster peer probe block2
peer probe: success
注意：在构建
GlusterFS
集群时，需要在其中任意一台存储服务器上依次将其他存储服务器添加到集群中。
3.2 查看集群信息
指令：gluster peer status
root@compute:/etc/glusterfs# gluster peer status
Number of Peers: 2
Hostname: block1
Port: 24007
Uuid: ac468b50-3d66-4290-aa8d-cb01279e9bf2
State: Peer in Cluster (Connected)
Hostname: block2
Port: 24007
Uuid: 06288da0-1b18-42fb-a7b8-cd9ca24d5f26
State: Peer in Cluster (Connected)
root@block1:/etc/glusterfs# gluster peer status
Number of Peers: 2
Hostname: 192.168.4.132
Port: 24007
Uuid: 9fdd15f2-c7e1-44e1-9315-d62e1dcfb44d
State: Peer in Cluster (Connected)
Hostname: block2
Uuid: 06288da0-1b18-42fb-a7b8-cd9ca24d5f26
State: Peer in Cluster (Connected)
root@block1:/etc/glusterfs# gluster volume info
No volumes presentConnected,说明集群节点添加成功，但其中compute是以192.168.4.132这个ip加入集群的，如果需要换成hostname，登陆集群中的集群中的另一个节点，执行：
root@block1:~# gluster peer detach 192.168.4.132
peer detach: success
root@block1:~#
root@block1:~# gluster peer status
Number of Peers: 1
Hostname: block2
Uuid: 06288da0-1b18-42fb-a7b8-cd9ca24d5f26
State: Peer in Cluster (Connected)
root@block1:~#
root@block1:~# gluster peer probe compute
peer probe: success.
root@block1:~#
root@block1:~# gluster peer status
Number of Peers: 2
Hostname: block2
Uuid: 06288da0-1b18-42fb-a7b8-cd9ca24d5f26
State: Peer in Cluster (Connected)
Hostname: compute
Uuid: 9fdd15f2-c7e1-44e1-9315-d62e1dcfb44d
State: Peer in Cluster (Connected)4 Volume卷操作
4.1．创建volume
a. 单磁盘，调试环境推荐
sudo gluster volume create vol_name 172.17.0.2:/d/disk0
b. 多磁盘，无raid，试验、测试环境推荐。
sudo gluster volume create vol_name 172.17.0.2:/d/disk0 172.17.0.3:/d/disk0 172.17.0.4:/d/disk0 172.17.0.5:/d/disk0
c. 多磁盘，有raid1。线上高并发环境推荐。
Usage: volume create <NEW-VOLNAME> [stripe <COUNT>] [replica <COUNT> [arbiter <COUNT>]] [disperse [<COUNT>]] [disperse-data <COUNT>] [redundancy <COUNT>] [transport <tcp|rdma|tcp,rdma>] <NEW-BRICK>?<vg_name>... [force]实验命令是在根目录下创建目录/glusterfs，要在所有的3台服务器上都创建这个目录，否则会出错。
root@compute:~# gluster volume create zhai-volume replica 2 192.168.4.132:/glusterfs/brick1 block1:/glusterfs/brick1 block2:/glusterfs/brick1
number of bricks is not a multiple of replica count
Usage: volume create <NEW-VOLNAME> [stripe <COUNT>] [replica <COUNT> [arbiter <COUNT>]] [disperse [<COUNT>]] [disperse-data <COUNT>] [redundancy <COUNT>] [transport <tcp|rdma|tcp,rdma>] <NEW-BRICK>?<vg_name>... [force]
root@compute:~#
root@compute:~# gluster volume create zhai-volume replica 2 compute:/glusterfs/brick1 compute:/glusterfs/brick2 block1:/glusterfs/brick1 block1:/glusterfs/brick2 block2:/glusterfs/brick1 block2:/glusterfs/brick2
volume create: zhai-volume: failed: The brick compute:/glusterfs/brick1 is being created in the root partition. It is recommended that you don't use the system's root partition for storage backend. Or use 'force' at the end of the command if you want to override this behavior.
root@compute:~#
root@compute:~# ls /glusterfs/
root@compute:~#
root@compute:~# gluster volume create zhai-volume replica 2 compute:/glusterfs/brick1 compute:/glusterfs/brick2 block1:/glusterfs/brick1 block1:/glusterfs/brick2 block2:/glusterfs/brick1 block2:/glusterfs/brick2 force
volume create: zhai-volume: failed: Staging failed on block2. Error: Failed to create brick directory for brick block2:/glusterfs/brick1. Reason : No such file or directory
Staging failed on block1. Error: Failed to create brick directory for brick block1:/glusterfs/brick1. Reason : No such file or directory
root@compute:~#
root@compute:~# gluster volume create zhai-volume replica 2 compute:/glusterfs/brick1 compute:/glusterfs/brick2 block1:/glusterfs/brick1 block1:/glusterfs/brick2 block2:/glusterfs/brick1 block2:/glusterfs/brick2 force
volume create: zhai-volume: success: please start the volume to access data
root@compute:~#
root@compute:~# gluster volume info
Volume Name: zhai-volume
Type: Distributed-Replicate
Volume ID: 779a7440-e127-43c6-a13f-7420ab8baec8
Status: Created
Snapshot Count: 0
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: compute:/glusterfs/brick1
Brick2: compute:/glusterfs/brick2
Brick3: block1:/glusterfs/brick1
Brick4: block1:/glusterfs/brick2
Brick5: block2:/glusterfs/brick1
Brick6: block2:/glusterfs/brick2
Options Reconfigured:
transport.address-family: inet
Note : The number of bricks should be a multiple of the replica count for a distributed replicated volume. Also, the order in which bricks are specified has a great effect on data protection. Each replica_count consecutive bricks in the list you give will form a replica set, with all replica sets combined into a volume-wide distribute set.
volume创建成功后，会在每台服务器的/glusterfs目录下生成brick1和brick2目录。
查看日志：
root@compute:~# tail -20 /var/log/glusterfs/cli.log
[2017-03-17 03:01:38.518626] I [cli.c:728:main] 0-cli: Started running gluster with version 3.8.10
[2017-03-17 03:01:38.608896] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2017-03-17 03:01:38.608958] I [socket.c:2403:socket_event_handler] 0-transport: disconnecting now
[2017-03-17 03:01:38.619870] I [cli-rpc-ops.c:1117:gf_cli_create_volume_cbk] 0-cli: Received resp to create volume
[2017-03-17 03:01:38.619953] I [input.c:31:cli_batch] 0-: Exiting with: -1
[2017-03-17 03:02:34.746102] I [cli.c:728:main] 0-cli: Started running gluster with version 3.8.10
[2017-03-17 03:02:34.837934] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2017-03-17 03:02:34.838008] I [socket.c:2403:socket_event_handler] 0-transport: disconnecting now
[2017-03-17 03:02:36.111778] I [cli-rpc-ops.c:1117:gf_cli_create_volume_cbk] 0-cli: Received resp to create volume
[2017-03-17 03:02:36.111888] I [input.c:31:cli_batch] 0-: Exiting with: 04.2．启动volume
刚创建好的volume还没有运行，需要执行运行命令方可使用。 在其中一台服务器上执行。sudo gluster volume start vol_name
root@compute:~# ps -ef | grep volume
root 12663 1650 0 11:16 pts/0 00:00:00 grep --color=auto volume
root@compute:~# gluster volume start zhai-volume
volume start: zhai-volume: success
root@compute:~# ps -ef | grep volume
root 12683 1 0 11:16 ? 00:00:00 /usr/sbin/glusterfsd -s compute --volfile-id zhai-volume.compute.glusterfs-brick1 -p /var/lib/glusterd/vols/zhai-volume/run/compute-glusterfs-brick1.pid -S /var/run/gluster/233b16b60a6dacf67c1b711bb1e7e89c.socket --brick-name /glusterfs/brick1 -l /var/log/glusterfs/bricks/glusterfs-brick1.log --xlator-option *-posix.glusterd-uuid=9fdd15f2-c7e1-44e1-9315-d62e1dcfb44d --brick-port 49152 --xlator-option zhai-volume-server.listen-port=49152
root 12702 1 0 11:16 ? 00:00:00 /usr/sbin/glusterfsd -s compute --volfile-id zhai-volume.compute.glusterfs-brick2 -p /var/lib/glusterd/vols/zhai-volume/run/compute-glusterfs-brick2.pid -S /var/run/gluster/55ba583ae215e8215759c8d1aca9548c.socket --brick-name /glusterfs/brick2 -l /var/log/glusterfs/bricks/glusterfs-brick2.log --xlator-option *-posix.glusterd-uuid=9fdd15f2-c7e1-44e1-9315-d62e1dcfb44d --brick-port 49153 --xlator-option zhai-volume-server.listen-port=49153
root 12729 1650 0 11:16 pts/0 00:00:00 grep --color=auto volumeroot@block1:~# ps -ef | grep zhai-volume
root 26213 1 0 11:16 ? 00:00:00 /usr/sbin/glusterfsd -s block1 --volfile-id zhai-volume.block1.glusterfs-brick1 -p /var/lib/glusterd/vols/zhai-volume/run/block1-glusterfs-brick1.pid -S /var/run/gluster/98664ae15cd31af2fd3394f1a239d85b.socket --brick-name /glusterfs/brick1 -l /var/log/glusterfs/bricks/glusterfs-brick1.log --xlator-option *-posix.glusterd-uuid=ac468b50-3d66-4290-aa8d-cb01279e9bf2 --brick-port 49152 --xlator-option zhai-volume-server.listen-port=49152
root 26232 1 0 11:16 ? 00:00:00 /usr/sbin/glusterfsd -s block1 --volfile-id zhai-volume.block1.glusterfs-brick2 -p /var/lib/glusterd/vols/zhai-volume/run/block1-glusterfs-brick2.pid -S /var/run/gluster/e4bf4eb272d50def1259ad279e81bc61.socket --brick-name /glusterfs/brick2 -l /var/log/glusterfs/bricks/glusterfs-brick2.log --xlator-option *-posix.glusterd-uuid=ac468b50-3d66-4290-aa8d-cb01279e9bf2 --brick-port 49153 --xlator-option zhai-volume-server.listen-port=49153root@block2:~# ps -ef | grep zhai-volume
root 24570 1 0 11:16 ? 00:00:00 /usr/sbin/glusterfsd -s block2 --volfile-id zhai-volume.block2.glusterfs-brick1 -p /var/lib/glusterd/vols/zhai-volume/run/block2-glusterfs-brick1.pid -S /var/run/gluster/b5c757abb3485edb13b25f9aaac3c387.socket --brick-name /glusterfs/brick1 -l /var/log/glusterfs/bricks/glusterfs-brick1.log --xlator-option *-posix.glusterd-uuid=06288da0-1b18-42fb-a7b8-cd9ca24d5f26 --brick-port 49152 --xlator-option zhai-volume-server.listen-port=49152
root 24589 1 0 11:16 ? 00:00:00 /usr/sbin/glusterfsd -s block2 --volfile-id zhai-volume.block2.glusterfs-brick2 -p /var/lib/glusterd/vols/zhai-volume/run/block2-glusterfs-brick2.pid -S /var/run/gluster/0acc5e5ccc951203695b9654ef4dbd67.socket --brick-name /glusterfs/brick2 -l /var/log/glusterfs/bricks/glusterfs-brick2.log --xlator-option *-posix.glusterd-uuid=06288da0-1b18-42fb-a7b8-cd9ca24d5f26 --brick-port 49153 --xlator-option zhai-volume-server.listen-port=491534.3．挂载volume
4.3.1 在3台服务器中任一服务器上挂载都可以。
For this step, we will use one of the servers to mount the volume. Typically, you would do this from an external machine, known as a "client". Since using this method would require additional packages to be installed on the client machine, we willuse one of the servers as a simple place to test first, as if it were that "client".
# 挂载任一节点即可(推荐)
root@compute:~# mount -t glusterfs 192.168.4.132:/zhai-volume /mnt
root@compute:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.9G 12K 3.9G 1% /dev
tmpfs 788M 660K 787M 1% /run
/dev/dm-0 909G 2.4G 860G 1% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 3.9G 0 3.9G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sda2 237M 91M 134M 41% /boot
/dev/sda1 511M 3.4M 508M 1% /boot/efi
192.168.4.132:/zhai-volume 2.7T 6.0G 2.6T 1% /mnt4.3.2 在3台服务器之外的服务器上挂载
需要安装glusterfs-client包
否则，出错：
root@network:~# mount -t glusterfs 192.168.4.132:/zhai-volume /mnt
mount: unknown filesystem type 'glusterfs'安装
sudo apt-get install glusterfs-client安装后，执行挂载
# mount -t glusterfs 10.10.30.4:/zhai-volume /mnt
Mount failed. Please check the log file for more details.4.3.3 开机挂载
# 若需要开机自动挂载，请更新/etc/fstab
4.4．使用GlusterFS
a. 挂载了GlusterFS的某个卷后，就可以将其当做本地文件访问，代码中只需使用原生的文件api即可。这种方式使用不一定需要root权限，只要拥有对应目录或文件的权限即可。b. 直接API方式，这种方式需要root权限才能使用，并且java、python、ruby的api包装目前都不够完整，一般情况不推荐使用。
到
测试1：
到/mnt目录下创建文件：
root@compute:/mnt# uname -a >info
root@compute:/mnt# ls
info
root@compute:/mnt# cat info
Linux compute 4.4.0-31-generic #50~14.04.1-Ubuntu SMP Wed Jul 13 01:07:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
root@block2:~# ls /glusterfs/brick1/
info
root@block2:~# ls /glusterfs/brick2
info
root@block2:~# cat /glusterfs/brick1/info
Linux compute 4.4.0-31-generic #50~14.04.1-Ubuntu SMP Wed Jul 13 01:07:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux测试2：
mount -t glusterfs server1:/gv0 /mnt
for i in `seq -w 1 100`; do cp -rp /var/log/messages /mnt/copy-test-$i; done
First, check the mount point:
ls -lA /mnt | wc -l
You should see 100 files returned. Next, check the GlusterFS mount points on each server:
ls -lA /data/brick1/gv0
You should see 100 files on each server using the method we listed here. Without replication, in a distribute only volume (not detailed here), you should see about 50 files on each one.
4.5．卸载volume
卸载与挂载操作是一对。虽然没有卸载也可以停止volume，但是这样做是会出问题，如果集群较大，可能导致后面volume启动失败。sudo umount /mnt
4.6．停止volume
停止与启动操作是一对。停止前最好先卸载所有客户端。sudo gluster volume stop vol_name
4.7．删除volume
删除与创建操作是一对。删除前需要先停止volume。在生产上一般不会删除volumesudo gluster volume delete vol_name
5 使用独立存储
使用独立/dev/sdb
5.1 分区
fdisk /dev/sdb //创建分区，输入,n,p,w，创建新主分区并写入分区表
5.2 Format and mount the bricks
(on both nodes): Note: These examples are going to assume the brick is going to reside on /dev/sdb1.
mkfs.xfs -i size=512 /dev/sdb1
mkdir -p /data/brick1
echo '/dev/sdb1 /data/brick1 xfs defaults 1 2' >> /etc/fstab
mount -a && mount
You should now see sdb1 mounted at /data/brick1
root@block1:~# mkfs.xfs -i size=512 /dev/sdb1
root@compute:~# echo '/dev/sdb1 /glusterfs xfs defaults 1 2' >> /etc/fstab
root@compute:~# mount -a && mount
/dev/mapper/compute--vg-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/cgroup type tmpfs (rw)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
none on /sys/firmware/efi/efivars type efivarfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
none on /run/user type tmpfs (rw,noexec,nosuid,nodev,size=104857600,mode=0755)
none on /sys/fs/pstore type pstore (rw)
tracefs on /var/lib/ureadahead/debugfs/tracing type tracefs (rw,relatime)
/dev/sda2 on /boot type ext2 (rw)
/dev/sda1 on /boot/efi type vfat (rw)
systemd on /sys/fs/cgroup/systemd type cgroup (rw,noexec,nosuid,nodev,none,name=systemd)
rpc_pipefs on /run/rpc_pipefs type rpc_pipefs (rw)
192.168.4.132:/zhai-volume on /mnt type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)
/dev/sdb1 on /glusterfs type xfs (rw)
root@compute:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.9G 12K 3.9G 1% /dev
tmpfs 788M 660K 787M 1% /run
/dev/dm-0 909G 2.4G 860G 1% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 3.9G 0 3.9G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sda2 237M 91M 134M 41% /boot
/dev/sda1 511M 3.4M 508M 1% /boot/efi
/dev/sdb1 932G 34M 932G 1% /glusterfsmount之后再安装glusterfs
5.3 其它命令
#apt-get install xfsprogs //安装xfs文件系统（如果没有的话），更安全可靠
#lshw -short //查看硬件信息，主要是查看硬盘信息，以决定哪块硬盘作为gluster来使用
或使用lsblk来查看块设备
#lsblk6 相关问题
1 Q: 创建/启用/停用/删除volume最终都干了些什么？
(1) 创建volume能看得到的结果是: glusterd在工作目录(默认为/var/lib/glusterd)的vols目录下，创建以volume名称为名字的目录，并在这个目录下创建相关文件记录volume的相关信息；看不到的结果是glusterd程序中也记录了volume的相关信息。
(2) 启用volume的结果是在指定brick的节点上启动glusterfsd和glusterfs进程(注意：如果多个brick在同一节点下，会有多个glusterfsd进程，但只有一个glusterfs进程)，同时在日志文件中更新记录的volume相关信息。
(3) 停用volume的结果是结束glusterfsd和glusterfs进程。
(4)删除volume则是将创建volume的相关文件信息全部清除。
2 错误
root@compute:~# gluster volume create volume-test block1:/glusterfs/repbrick block2:/glusterfs/repbrick
volume create: volume-test: failed: Staging failed on block2. Error: Brick: block2:/glusterfs/repbrick not available. Brick may be containing or be contained by an existing brick
Staging failed on block1. Error: Brick: block1:/glusterfs/repbrick not available. Brick may be containing or be contained by an existing brick这是因为创建volume时，在brick对应的存储目录上，增加了"trusted.glusterfs.volume-id"和"trusted.gfid"两个扩展属性，在删除volume时并未移除这两个属性，再次创建时，对brick的存储目录进行校验，发现已经有了扩展属性，因此会有brick已经是volume的一部分的提示。
通过getfattr命令可以查看这些存储路径的扩展属性
root@block2:~# getfattr -d -m ".*" /glusterfs/repbrick
getfattr: Removing leading '/' from absolute path names
# file: glusterfs/repbrick
trusted.glusterfs.volume-id=0sS29pmfEgRDyMNCX3YAbwsQ==
设置属性
setfattr -n trusted.glusterfs.volume-id -v 0x$(grep volume-id /var/lib/glusterd/vols/volume-down-fault/info | cut -d= -f2 | sed 's/-//g') /glusterfs/brick23 可执行程序
glusterfs安装后，会有gluster，glusterd，glusterfs，glusterfsd这么几个可执行程序，其作用分别为：
gluster：Glusterfs控制台管理程序（Gluster Console Manager），可以以命令形式或者交互式形式对glusterfs中的volume，brick，集群节点等信息进行查看及操作（增，删，改）。
glusterd：软链接指向glusterfsd，Glusterfs的管理进程，负责处理来自gluster的命令。
glusterfs：软链接指向glusterfsd，Glusterfs自带的客户端
glusterfsd：Glusterfs服务端程序
注：glusterd,glusterfs,glusterfsd为最终是运行同一个程序(glusterfsd)，程序内部会根据可执行程序名称加以区别。
staticuint8_t gf_get_process_mode (char*exec_name)
{
char*dup_execname = NULL, *base = NULL;
uint8_t ret = 0;
dup_execname = gf_strdup (exec_name);
base = basename (dup_execname);
if(!strncmp (base, "glusterfsd", 10)) {
ret = GF_SERVER_PROCESS;
}else if (!strncmp(base, "glusterd", 8)) {
ret = GF_GLUSTERD_PROCESS;
}else {
ret = GF_CLIENT_PROCESS;
}
GF_FREE (dup_execname);
returnret;
}4 客户端依靠nfs协议（或gluster native协议）动态的从服务端获取卷的信息。
5 如果建立的是复制卷，gluster客户端会并发的向几个服务器写入\读取数据，而不是通过服务器之间进行数据的传输。
2913
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
