#glusterfs error

GlusterFS: {path} or a prefix of it is already part of a volume

Starting with GlusterFS 3.3, one change has been the check to see if a directory (or any of it's ancestors) is already part of a volume. This is causing many support questions in #gluster.

This was implemented because if you remove a brick from a volume and continue to use the volume, you can get file into a state where re-adding a former brick can cause all sort of problems, many of which can result in data loss.

If you're going to reuse a brick, make sure you know what you're doing.

The Solution

For the directory (or any parent directories) that was formerly part of a volume, simply:

setfattr -x trusted.glusterfs.volume-id $brick_path
setfattr -x trusted.gfid $brick_path
rm -rf $brick_path/.glusterfs

Don't worry if it says that the attribute does not exist. As long as it doesn't exist, you're in good shape.

Finally, restart glusterd to ensure it's not "remembering" the old bricks.

See the bugzilla entry for more details and see Jeff Darcy's article for more information about how GlusterFS uses extended attributes.

[root@test glusterfs]# gluster volume create hello  replica 3  test.144:/data0/glusterfs test.145:/data0/glusterfs test.146:/data0/glusterfs
volume create: hello: success: please start the volume to access data
[root@test glusterfs]# gluster volume delete hello
Deleting volume will erase all information about the volume. Do you want to continue? (y/n) y 
volume delete: hello: success
[root@test glusterfs]# gluster volume create gfs  replica 3  test.144:/data0/glusterfs test.145:/data0/glusterfs test.146:/data0/glusterfs
volume create: gfs: failed: Staging failed on test.144. Error: /data0/glusterfs is already part of a volume
Staging failed on test.146. Error: /data0/glusterfs is already part of a volume
Staging failed on test.145. Error: /data0/glusterfs is already part of a volume
[root@test glusterfs]# ssh test.144 'setfattr -x trusted.glusterfs.volume-id /data0/glusterfs/ && setfattr -x trusted.gfid /data0/glusterfs/ && rm -rf /data0/glusterfs/..glusterfs'
setfattr: /data0/glusterfs/: No such attribute
[root@test glusterfs]# ssh test.145 'setfattr -x trusted.glusterfs.volume-id /data0/glusterfs/ && setfattr -x trusted.gfid /data0/glusterfs/ && rm -rf /data0/glusterfs/..glusterfs'
setfattr: /data0/glusterfs/: No such attribute
[root@test glusterfs]# ssh test.146 'setfattr -x trusted.glusterfs.volume-id /data0/glusterfs/ && setfattr -x trusted.gfid /data0/glusterfs/ && rm -rf /data0/glusterfs/..glusterfs'
setfattr: /data0/glusterfs/: No such attribute
[root@test glusterfs]# 
[root@test glusterfs]# gluster volume create gfs  replica 3  test.144:/data0/glusterfs test.145:/data0/glusterfs test.146:/data0/glusterfs
volume create: gfs: success: please start the volume to access data
[root@test glusterfs]#