glusterfs Failed to find brick directory

k8s heketi pod 一直处于 creating 状态，describe 这个pod 提示查看如下图日志

failed to get the port number for remote subvolume. please check 'gluster volume status ’ on servers to see if bricks process is running

[root@k8s /]# gluster volume list
heketidbstorage
...
[root@k8s /]# gluster volume info heketidbstorage

Volume Name: heketidbstorage
Type: Replicate
Volume ID: b70fd0a7-6e35-4172-8827-60d5ada55b54
### started ###
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.0.2.34:/var/lib/heketi/mounts/vg_108cece1b709b2d3e0cdebf35a2fec6d/brick_5810845ead005aad0bba6f3f79d1b125/brick
Brick2: 10.0.2.35:/var/lib/heketi/mounts/vg_f47e1d26a2086bb2121f00512259f0e0/brick_af5d5aafafa8ec7ec5d558b375cde722/brick
Brick3: 10.0.2.33:/var/lib/heketi/mounts/vg_250a0c9d90faa2ad4b39b9dbe6c98866/brick_9054bd3d09af01a3f4e94b32afc60e22/brick
Options Reconfigured:
performance.client-io-threads: off
nfs.disable: on
transport.address-family: inet
performance.stat-prefetch: off
performance.write-behind: off
performance.open-behind: off
performance.quick-read: off
performance.strict-o-direct: on
performance.read-ahead: off
performance.io-cache: off
performance.readdir-ahead: off
user.heketi.dbstoragelevel: 1
user.heketi.id: 88e8ff5a694a6e16093b2cfc06396a9f
[root@k8s /]# gluster volume stop heketidbstorage
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
volume stop: heketidbstorage: success
[root@k8s /]# gluster volume start heketidbstorage
volume start: heketidbstorage: failed: Failed to find brick directory /var/lib/heketi/mounts/vg_108cece1b709b2d3e0cdebf35a2fec6d/brick_5810845ead005aad0bba6f3f79d1b125/brick for volume heketidbstorage. Reason : No such file or directory
[root@k8s /]# gluster volume info heketidbstorage

Volume Name: heketidbstorage
Type: Replicate
Volume ID: b70fd0a7-6e35-4172-8827-60d5ada55b54
### stopped ###
Status: Stopped
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.0.2.34:/var/lib/heketi/mounts/vg_108cece1b709b2d3e0cdebf35a2fec6d/brick_5810845ead005aad0bba6f3f79d1b125/brick
Brick2: 10.0.2.35:/var/lib/heketi/mounts/vg_f47e1d26a2086bb2121f00512259f0e0/brick_af5d5aafafa8ec7ec5d558b375cde722/brick
Brick3: 10.0.2.33:/var/lib/heketi/mounts/vg_250a0c9d90faa2ad4b39b9dbe6c98866/brick_9054bd3d09af01a3f4e94b32afc60e22/brick
Options Reconfigured:
performance.client-io-threads: off
nfs.disable: on
transport.address-family: inet
performance.stat-prefetch: off
performance.write-behind: off
performance.open-behind: off
performance.quick-read: off
performance.strict-o-direct: on
performance.read-ahead: off
performance.io-cache: off
performance.readdir-ahead: off
user.heketi.dbstoragelevel: 1
user.heketi.id: 88e8ff5a694a6e16093b2cfc06396a9f
[root@k8s /]# gluster volume start heketidbstorage
volume start: heketidbstorage: failed: Failed to find brick directory /var/lib/heketi/mounts/vg_108cece1b709b2d3e0cdebf35a2fec6d/brick_5810845ead005aad0bba6f3f79d1b125/brick for volume heketidbstorage. Reason : No such file or directory

登录glusterfs pod ，查看 heketidbstorage gluster volume info ， 副本卷 brick（副本为3） 不知为何都不存在了 - -