ceph集群提示pgs: 100.000% pgs unknown的一个解决办法

最新推荐文章于 2024-10-09 14:17:27 发布

原创最新推荐文章于 2024-10-09 14:17:27 发布 · 1w 阅读

8 ·

CC 4.0 BY-SA版权

文章标签：

#ceph

ceph 专栏收录该内容

23 篇文章

订阅专栏

本文详细记录了解决Ceph集群健康警告的过程，通过调整crushmap设置，将主机节点正确地加入到集群的root中，最终使集群状态恢复正常。

搭完集群后，ceph -w提示

[root@node1 ~]# ceph -w
  cluster:
    id:     2386c327-8eb1-4dd7-9fed-fedff947c383
    health: HEALTH_WARN
            Reduced data availability: 128 pgs inactive
 
  services:
    mon: 2 daemons, quorum node0,node1
    mgr: openstack(active)
    osd: 3 osds: 3 up, 3 in
 
  data:
    pools:   1 pools, 128 pgs
    objects: 0  objects, 0 B
    usage:   6.0 GiB used, 118 GiB / 124 GiB avail
    pgs:     100.000% pgs unknown
             128 unknown

在网上查了很久说是需要将osd crush update on start 设置为true，我试了之后发现也不行，最后发现是crush map设置的不对。
从下面可以看出我的host不在root之中

[root@node1 ~]# ceph osd crush tree
ID CLASS WEIGHT  TYPE NAME    
-7       1.00000 host node2   
 2   hdd 1.00000     osd.2    
-5       1.00000 host node1   
 1   hdd 1.00000     osd.1    
-2       1.00000 host node0   
 0   hdd 1.00000     osd.0    
-1             0 root default 




host node0 {
        id -2           # do not change unnecessarily
        id -3 class hdd         # do not change unnecessarily
        # weight 1.000
        alg straw2
        hash 0  # rjenkins1
        item osd.0 weight 1.000
}
host node1 {
        id -5           # do not change unnecessarily
        id -6 class hdd         # do not change unnecessarily
        # weight 1.000
        alg straw2
        hash 0  # rjenkins1
        item osd.1 weight 1.000
}
host node2 {
        id -7           # do not change unnecessarily
        id -8 class hdd         # do not change unnecessarily
        # weight 1.000
        alg straw2
        hash 0  # rjenkins1
        item osd.2 weight 1.000
}
# buckets
root default {
        id -1           # do not change unnecessarily
        id -4 class hdd         # do not change unnecessarily
        # weight 0.000
        alg straw2
        hash 0  # rjenkins1
}

修改方法
1：

将crush map导出
ceph osd getcrushmap -o /tmp/mycrushmap

2：

将crush map反编译为可读模式
crushtool -d /tmp/mycrushmap > /tmp/mycrushmap.txt

3：

将host类型所属的主机节点加入到root中，并注意root列必须位于所有的host列下面
# buckets
root default {
        id -1           # do not change unnecessarily
        id -4 class hdd         # do not change unnecessarily
        # weight 0.000
        alg straw2
        hash 0  # rjenkins1
        item node0 weight 1.000
        item node1 weight 1.000
        item node2 weight 1.000
}

4：

	将修改过的crush map导入集群
	ceph osd setcrushmap -i /tmp/newcrushmap

查看集群状态已经变成正常

[root@node1 ~]# ceph -s
  cluster:
    id:     2386c327-8eb1-4dd7-9fed-fedff947c383
    health: HEALTH_OK
 
  services:
    mon: 2 daemons, quorum node0,node1
    mgr: openstack(active)
    osd: 3 osds: 3 up, 3 in
 
  data:
    pools:   1 pools, 128 pgs
    objects: 0  objects, 0 B
    usage:   6.0 GiB used, 118 GiB / 124 GiB avail
    pgs:     128 active+clean