ceph crush 基本操作

ceph crush 基本操作

方法一: 命令行修改

root@ceph-client:~# ceph osd crush add-bucket test-bucket root
added bucket test-bucket type root to crush map

root@ceph-client:~# ceph osd tree
ID WEIGHT  TYPE NAME          UP/DOWN REWEIGHT PRIMARY-AFFINITY 
-7       0 root test-bucket                                     
-1 4.00000 root default                                         
-2       0     host ceph-osd1                                   
-3       0     host ceph-osd2                                   
-4 1.00000     host ceph-mon2                                   
 2 1.00000         osd.2           up  1.00000          1.00000 
-5 2.00000     host ceph-mon1                                   
 3 1.00000         osd.3           up  1.00000          1.00000 
 1 1.00000         osd.1           up  1.00000          1.00000 
-6 1.00000     host ceph-mon                                    
 0 1.00000         osd.0           up  1.00000          1.00000 

root@ceph-client:~# ceph osd crush move ceph-osd1 root=test-bucket
moved item id -2 name 'ceph-osd1' to location {root=test-bucket} in crush map

root@ceph-client:~# ceph osd tree
ID WEIGHT  TYPE NAME          UP/DOWN REWEIGHT PRIMARY-AFFINITY 
-7       0 root test-bucket                                     
-2       0     host ceph-osd1                                   
-1 4.00000 root default                                         
-3       0     host ceph-osd2                                   
-4 1.00000     host ceph-mon2                                   
 2 1.00000         osd.2           up  1.00000          1.00000 
-5 2.00000     host ceph-mon1                                   
 3 1.00000         osd.3           up  1.00000          1.00000 
 1 1.00000         osd.1           up  1.00000          1.00000 
-6 1.00000     host ceph-mon                                    
 0 1.00000         osd.0           up  1.00000          1.00000 

【如果新建了rule,并且将rule绑定了这个bucket下,需要先将rule删除;然后再删除这个bucket】
root@ceph-client:~# ceph osd crush remove test-bucket
removed item id -7 name 'test-bucket' from crush map

root@ceph-client:~# ceph osd tree
ID WEIGHT  TYPE NAME          UP/DOWN REWEIGHT PRIMARY-AFFINITY 
-1 4.00000 root default                                         
-2       0     host ceph-osd1                                   
-3       0     host ceph-osd2                                   
-4 1.00000     host ceph-mon2                                   
 2 1.00000         osd.2           up  1.00000          1.00000 
-5 2.00000     host ceph-mon1                                   
 3 1.00000         osd.3           up  1.00000          1.00000 
 1 1.00000         osd.1           up  1.00000          1.00000 
-6 1.00000     host ceph-mon                                    
 0 1.00000         osd.0           up  1.00000          1.00000 

方法二: 修改crush map文件

ceph官网crush map操作说明:

1. GET A CRUSH MAP
To get the CRUSH map for your cluster, execute the following:

ceph osd getcrushmap -o {compiled-crushmap-filename}

Ceph will output (-o) a compiled CRUSH map to the filename you specified. Since the CRUSH map is in a compiled form, you must decompile it first before you can edit it.

2. DECOMPILE A CRUSH MAP

To decompile a CRUSH map, execute the following:

crushtool -d {compiled-crushmap-filename} -o {decompiled-crushmap-filename}

Ceph will decompile (-d) the compiled CRUSH map and output (-o) it to the filename you specified.

3. Edit crush map

4. COMPILE A CRUSH MAP

To compile a CRUSH map, execute the following:

crushtool -c {decompiled-crush-map-filename} -o {compiled-crush-map-filename}

Ceph will store a compiled CRUSH map to the filename you specified.

5. SET A CRUSH MAP

To set the CRUSH map for your cluster, execute the following:

ceph osd setcrushmap -i  {compiled-crushmap-filename}

Ceph will input the compiled CRUSH map of the filename you specified as the CRUSH map for the cluster.

具体用例:

root@ceph-client:~# ceph osd getcrushmap -o crushmap.map
got crush map from osdmap epoch 114

root@ceph-client:~# crushtool -d crushmap.map -o crushmap.txt

root@ceph-client:~# vim crushmap.txt 
//编辑crushmap文件,修改成期望的crush结构

root@ceph-client:~# crushtool -c crushmap.txt -o crushmap.new

root@ceph-client:~# ceph osd setcrushmap -i crushmap.new 
set crush map

crushmap文件修改:

  • 增加一个bucket:
    在bucket最后直接添加:(复制之前的一个bucket,然后修改bucket name、id(id继续增加就行(注意是负的))、把它下面的item去掉,目前还没有item)
root test-bucket {
        id -7           # do not change unnecessarily
        # weight 0.000
        alg straw
        hash 0  # rjenkins1
}


  • 移动一个host bucket到root节点上:
    直接将下面的两个节点copy到root test-bucket之前;再test-bucket里面加上

    item ceph-osd1 weight 0.000 <br>
    item ceph-osd2 weight 0.000</li>


当然weight要按照host本身的weight;这个时候一定注意原来所在的root节点中的那些host删除。
【这些不用担心,在编译crushmap的时候,会报错的。】

host ceph-osd1 {
        id -2           # do not change unnecessarily
        # weight 0.000
        alg straw
        hash 0  # rjenkins1
}
host ceph-osd2 {
        id -3           # do not change unnecessarily
        # weight 0.000
        alg straw
        hash 0  # rjenkins1
}
  • 删除一个bucket:
    直接删除该bucket的字段,注意同时去掉关联的东西。

crushmap.txt 内容:

# begin crush map
tunable choose_local_tries 0
tunable choose_local_fallback_tries 0
tunable choose_total_tries 50
tunable chooseleaf_descend_once 1
tunable straw_calc_version 1

# devices
device 0 osd.0
device 1 osd.1
device 2 osd.2
device 3 osd.3

# types
type 0 osd
type 1 host
type 2 chassis
type 3 rack
type 4 row
type 5 pdu
type 6 pod
type 7 room
type 8 datacenter
type 9 region
type 10 root

# buckets
host ceph-mon2 {
        id -4           # do not change unnecessarily
        # weight 1.000
        alg straw
        hash 0  # rjenkins1
        item osd.2 weight 1.000
}
host ceph-mon1 {
        id -5           # do not change unnecessarily
        # weight 2.000
        alg straw
        hash 0  # rjenkins1
        item osd.3 weight 1.000
        item osd.1 weight 1.000
}
host ceph-mon {
        id -6           # do not change unnecessarily
        # weight 1.000
        alg straw
        hash 0  # rjenkins1
        item osd.0 weight 1.000
}
root default {
        id -1           # do not change unnecessarily
        # weight 4.000
        alg straw
        hash 0  # rjenkins1
        item ceph-mon2 weight 1.000
        item ceph-mon1 weight 2.000
        item ceph-mon weight 1.000
}
host ceph-osd1 {
        id -2           # do not change unnecessarily
        # weight 0.000
        alg straw
        hash 0  # rjenkins1
}
host ceph-osd2 {
        id -3           # do not change unnecessarily
        # weight 0.000
        alg straw
        hash 0  # rjenkins1
}
root test-bucket {
        id -7           # do not change unnecessarily
        # weight 0.000
        alg straw
        hash 0  # rjenkins1
        item ceph-osd1 weight 0.000
        item ceph-osd2 weight 0.000
}

# rules
rule replicated_ruleset {
        ruleset 0
        type replicated
        min_size 1
        max_size 10
        step take default
        step chooseleaf firstn 0 type host
        step emit
}

# end crush map 
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值