ceph rbd 对象组成

/* New-style rbd image 'foo' consists of objects
 *   rbd_id.foo              - id of image
 *   rbd_header.<id>         - image metadata
 *   rbd_object_map.<id>     - optional image object map
 *   rbd_data.<id>.00000000
 *   rbd_data.<id>.00000001
 *   ...                     - data
 */

#define RBD_HEADER_PREFIX      "rbd_header."
#define RBD_OBJECT_MAP_PREFIX  "rbd_object_map."
#define RBD_DATA_PREFIX        "rbd_data."
#define RBD_ID_PREFIX          "rbd_id."

/*
 * old-style rbd image 'foo' consists of objects
 *   foo.rbd      - image metadata
 *   rb.<idhi>.<idlo>.00000000
 *   rb.<idhi>.<idlo>.00000001
 *   ...          - data
 */

#define RBD_SUFFIX      ".rbd"
#define RBD_DIRECTORY           "rbd_directory"
#define RBD_INFO                "rbd_info"

/*
 * rbd_children object in each pool contains omap entries
 * that map parent (poolid, imageid, snapid) to a list of children
 * (imageids; snapids aren't required because we get all the snapshot
 * info from a read of the child's header object anyway).
 *
 * The clone operation writes a new item to this child list, and rm or
 * flatten removes an item, and may remove the whole entry if no children
 * exist after the rm/flatten.
 *
 * When attempting to remove a parent, all pools are searched for
 * rbd_children objects with entries referring to that parent; if any
 * exist (and those children exist), the parent removal is prevented.
 */
#define RBD_CHILDREN        "rbd_children"

老镜像格式：
导入数据创建镜像使用默认镜像格式

[root@ceph-osd-241 ~]# rbd -p liyankun import BIOS_CR1RR_LN_2.5.4.BIN test
Importing image: 100% complete...done.
[root@ceph-osd-241 ~]# rados -p liyankun ls
rb.0.bbca0.74b0dc51.000000000007
rb.0.bbca0.74b0dc51.000000000001
rb.0.bbca0.74b0dc51.000000000008
rb.0.bbca0.74b0dc51.000000000005  //data 
test.rbd                          //镜像的metadata
rbd_directory
rb.0.bbca0.74b0dc51.000000000004
rb.0.bbca0.74b0dc51.000000000010
rb.0.bbca0.74b0dc51.00000000000f
rb.0.bbca0.74b0dc51.000000000006
rb.0.bbca0.74b0dc51.00000000000b
rb.0.bbca0.74b0dc51.000000000003
rb.0.bbca0.74b0dc51.000000000009
rb.0.bbca0.74b0dc51.00000000000e
rb.0.bbca0.74b0dc51.00000000000d
rb.0.bbca0.74b0dc51.00000000000c
rb.0.bbca0.74b0dc51.00000000000a
rb.0.bbca0.74b0dc51.000000000002
rb.0.bbca0.74b0dc51.000000000000

对镜像创建快照

[root@ceph-osd-241 ~]# rbd --pool liyankun snap create --snap testsnap test

镜像快照创建成功

[root@ceph-osd-241 ~]# rbd -p liyankun snap ls  test
SNAPID NAME         SIZE 
     4 testsnap 65553 kB

查看对象个数是否发生变化

[root@ceph-osd-241 ~]# rados -p liyankun ls
rb.0.bbca0.74b0dc51.000000000007
rb.0.bbca0.74b0dc51.000000000001
rb.0.bbca0.74b0dc51.000000000008
rb.0.bbca0.74b0dc51.000000000005
test.rbd
rbd_directory
rb.0.bbca0.74b0dc51.000000000004
rb.0.bbca0.74b0dc51.000000000010
rb.0.bbca0.74b0dc51.00000000000f
rb.0.bbca0.74b0dc51.000000000006
rb.0.bbca0.74b0dc51.00000000000b
rb.0.bbca0.74b0dc51.000000000003
rb.0.bbca0.74b0dc51.000000000009
rb.0.bbca0.74b0dc51.00000000000e
rb.0.bbca0.74b0dc51.00000000000d
rb.0.bbca0.74b0dc51.00000000000c
rb.0.bbca0.74b0dc51.00000000000a
rb.0.bbca0.74b0dc51.000000000002
rb.0.bbca0.74b0dc51.000000000000

保护快照防止数据丢失，为clone做准备，从输出信息可以得知该功能没有实现

[root@ceph-osd-241 ~]# rbd -p liyankun snap protect --image test --snap testsnap
rbd: protecting snap failed: (38) Function not implemented
2016-09-02 11:40:32.276782 7f44a6c0e760 -1 librbd: snap_protect: image must support layering

从创建的快照中克隆一个镜像，由于image-format 1 不支持clone，所以失败。

[root@ceph-osd-241 ~]# rbd -p liyankun  clone --image test --snap testsnap --dest-pool liyankun --dest test_child
2016-09-02 11:43:47.685116 7f6db89b1760 -1 librbd: parent image must be in new format
rbd: clone error: (22) Invalid argument

确定当前的image format为 1

[root@ceph-osd-241 ~]# rbd -p liyankun info test
rbd image 'test':
        size 65553 kB in 17 objects
        order 22 (4096 kB objects)
        block_name_prefix: rb.0.bbca0.74b0dc51
        format: 1
[root@ceph-osd-241 ~]# 

新镜像格式：
清除环境重新导入镜像，镜像格式指定为 2

[root@ceph-osd-241 ~]# rbd -p liyankun import BIOS_CR1RR_LN_2.5.4.BIN test --image-format 2
Importing image: 100% complete...done.
[root@ceph-osd-241 ~]# rados -p liyankun ls
rbd_data.bbe1d74b0dc51.0000000000000010
rbd_data.bbe1d74b0dc51.0000000000000004
rbd_data.bbe1d74b0dc51.0000000000000003
rbd_data.bbe1d74b0dc51.0000000000000002
rbd_data.bbe1d74b0dc51.0000000000000006
rbd_data.bbe1d74b0dc51.000000000000000a
rbd_data.bbe1d74b0dc51.000000000000000b
rbd_data.bbe1d74b0dc51.000000000000000d
rbd_directory
rbd_data.bbe1d74b0dc51.0000000000000000
rbd_data.bbe1d74b0dc51.0000000000000007       //data
rbd_data.bbe1d74b0dc51.000000000000000e
rbd_header.bbe1d74b0dc51                     //image metadata
rbd_data.bbe1d74b0dc51.0000000000000005
rbd_data.bbe1d74b0dc51.0000000000000009
rbd_data.bbe1d74b0dc51.000000000000000f
rbd_data.bbe1d74b0dc51.0000000000000001
rbd_id.test                                  //id of image
rbd_data.bbe1d74b0dc51.000000000000000c
rbd_data.bbe1d74b0dc51.0000000000000008

创建镜像快照

[root@ceph-osd-241 ~]# rbd --pool liyankun snap create --snap testsnap test

保护镜像

[root@ceph-osd-241 ~]# rbd -p liyankun snap protect --image test --snap testsnap

从镜像的快照中克隆出新的镜像

[root@ceph-osd-241 ~]# rbd clone -p liyankun  --image test --snap testsnap --dest-pool liyankun --dest test_child
[root@ceph-osd-241 ~]# rados -p liyankun ls
rbd_data.bbe1d74b0dc51.0000000000000010
rbd_data.bbe1d74b0dc51.0000000000000004
rbd_data.bbe1d74b0dc51.0000000000000003
rbd_data.bbe1d74b0dc51.0000000000000002
rbd_data.bbe1d74b0dc51.0000000000000006
rbd_header.bb0e62ae8944a
rbd_children
rbd_data.bbe1d74b0dc51.000000000000000a
rbd_data.bbe1d74b0dc51.000000000000000b
rbd_data.bbe1d74b0dc51.000000000000000d
rbd_directory
rbd_data.bbe1d74b0dc51.0000000000000000
rbd_data.bbe1d74b0dc51.0000000000000007
rbd_data.bbe1d74b0dc51.000000000000000e
rbd_header.bbe1d74b0dc51
rbd_data.bbe1d74b0dc51.0000000000000005
rbd_data.bbe1d74b0dc51.0000000000000009
rbd_data.bbe1d74b0dc51.000000000000000f
rbd_id.test_child
rbd_data.bbe1d74b0dc51.0000000000000001
rbd_id.test
rbd_data.bbe1d74b0dc51.000000000000000c
rbd_data.bbe1d74b0dc51.0000000000000008

查看镜像格式

[root@ceph-osd-241 ~]# rbd -p liyankun info test
rbd image 'test':
        size 65553 kB in 17 objects
        order 22 (4096 kB objects)
        block_name_prefix: rbd_data.bbe1d74b0dc51
        format: 2
        features: layering
        flags: 

列举rbd_children omap对象中的keys

[root@ceph-osd-241 ~]# rados -p liyankun listomapkeys rbd_children 
bbe1d74b0dc51

列举rbd_children omap对象中的keys and values

[root@ceph-osd-241 ~]# rados -p liyankun listomapvals rbd_children 
key (33 bytes):
0000 : 10 00 00 00 00 00 00 00 0d 00 00 00 62 62 65 31 : ............bbe1
0010 : 64 37 34 62 30 64 63 35 31 06 00 00 00 00 00 00 : d74b0dc51.......
0020 : 00                                              : .

value (21 bytes) :
0000 : 01 00 00 00 0d 00 00 00 62 62 30 65 36 32 61 65 : ........bb0e62ae
0010 : 38 39 34 34 61                                  : 8944a

[root@ceph-osd-241 ~]# rados -p liyankun ls
rbd_data.bbe1d74b0dc51.0000000000000010
rbd_data.bbe1d74b0dc51.0000000000000004
rbd_data.bbe1d74b0dc51.0000000000000003
rbd_data.bbe1d74b0dc51.0000000000000002
rbd_data.bbe1d74b0dc51.0000000000000006
rbd_header.bb0e62ae8944a
rbd_children
rbd_data.bbe1d74b0dc51.000000000000000a
rbd_data.bbe1d74b0dc51.000000000000000b
rbd_data.bbe1d74b0dc51.000000000000000d
rbd_directory
rbd_data.bbe1d74b0dc51.0000000000000000
rbd_data.bbe1d74b0dc51.0000000000000007
rbd_data.bbe1d74b0dc51.000000000000000e
rbd_header.bbe1d74b0dc51
rbd_data.bbe1d74b0dc51.0000000000000005
rbd_data.bbe1d74b0dc51.0000000000000009
rbd_data.bbe1d74b0dc51.000000000000000f
rbd_id.test_child
rbd_data.bbe1d74b0dc51.0000000000000001
rbd_id.test
rbd_data.bbe1d74b0dc51.000000000000000c
rbd_data.bbe1d74b0dc51.0000000000000008
[root@ceph-osd-241 ~]# rados -p liyankun stat rbd_id.test_child
liyankun/rbd_id.test_child mtime 2016-09-02 13:34:00.000000, size 17

[root@ceph-osd-241 ~]# rados -p liyankun ls | grep header
rbd_header.bb0e62ae8944a
rbd_header.bbe1d74b0dc51

查看rbd_childre omap对象内容，确定镜像之间的父子关系维护在rbd_children omap对象中。

[root@ceph-osd-241 ~]# rados -p liyankun listomapvals rbd_children 
key (33 bytes):
0000 : 10 00 00 00 00 00 00 00 0d 00 00 00 62 62 65 31 : ............bbe1
0010 : 64 37 34 62 30 64 63 35 31 06 00 00 00 00 00 00 : d74b0dc51.......
0020 : 00                                              : .

value (21 bytes) :
0000 : 01 00 00 00 0d 00 00 00 62 62 30 65 36 32 61 65 : ........bb0e62ae
0010 : 38 39 34 34 61                                  : 8944a

[root@ceph-osd-241 ~]# 

如果一个镜像是克隆而来，那么他的格式和父镜像格式一致。

[root@ceph-osd-241 ~]# rbd -p liyankun info test_child
rbd image 'test_child':
        size 65553 kB in 17 objects
        order 22 (4096 kB objects)
        block_name_prefix: rbd_data.bb0e62ae8944a
        format: 2
        features: layering
        flags: 
        parent: liyankun/test@testsnap
        overlap: 65553 kB