Ceph高级工具介绍之ceph-objectstore-tool的使用

转载：https://blog.csdn.net/scaleqiao/article/details/52024024

0 简介

ceph-objectstore-tool工具是Ceph提供的一个low level（可以操作object和pg）的工具，可以用来查看、修改、删除ObjectStore上的object、pg以及dump OSD journal中的数据，请谨慎使用这个工具，它有可能会造成数据的永久丢失，可能这也是很少有文章介绍它的一个原因吧。

1 使用

ceph-objectstore-tool目前没有manpage，它的help文档写的也不是很友好。

[root@ceph03 ~]# ceph-objectstore-tool -h
Must provide --data-path
 
Allowed options:
  --help                      produce help message
  --type arg                  Arg is one of [filestore (default), memstore]
  --data-path arg             path to object store, mandatory
  --journal-path arg          path to journal, mandatory for filestore type
  --pgid arg                  PG id, mandatory for info, log, remove, export,
                              rm-past-intervals, mark-complete
  --op arg                    Arg is one of [info, log, remove, mkfs, fsck,
                              fuse, export, import, list, fix-lost, list-pgs,
                              rm-past-intervals, dump-journal, dump-super,
                              meta-list, get-osdmap, set-osdmap,
                              get-inc-osdmap, set-inc-osdmap, mark-complete]
  --epoch arg                 epoch# for get-osdmap and get-inc-osdmap, the
                              current epoch in use if not specified
  --file arg                  path of file to export, import, get-osdmap,
                              set-osdmap, get-inc-osdmap or set-inc-osdmap
  --mountpoint arg            fuse mountpoint
  --format arg (=json-pretty) Output format which may be json, json-pretty,
                              xml, xml-pretty
  --debug                     Enable diagnostic output to stderr
  --force                     Ignore some types of errors and proceed with
                              operation - USE WITH CAUTION: CORRUPTION POSSIBLE
                              NOW OR IN THE FUTURE
  --skip-journal-replay       Disable journal replay
  --skip-mount-omap           Disable mounting of omap
  --head                      Find head/snapdir when searching for objects by
                              name
  --dry-run                   Don't modify the objectstore
 
Positional syntax:
ceph-objectstore-tool ... <object> (get|set)-bytes [file]
ceph-objectstore-tool ... <object> set-(attr|omap) <key> [file]
ceph-objectstore-tool ... <object> (get|rm)-(attr|omap) <key>
ceph-objectstore-tool ... <object> get-omaphdr
ceph-objectstore-tool ... <object> set-omaphdr [file]
ceph-objectstore-tool ... <object> list-attrs
ceph-objectstore-tool ... <object> list-omap
ceph-objectstore-tool ... <object> remove
ceph-objectstore-tool ... <object> dump
ceph-objectstore-tool ... <object> set-size
ceph-objectstore-tool ... <object> remove-clone-metadata <cloneid>
 
<object> can be a JSON object description as displayed by --op list.
<object> can be an object name which will be looked up in all the OSD's PGs.
<object> can be the empty string ('') which with a provided pgid specifies the pgmeta object
 
The optional [file] argument will read stdin or write stdout if not specified or if '-' specified.
从上面的help文档中，发现没有提供使用方法或者例子，一般需要按照以下方法使用：
ceph-objectstore-tool --data-path <osd的data目录的路径> --journal-path <osd journal的路径> --type <objectstore的类型> --op <具体操作的类型> ，其中type目前支持filestore和bluestore两种，op操作，支持help种列举出来的操作。
3 使用事例

下面介绍一下，ceph-objectstore-tool的常用用法

1）list osd上有哪些pg，比如如果要list osd.2种存储哪些pg，可以执行以下命令：

<span style="font-size:10px;">[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path 
/var/lib/ceph/osd/ceph-2/journal --type filestore --op list-pgs</span>
2）除了能list pg之外，还能list某个pg的pg_info信息

[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path 
/var/lib/ceph/osd/ceph-2/journal --type filestore  --pgid 0.0 --op info</span></span>
3)  list某个pg中object的信息

[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --type filestore  --pgid 0.38 --op list
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
["0.38",{"oid":"rbd_id.test","key":"","snapid":-2,"hash":2586801272,"max":0,"pool":0,"namespace":"","max":0}]
["0.38",{"oid":"rbd_data.6e742ae8944a.0000000000000000","key":"","snapid":-2,"hash":3591771128,"max":0,"pool":0,"namespace":"","max":0}]
3) dump某个pg的pg_log信息

[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --type filestore  --pgid 0.36 --op log
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
{
    "pg_log_t": {
        "head": "400'27",
        "tail": "0'0",
        "log": [
            {
                "op": "modify  ",
                "object": "0:6c8c8ef5:::rb.0.5e9a.2ae8944a.0000000000db:head",
                "version": "376'1",
                "prior_version": "0'0",
                "reqid": "client.24221.1:135326",
                "extra_reqids": [],
                "mtime": "2016-07-21 22:19:29.377000",
                "mod_desc": {
                    "object_mod_desc": {
                        "can_local_rollback": false,
                        "rollback_info_completed": true,
                        "ops": []
                    }
                }
            },
...
        ]
    },
    "pg_missing_t": {
        "missing": []
    },
    "map": {
        "divergent_priors": []
    }
4) remove某个pg

[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --type filestore  --pgid 0.36 --op remove
5) export/import 一个pg

[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --type filestore  --pgid 0.36 --op export --file 1.txt
[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --type filestore  --op import --file 1.txt
6) 该命令还可以dump出OSD的super_block信息

[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --op dump-super
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
{
    "cluster_fsid": "7b4eb697-fd53-4bd8-9b7a-ca1b37110623",
    "osd_fsid": "3549e257-373a-44a0-a7d9-99068e9854dc",
    "whoami": 2,
    "current_epoch": 442,
    "oldest_map": 1,
    "newest_map": 442,
    "weight": 0.000000,
    "compat": {
        "compat": {},
        "ro_compat": {},
        "incompat": {
            "feature_1": "initial feature set(~v.18)",
            "feature_2": "pginfo object",
            "feature_3": "object locator",
            "feature_4": "last_epoch_clean",
            "feature_5": "categories",
            "feature_6": "hobjectpool",
            "feature_7": "biginfo",
            "feature_8": "leveldbinfo",
            "feature_9": "leveldblog",
            "feature_10": "snapmapper",
            "feature_12": "transaction hints",
            "feature_13": "pg meta object"
        }
    },
    "clean_thru": 442,
    "last_epoch_mounted": 439
}
7）dump一个object所有的元数据信息

[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal rbd_data.6e742ae8944a.0000000000000013 dump
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
{
    "id": {
        "oid": "rbd_data.6e742ae8944a.0000000000000013",
        "key": "",
        "snapid": -2,
        "hash": 4028149687,
        "max": 0,
        "pool": 0,
        "namespace": "",
        "max": 0
    },
    "info": {
        "oid": {
            "oid": "rbd_data.6e742ae8944a.0000000000000013",
            "key": "",
            "snapid": -2,
            "hash": 4028149687,
            "max": 0,
            "pool": 0,
            "namespace": ""
        },
        "version": "442'1060",
        "prior_version": "442'1059",
        "last_reqid": "osd.1.0:59339",
        "user_version": 1059,
        "size": 4194304,
        "mtime": "2016-07-22 14:58:47.086000",
        "local_mtime": "2016-07-22 14:58:47.306385",
        "lost": 0,
        "flags": 52,
        "snaps": [],
        "truncate_seq": 0,
        "truncate_size": 0,
        "data_digest": 3408368108,
        "omap_digest": 4294967295,
        "watchers": {}
    },
    "stat": {
        "size": 4194304,
        "blksize": 4096,
        "blocks": 8192,
        "nlink": 1
    },
    "SnapSet": {
        "snap_context": {
            "seq": 0,
            "snaps": []
        },
        "head_exists": 1,
        "clones": []
    }
}
8)  当然，它可以使用get-osdmap导出集群中leveldb数据库中的数据，比如OSDMap，而且可以dump出各个epoch的map，这个大家可以在下面自己试验一下。
--------------------- 
作者：瞧见风 
来源：CSDN 
原文：https://blog.csdn.net/scaleqiao/article/details/52024024 
版权声明：本文为博主原创文章，转载请附上博文链接！