HFile校验

 HBase 表大小统计:

hbase org.apache.hadoop.hbase.mapreduce.RowCounter test:standard

HFile校验:

#]hbase hfile

usage: HFile [-a] [-b] [-e] [-f <arg> | -r <arg>] [-h] [-k] [-m] [-p]
       [-s] [-v] [-w <arg>]
 -a,--checkfamily         Enable family check
 -b,--printblocks         Print block index meta data
 -e,--printkey            Print keys
 -f,--file <arg>          File to scan. Pass full-path; e.g.
                          hdfs://a:9000/hbase/hbase:meta/12/34
 -h,--printblockheaders   Print block headers for each block.
 -k,--checkrow            Enable row order check; looks for out-of-order
                          keys
 -m,--printmeta           Print meta data of file
 -p,--printkv             Print key/value pairs
 -r,--region <arg>        Region to scan. Pass region name; e.g.
                          'hbase:meta,,1'
 -s,--stats               Print statistics
 -v,--verbose             Verbose output; emits file and meta data
                          delimiters
 -w,--seekToRow <arg>     Seek to this row and print all the kvs for this
                          row only

hbase hfile [option] pathInHDFS

示例:查看test:standard表property列族下的hfile信息

]# hbase hfile -v -m -f /hbase/data/test/standard/ee88aa5866895a0e95577ae41f63d74e/property/3f6285a2acbb4acebeed26dd7282067e

....

Scanning -> /hbase/data/zqykj/standard_model_entity/ee88aa5866895a0e95577ae41f63d74e/property/3f6285a2acbb4acebeed26dd7282067e
2019-03-25 17:25:58,063 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2019-03-25 17:25:58,515 INFO  [main] compress.CodecPool: Got brand-new decompressor [.snappy]
Block index size as per heapsize: 944
reader=/hbase/data/zqykj/standard_model_entity/ee88aa5866895a0e95577ae41f63d74e/property/3f6285a2acbb4acebeed26dd7282067e,
    compression=snappy,
    cacheConf=CacheConfig:disabled,
    firstKey=666667cc38d25b0b/property:entity_id/1553166924854/Put,
    lastKey=7fffff0cc74f87d/property:user_telphone/1553166927983/Put,
    avgKeyLen=48,
    avgValueLen=14,
    entries=13251766,
    length=220263815
Trailer:
    fileinfoOffset=220258881,
    loadOnOpenDataOffset=220258502,
    dataIndexCount=7,
    metaIndexCount=0,
    totalUncomressedBytes=968894120,
    entryCount=13251766,
    compressionCodec=SNAPPY,
    uncompressedDataIndexSize=818208,
    numDataIndexLevels=2,
    firstDataBlockOffset=0,
    lastDataBlockOffset=220164473,
    comparatorClassName=org.apache.hadoop.hbase.KeyValue$KeyComparator,
    encryptionKey=NONE,
    majorVersion=3,
    minorVersion=0
Fileinfo:
    BLOOM_FILTER_TYPE = ROW
    DELETE_FAMILY_COUNT = \x00\x00\x00\x00\x00\x00\x00\x00
    EARLIEST_PUT_TS = \x00\x00\x01i\x7F\xD8\x98\x00
    KEY_VALUE_VERSION = \x00\x00\x00\x01
    LAST_BLOOM_KEY = 7fffff0cc74f87d
    MAJOR_COMPACTION_KEY = \xFF
    MAX_MEMSTORE_TS_KEY = \x00\x00\x00\x00\x00\x00\x00\xBA
    MAX_SEQ_ID_KEY = 186
    TIMERANGE = 1552628094976....1553166927983
    hfile.AVG_KEY_LEN = 48
    hfile.AVG_VALUE_LEN = 14
    hfile.CREATE_TIME_TS = \x00\x00\x01i\xA0\xB8Xn
    hfile.LASTKEY = \x00\x0F7fffff0cc74f87d\x08propertyuser_telphone\x00\x00\x01i\x9F\xF6\x88o\x04
Mid-key: \x00\x10733522795fb9dd40\x08propertyperson_h\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\x00\x00\x00\x00\x06\x8Ey\x0F\x00\x008H
Bloom filter:
    BloomSize: 1507328
    No of Keys in bloom: 1241321
    Max Keys for bloom: 1257019
    Percentage filled: 99%
    Number of chunks: 12
    Comparator: RawBytesComparator
Delete Family Bloom filter:
    Not present
Scanned kv count -> 13251766