先做个记录,监控数据量过大时可以设置表的数据过期时间来清理数据。
1. 查找本地数据表大小
[root@ZWZF-CWY-LZY-12 ~]# cd /home/pinpoint/hbase/data/default/
[root@ZWZF-CWY-LZY-12 default]# du -sh *
16K AgentEvent
64K AgentInfo
16K AgentLifeCycle
72K AgentStat
632K AgentStatV2
40K ApiMetaData
16K ApplicationIndex
72K ApplicationMapStatisticsCallee_Ver2
72K ApplicationMapStatisticsCaller_Ver2
80K ApplicationMapStatisticsSelf_Ver2
72K ApplicationTraceIndex
20K HostApplicationMap_Ver2
72K SqlMetaData_Ver2
40K StringMetaData
264K Traces
1.1M TraceV2
2. 设置表的数据过期时间
2.1 查看hbase表:list
[root@monitor default]# cd /home/pinpoint/hbase-1.7.1/bin/
[root@monitor bin]# ./hbase shell
2022-10-19 15:03:48,097 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
Version 1.7.1, r2d9273667e418e7023f9104a830cdcb8233b6f25, Fri Jul 16 00:20:26 PDT 2021
hbase(main):001:0>list
TABLE
AgentEvent
AgentInfo
AgentLifeCycle
AgentStatV2
ApiMetaData
ApplicationIndex
ApplicationMapStatisticsCallee_Ver2
ApplicationMapStatisticsCaller_Ver2
ApplicationMapStatisticsSelf_Ver2
ApplicationStatAggre
ApplicationTraceIndex
HostApplicationMap_Ver2
SqlMetaData_Ver2
StringMetaData
TraceV2
15 row(s) in 0.1740 seconds
=> ["AgentEvent", "AgentInfo", "AgentLifeCycle", "AgentStatV2", "ApiMetaData", "ApplicationIndex", "ApplicationMapStatisticsCallee_Ver2", "ApplicationMapStatisticsCaller_Ver2", "ApplicationMapStatisticsSelf_Ver2", "ApplicationStatAggre", "ApplicationTraceIndex", "HostApplicationMap_Ver2", "SqlMetaData_Ver2", "StringMetaData", "TraceV2"]
2.2 查看表描述:desc 'table_name'
hbase(main):002:0> desc 'AgentInfo'
Table AgentInfo is ENABLED
AgentInfo
COLUMN FAMILIES DESCRIPTION
{NAME => 'Info', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'PREFIX', TTL
=> '31536000 SECONDS (365 DAYS)', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '
0'}
1 row(s) in 0.1760 seconds
2.3 修改表信息
将AgentInfo表的过期时间(TTL)修改为180天(TTL单位:秒)。
2.3.1 停用表:disable 'table_name'
hbase(main):003:0> disable 'AgentInfo'
0 row(s) in 2.3130 seconds
2.3.2 修改表信息:alter 'table_name' , {NAME=>'xxx',TTL=>'xxx'}
hbase(main):004:0> alter 'AgentInfo', {NAME=>'Info',TTL=>'15552000'}
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 1.9670 seconds
2.3.3 启用表:enable 'table_name'
hbase(main):005:0> enable 'AgentInfo'
0 row(s) in 1.3030 seconds
2.3.4 查看表信息(确认修改结果)
hbase(main):006:0> desc 'AgentInfo'
Table AgentInfo is DISABLED
AgentInfo
COLUMN FAMILIES DESCRIPTION
{NAME => 'Info', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'PREFIX', TTL
=> '15552000 SECONDS (180 DAYS)', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '
0'}
1 row(s) in 0.0310 seconds
3. 修改表过期时间后,清除过期数据:major_compact 'table_name'
定期运行一次Major Compact(通过crontab 任务),用来删除早于给定TTL值的所有数据。
注意:如果只设置TTL而不定期运行 major_compact,只是标记删除,并不会真正从磁盘上删除。
hbase(main):007:0> major_compact 'AgentInfo'
0 row(s) in 0.4290 seconds