应用向HBASE写入慢

向上走

于 2023-03-29 15:28:28 发布

阅读量146

点赞数

文章标签： hbase hadoop java

本文链接：https://blog.csdn.net/songlai1986/article/details/129837776

版权

HBASE为三个节点的集群。每秒约有3至6千条数据进行写入。在多应用同时向同一个表进行写入时，会报如下错误

2022/09/29-10:12:51 >>   INFO >> org.apache.hadoop.hbase.client.AsyncProcess.logAndResubmit(AsyncProcess.java:673) >> #6871,
 table=table_20220929, attempt=10/35 failed 511 ops, last exception: 
 org.apache.hadoop.hbase.RegionTooBusyException: org.apache.hadoop.hbase.RegionTooBusyException: Above memstore limit, 
 regionName=table_20220929,ESB_LOG_1538186011510_00123,1538186845003.256169b356626a2da50ed2755d2cf166., server=monapp,60020,1533298812486,
  memstoreSize=268610336, blockingMemStoreSize=268435456
        at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:2546)
        at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:1948)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4043)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3354)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3258)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26935)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2175)
        at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1879)
 on monapp,60020,1533298812486, tracking started Sat Sep 29 10:12:22 CST 2022, retrying after 10017 ms, replay 511 ops.

其实就是行成了热块，无法有序进行写入。在HBASE集群中任意节点重新注册下表，进行数据重分布。

在处理hbase之前，可先对应用进行重启。进入hbase命令行

hbase shell

执行：报错的表名是table_20220929

alter 'table_20220929', {MEMSTORE_FLUSHSIZE => '268435456'};

完成后退出hbase

exit

退出时显示内容

alter 'table_20220929',{MEMSTORE_FLUSHSIZE=>'268435456'};
Updating all regions with the new schema...
0/69 regions updated.
1/69 regions updated.
3/69 regions updated.
6/69 regions updated.
10/69 regions updated.
13/69 regions updated.
17/69 regions updated.
20/69 regions updated.
20/69 regions updated.
21/69 regions updated.
25/69 regions updated.
28/69 regions updated.
28/69 regions updated.
29/69 regions updated.
32/69 regions updated.
35/69 regions updated.
36/69 regions updated.
39/69 regions updated.
42/69 regions updated.
47/69 regions updated.
53/69 regions updated.
58/69 regions updated.
61/69 regions updated.
61/69 regions updated.
64/69 regions updated.
67/69 regions updated.
68/69 regions updated.
68/69 regions updated.
68/69 regions updated.
68/69 regions updated.
68/69 regions updated.
68/69 regions updated.
68/69 regions updated.
68/69 regions updated.
68/69 regions updated.
68/69 regions updated.
68/69 regions updated.
68/69 regions updated.
69/69 regions updated.
Done.
0 row(s) in 42.6190 seconds