Phoenix查询抛InsufficientMemoryException异常

测试集群中使用Phoenix进行数据查询抛出如下异常:

org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException: C_PICRECORD,,1463484729784.fccf0ba342b4fe6660edb4081ac498ad.: Requested memory of 142000000 bytes could not be allocated from remaining memory of 710007810 bytes from global pool of 781752729 bytes after waiting for 10000ms.
    at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:84)
    at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:52)
    at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:205)
    at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$52.call(RegionCoprocessorHost.java:1308)
    at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1663)
    at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1738)
    at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperationWithResult(RegionCoprocessorHost.java:1702)
    at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1303)
    at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2119)
    at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
    at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
    at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
    at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
    at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.phoenix.memory.InsufficientMemoryException: Requested memory of 142000000 bytes could not be allocated from remaining memory of 710007810 bytes from global pool of 781752729 bytes after waiting for 10000ms.
    at org.apache.phoenix.memory.GlobalMemoryManager.allocateBytes(GlobalMemoryManager.java:78)
    at org.apache.phoenix.memory.GlobalMemoryManager.allocate(GlobalMemoryManager.java:97)
    at org.apache.phoenix.memory.GlobalMemoryManager.allocate(GlobalMemoryManager.java:103)
    at org.apache.phoenix.coprocessor.ScanRegionObserver.getTopNScanner(ScanRegionObserver.java:233)
    at org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:220)
    at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:178)
    ... 12 more
; 

数据量千万级别(列数为44),构建了二级索引表,确保查询能够走二级索引表,查询语句为:

SELECT
    ID,
    WAY_NUM,
    CAR_NUM,
    ORG_NAME,
    ORG_ID,
    CAR_SPEED,
    CAR_NUM_TYPE,
    CAR_NUM_COLOR,
    CAR_TYPE,
    CAR_COLOR,
    CAR_LOGO_TYPE,
    LONGITUDE,
    LATITUDE,
    CAR_IMG_URL,
    CAP_DATE,
    GROUPID,
    DEV_CHN_NAME
FROM
    C_PICRECORD
WHERE
    CAP_DATE >= '2016-05-05 11:14:40'
AND CAP_DATE <= '2016-05-12 11:14:40'
AND CAR_NUM = '粤VFX350'
AND CAP_TYPE=0
ORDER BY CAP_DATE ASC
LIMIT 100000

报错原因为使用了ORDER BY。

google论坛显示如下解决方案:
https://groups.google.com/forum/#!topic/phoenix-hbase-user/KEiDCLVXf6k

phoenix.query.maxGlobalMemoryPercentage 调整为40%

<property>
    <name>phoenix.query.maxGlobalMemoryPercentage</name>
    <value>40</value>
</property>

【注意】 必须在hbase集群的regionserver hbase-site.xml里面配置该项并重启hbase集群

经过调整后暂时没出现这个异常。

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
Phoenix查询语法可以通过JDBC来执行。在Phoenix中,可以使用标准的SQL语法来查询数据。以下是一些常用的Phoenix查询语法: 1. 查询所有数据: SELECT * FROM table_name; 2. 查询指定列的数据: SELECT column1, column2 FROM table_name; 3. 使用WHERE子句进行条件查询: SELECT * FROM table_name WHERE condition; 4. 使用ORDER BY子句对结果进行排序: SELECT * FROM table_name ORDER BY column_name; 5. 使用LIMIT子句限制返回的结果数量: SELECT * FROM table_name LIMIT number; 6. 使用GROUP BY子句进行分组查询: SELECT column1, COUNT(column2) FROM table_name GROUP BY column1; 7. 使用JOIN子句进行表连接查询: SELECT * FROM table1 JOIN table2 ON table1.column = table2.column; 8. 使用LIKE子句进行模糊查询: SELECT * FROM table_name WHERE column_name LIKE 'keyword'; 需要注意的是,在Phoenix中,表名、列族和列名需要使用双引号括起来,以保持大小写的一致性。此外,Phoenix还支持更多高级的查询语法和函数,可以根据具体需求进行深入学习和使用。\[2\]\[3\] #### 引用[.reference_title] - *1* *2* [Phoenix(HBase) 操作语法](https://blog.csdn.net/hell_oword/article/details/119350442)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v91^control_2,239^v3^insert_chatgpt"}} ] [.reference_item] - *3* [Phoenix命令及语法](https://blog.csdn.net/JacksonKing/article/details/106055095)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v91^control_2,239^v3^insert_chatgpt"}} ] [.reference_item] [ .reference_list ]
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值