Impala: Reducing query concurrency or configuring admission control may help avoid this error

关键字

• Impala
• 并发
• 内存
• 报错

问题描述

在进行impala查询时，抛出错误如下

ExecQueryFInstances rpc query_id=834132baf107d21a:e64f75ab00000000 failed: Failed to get minimum memory reservation of 137.00 MB on daemon cdh1:22000 for query 834132baf107d21a:e64f75ab00000000 due to following error: Failed to increase reservation by 137.00 MB because it would exceed the applicable reservation limit for the “Process” ReservationTracker: reservation_limit=406.30 MB reservation=274.00 MB used_reservation=0 child_reservations=274.00 MB The top 5 queries that allocated memory under this tracker are: Query(bd4eefab92013517:14a5ad300000000): Reservation=137.00 MB ReservationLimit=382.40 MB OtherMemory=3.43 MB Total=140.43 MB Peak=140.43 MB Query(714067a7cd0fe643:36e307d00000000): Reservation=137.00 MB ReservationLimit=382.40 MB OtherMemory=3.08 MB Total=140.08 MB Peak=140.43 MB Query(834132baf107d21a:e64f75ab00000000): Reservation=0 ReservationLimit=382.40 MB OtherMemory=0 Total=0 Peak=0 Memory is likely oversubscribed. Reducing query concurrency or configuring admission control may help avoid this error. ;
nested exception is java.sql.SQLException: ExecQueryFInstances rpc query_id=834132baf107d21a:e64f75ab00000000 failed: Failed to get minimum memory reservation of 137.00 MB on daemon cdh1:22000 for query 834132baf107d21a:e64f75ab00000000 due to following error: Failed to increase reservation by 137.00 MB because it would exceed the applicable reservation limit for the “Process” ReservationTracker: reservation_limit=406.30 MB reservation=274.00 MB used_reservation=0 child_reservations=274.00 MB The top 5 queries that allocated memory under this tracker are: Query(bd4eefab92013517:14a5ad300000000): Reservation=137.00 MB ReservationLimit=382.40 MB OtherMemory=3.43 MB Total=140.43 MB Peak=140.43 MB Query(714067a7cd0fe643:36e307d00000000): Reservation=137.00 MB ReservationLimit=382.40 MB OtherMemory=3.08 MB Total=140.08 MB Peak=140.43 MB Query(834132baf107d21a:e64f75ab00000000): Reservation=0 ReservationLimit=382.40 MB OtherMemory=0 Total=0 Peak=0 Memory is likely oversubscribed.
Reducing query concurrency or configuring admission control may help avoid this error.

解决办法

• 从报错中可以看出是由于并发查询所导致的资源不足，因此，需要先看下impala的配置，找到相关配置。
• 于是找到了如下配置，果然，内存限制太小导致的，只有512M，远远不够任务所需要的，调大此配置。