带你走进Impala query profile第一篇

AllenGd

已于 2022-08-25 13:28:10 修改

阅读量1.8k

点赞数 2

分类专栏： KUDU 文章标签： kudu

于 2020-10-13 16:55:22 首次发布

本文链接：https://blog.csdn.net/Allenzyg/article/details/107105335

版权

KUDU 专栏收录该内容

10 篇文章 5 订阅

订阅专栏

我为什么会写Impala query profile系列文章呢？

一方面：

在日日顺三期项目上，在跟客户做报表时，发现执行一个sql很慢，只select count(*) 就要花费15分钟左右，如果加上逻辑处理，关联配置表，速度会更慢。

从HUE配置调度，执行时间跑了2小时10分钟13秒，最后我把这个进程kill掉了，我打算一探究竟。

首先，查看一下我处理的这些数据量级：

[rrs-hdp-dn01:25003] > select count(*) from bms_ldgdb.ldg_690marketization_ledger t where check_flag in (4,-1) and ledger_status = 2 and is_delete = 0 and module in (1,2) and account_date >= '2020-06-01' and industry_code != '';

Query: select count(*) from bms_ldgdb.ldg_690marketization_ledger t where check_flag in (4,-1) and ledger_status = 2 and is_delete = 0 and module in (1,2) and account_date >= '2020-06-01' and industry_code != ''

Query submitted at: 2020-07-03 11:58:47 (Coordinator: http://rrs-hdp-dn04:25000)

Query progress can be monitored at: http://rrs-hdp-dn04:25000/query_plan?query_id=2945a77ff619defe:b658730000000000

+----------+

| count(*) |

+----------+

| 6100865 |

+----------+

Fetched 1 row(s) in 848.91s

从查询结果来看，数据了并不算大，610万，用时14分钟8秒。那是什么导致查询这么慢呢？查看一下执行计划。

另一方面：

很多Impala用户不知道如何阅读Impala query profile来了解一个查询背后正在执行的操作，从而在此基础上对查询进行调优以充分发挥查询的性能。因此我想写一篇简单的文章来分享我的经验，并希望它可以对希望了解更多信息的人有所帮助。

这是本系列的第1篇，我将介绍一些Impala query profile的基础知识和查看Profile时特别要注意的内容。

获取Impala query profile

首先，获取Impala query profile有两种方法，最简单的方法是在impala-shell中运行查询后执行“PROFILE”语句，如下所示：后面2篇会详细介绍这个profile

具体内容：

[ip:25003] > profile;

Query Runtime Profile:

Query (id=2945a77ff619defe:b658730000000000):

Summary:

Session ID: 24abae22c723db5:2c0a25a81814e8a8

Session Type: BEESWAX

Start Time: 2020-07-03 11:58:47.317039000

End Time: 2020-07-03 12:12:56.216137000

Query Type: QUERY

Query State: FINISHED

Query Status: OK

Impala Version: impalad version 2.10.0-cdh5.13.3 RELEASE (build 15a453e15865344e75ce0fc6c4c760696d50f626)

User: root

Connected User: root

Delegated User:

Network Address: 10.138.232.87:36237

Default Db: default

Sql Statement: select count(*) from bms_ldgdb.ldg_690marketization_ledger t where check_flag in (4,-1) and ledger_status = 2 and is_delete = 0 and module in (1,2) and account_date >= '2020-06-01' and industry_code != ''

Coordinator: rrs-hdp-dn04:22000

Query Options (set by configuration):

Query Options (set by configuration and planner): MT_DOP=0

Plan:

----------------

Max Per-Host Resource Reservation: Memory=0B

Per-Host Resource Estimates: Memory=20.00MB

WARNING: The following tables are missing relevant table and/or column statistics.

bms_ldgdb.ldg_690marketization_ledger

F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1

| Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B

PLAN-ROOT SINK

| mem-estimate=0B mem-reservation=0B

03:AGGREGATE [FINALIZE]

| output: count:merge(*)

| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB

| tuple-ids=1 row-size=8B cardinality=1

02:EXCHANGE [UNPARTITIONED]

| mem-estimate=0B mem-reservation=0B

| tuple-ids=1 row-size=8B cardinality=1

F00:PLAN FRAGMENT [RANDOM] hosts=16 instances=16

Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B

01:AGGREGATE

| output: count(*)

| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB

| tuple-ids=1 row-size=8B cardinality=1

00:SCAN KUDU [bms_ldgdb.ldg_690marketization_ledger t]

predicates: industry_code != ''

kudu predicates: is_delete = 0, ledger_status = 2, check_flag IN (4, -1), module IN (1, 2), account_date >= '2020-06-01'

mem-estimate=0B mem-reservation=0B

tuple-ids=0 row-size=15B cardinality=unavailable

----------------

Estimated Per-Host Mem: 20971520

Tables Missing Stats: bms_ldgdb.ldg_690marketization_ledger

Per Host Min Reservation: rrs-hdp-dn03:22000(0) rrs-hdp-dn04:22000(0) rrs-hdp-dn05:22000(0) rrs-hdp-dn08:22000(0) rrs-hdp-dn11:22000(0) rrs-hdp-dn13:22000(0)

Request Pool: root.default

Admission result: Admitted immediately

ExecSummary:

Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail

----------------------------------------------------------------------------------------------------------------------------------

03:AGGREGATE 1 316.053us 316.053us 1 1 40.00 KB 10.00 MB FINALIZE

02:EXCHANGE 1 14m8s 14m8s 6 1 0 0 UNPARTITIONED

01:AGGREGATE 6 2.746ms 5.205ms 6 1 67.00 KB 10.00 MB

00:SCAN KUDU 6 2m33s 14m8s 6.10M -1 299.00 KB 0 bms_ldgdb.ldg_690marketization_ledger t

Errors:

Planner Timeline: 21.773ms

- Analysis finished: 1.432ms (1.432ms)

- Equivalence classes computed: 1.478ms (46.487us)

- Single node plan created: 21.005ms (19.526ms)

- Runtime filters computed: 21.020ms (14.599us)

- Distributed plan created: 21.096ms (76.103us)

- Lineage info computed: 21.156ms (59.924us)

- Planning finished: 21.773ms (617.260us)

Query Timeline: 14m8s

- Query submitted: 250.799us (250.799us)

- Planning finished: 24.359ms (24.108ms)

- Submit for admission: 25.375ms (1.016ms)

- Completed admission: 25.565ms (189.521us)

- Ready to start on 6 backends: 26.997ms (1.432ms)

- All 6 execution backends (7 fragment instances) started: 33.022ms (6.024ms)

- Rows available: 14m8s (14m8s)

- First row fetched: 14m8s (695.112ms)

- Unregister query: 14m8s (5.330ms)

- ComputeScanRangeAssignmentTimer: 110.741us

ImpalaServer:

- ClientFetchWaitTimer: 697.276ms

- RowMaterializationTimer: 3.129ms

Execution Profile 2945a77ff619defe:b658730000000000:(Total: 14m8s, non-child: 0.000ns, % non-child: 0.00%)

Number of filters: 0

Filter routing table:

ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled

-------------------------------------------------------------------------------------------------------------------

Backend startup latencies: Count: 6, min / max: 3ms / 5ms, 25th %-ile: 3ms, 50th %-ile: 3ms, 75th %-ile: 4ms, 90th %-ile: 4ms, 95th %-ile: 5ms, 99.9th %-ile: 5ms

Per Node Peak Memory Usage: rrs-hdp-dn03:22000(162.18 KB) rrs-hdp-dn04:22000(383.19 KB) rrs-hdp-dn05:22000(213.18 KB) rrs-hdp-dn08:22000(213.18 KB) rrs-hdp-dn11:22000(213.18 KB) rrs-hdp-dn13:22000(213.18 KB)

- FiltersReceived: 0 (0)

- FinalizationTimer: 0.000ns

Averaged Fragment F01:(Total: 14m8s, non-child: 696.183ms, % non-child: 0.08%)

split sizes: min: 0, max: 0, avg: 0, stddev: 0

completion times: min:14m8s max:14m8s mean: 14m8s stddev:0.000ns

execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec

num instances: 1

- AverageThreadTokens: 0.00

- BloomFilterBytes: 0

- PeakMemoryUsage: 48.02 KB (49168)

- PeakReservation: 0

- PeakUsedReservation: 0

- PerHostPeakMemUsage: 383.19 KB (392387)

- RowsProduced: 1 (1)

- TotalNetworkReceiveTime: 14m8s

- TotalNetworkSendTime: 0.000ns

- TotalStorageWaitTime: 0.000ns

- TotalThreadsInvoluntaryContextSwitches: 0 (0)

- TotalThreadsTotalWallClockTime: 14m8s

- TotalThreadsSysTime: 1.000ms

- TotalThreadsUserTime: 0.000ns

- TotalThreadsVoluntaryContextSwitches: 8 (8)

Fragment Instance Lifecycle Timings:

- ExecTime: 695.216ms

- ExecTreeExecTime: 8.846us

- OpenTime: 14m8s

- ExecTreeOpenTime: 14m8s

- PrepareTime: 40.847ms

- ExecTreePrepareTime: 145.192us

PLAN_ROOT_SINK:

- PeakMemoryUsage: 0

CodeGen:(Total: 39.850ms, non-child: 39.850ms, % non-child: 100.00%)

- CodegenTime: 0.000ns

- CompileTime: 0.000ns

- LoadTime: 0.000ns

- ModuleBitcodeSize: 1.95 MB (2041444)

- NumFunctions: 0 (0)

- NumInstructions: 0 (0)

- OptimizationTime: 0.000ns

- PeakMemoryUsage: 0

- PrepareTime: 38.755ms

AGGREGATION_NODE (id=3):(Total: 14m8s, non-child: 316.053us, % non-child: 0.00%)

- BuildTime: 15.909us

- GetResultsTime: 0.000ns

- HTResizeTime: 0.000ns

- HashBuckets: 0 (0)

- LargestPartitionPercent: 0 (0)

- MaxPartitionLevel: 0 (0)

- NumRepartitions: 0 (0)

- PartitionsCreated: 0 (0)

- PeakMemoryUsage: 40.00 KB (40960)

- RowsRepartitioned: 0 (0)

- RowsReturned: 1 (1)

- RowsReturnedRate: 0

- SpilledPartitions: 0 (0)

EXCHANGE_NODE (id=2):(Total: 14m8s, non-child: 14m8s, % non-child: 100.00%)

- ConvertRowBatchTime: 24.499us

- PeakMemoryUsage: 0

- RowsReturned: 6 (6)

- RowsReturnedRate: 0

DataStreamReceiver:

- BytesReceived: 96.00 B (96)

- DeserializeRowBatchTimer: 146.930us

- FirstBatchArrivalWaitTime: 5s982ms

- PeakMemoryUsage: 4.02 KB (4112)

- SendersBlockedTimer: 0.000ns

- SendersBlockedTotalTimer(*): 0.000ns

Coordinator Fragment F01:

Instance 2945a77ff619defe:b658730000000000 (host=rrs-hdp-dn04:22000):(Total: 14m8s, non-child: 696.183ms, % non-child: 0.08%)

MemoryUsage(16s000ms): 26.75 KB, 40.01 KB, 42.13 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB, 44.01 KB

- AverageThreadTokens: 0.00

- BloomFilterBytes: 0

- PeakMemoryUsage: 48.02 KB (49168)

- PeakReservation: 0

- PeakUsedReservation: 0

- PerHostPeakMemUsage: 383.19 KB (392387)

- RowsProduced: 1 (1)

- TotalNetworkReceiveTime: 14m8s

- TotalNetworkSendTime: 0.000ns

- TotalStorageWaitTime: 0.000ns

- TotalThreadsInvoluntaryContextSwitches: 0 (0)

- TotalThreadsTotalWallClockTime: 14m8s

- TotalThreadsSysTime: 1.000ms

- TotalThreadsUserTime: 0.000ns

- TotalThreadsVoluntaryContextSwitches: 8 (8)

Fragment Instance Lifecycle Timings:

- ExecTime: 695.216ms

- ExecTreeExecTime: 8.846us

- OpenTime: 14m8s

- ExecTreeOpenTime: 14m8s

- PrepareTime: 40.847ms

- ExecTreePrepareTime: 145.192us

PLAN_ROOT_SINK:

- PeakMemoryUsage: 0

CodeGen:(Total: 39.850ms, non-child: 39.850ms, % non-child: 100.00%)

- CodegenTime: 0.000ns

- CompileTime: 0.000ns

- LoadTime: 0.000ns

- ModuleBitcodeSize: 1.95 MB (2041444)

- NumFunctions: 0 (0)

- NumInstructions: 0 (0)

- OptimizationTime: 0.000ns

- PeakMemoryUsage: 0

- PrepareTime: 38.755ms

AGGREGATION_NODE (id=3):(Total: 14m8s, non-child: 316.053us, % non-child: 0.00%)

- BuildTime: 15.909us

- GetResultsTime: 0.000ns

- HTResizeTime: 0.000ns

- HashBuckets: 0 (0)

- LargestPartitionPercent: 0 (0)

- MaxPartitionLevel: 0 (0)

- NumRepartitions: 0 (0)

- PartitionsCreated: 0 (0)

- PeakMemoryUsage: 40.00 KB (40960)

- RowsRepartitioned: 0 (0)

- RowsReturned: 1 (1)

- RowsReturnedRate: 0

- SpilledPartitions: 0 (0)

EXCHANGE_NODE (id=2):(Total: 14m8s, non-child: 14m8s, % non-child: 100.00%)

- ConvertRowBatchTime: 24.499us

- PeakMemoryUsage: 0

- RowsReturned: 6 (6)

- RowsReturnedRate: 0

DataStreamReceiver:

BytesReceived(16s000ms): 29.00 B, 64.00 B, 72.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B, 80.00 B

- BytesReceived: 96.00 B (96)

- DeserializeRowBatchTimer: 146.930us

- FirstBatchArrivalWaitTime: 5s982ms

- PeakMemoryUsage: 4.02 KB (4112)

- SendersBlockedTimer: 0.000ns

- SendersBlockedTotalTimer(*): 0.000ns

Averaged Fragment F00:(Total: 2m33s, non-child: 2.541ms, % non-child: 0.00%)

split sizes: min: 0, max: 0, avg: 0, stddev: 0

completion times: min:6s025ms max:14m8s mean: 2m33s stddev:5m10s

execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec

num instances: 6

- AverageThreadTokens: 1.98

- BloomFilterBytes: 0

- PeakMemoryUsage: 225.68 KB (231099)

- PeakReservation: 0

- PeakUsedReservation: 0

- PerHostPeakMemUsage: 233.02 KB (238609)

- RowsProduced: 1 (1)

- TotalNetworkReceiveTime: 0.000ns

- TotalNetworkSendTime: 635.297us

- TotalStorageWaitTime: 2m33s

- TotalThreadsInvoluntaryContextSwitches: 1 (1)

- TotalThreadsTotalWallClockTime: 5m7s

- TotalThreadsSysTime: 22.496ms

- TotalThreadsUserTime: 189.971ms

- TotalThreadsVoluntaryContextSwitches: 1.31K (1311)

Fragment Instance Lifecycle Timings:

- ExecTime: 1.013ms

- ExecTreeExecTime: 7.958us

- OpenTime: 2m33s

- ExecTreeOpenTime: 2m33s

- PrepareTime: 41.091ms

- ExecTreePrepareTime: 133.511us

DataStreamSender (dst_id=2):(Total: 461.521us, non-child: 461.521us, % non-child: 100.00%)

- BytesSent: 16.00 B (16)

- NetworkThroughput(*): 265.91 KB/sec

- OverallThroughput: 34.96 KB/sec

- PeakMemoryUsage: 16.00 KB (16384)

- RowsReturned: 1 (1)

- SerializeBatchTime: 15.124us

- TransmitDataRPCTime: 63.344us

- UncompressedRowBatchSize: 16.00 B (16)

CodeGen:(Total: 68.504ms, non-child: 68.504ms, % non-child: 100.00%)

- CodegenTime: 693.533us

- CompileTime: 6.554ms

- LoadTime: 0.000ns

- ModuleBitcodeSize: 1.95 MB (2041444)

- NumFunctions: 17 (17)

- NumInstructions: 223 (223)

- OptimizationTime: 21.630ms

- PeakMemoryUsage: 111.50 KB (114176)

- PrepareTime: 39.283ms

AGGREGATION_NODE (id=1):(Total: 2m33s, non-child: 2.746ms, % non-child: 0.00%)

- BuildTime: 2.726ms

- GetResultsTime: 0.000ns

- HTResizeTime: 0.000ns

- HashBuckets: 0 (0)

- LargestPartitionPercent: 0 (0)

- MaxPartitionLevel: 0 (0)

- NumRepartitions: 0 (0)

- PartitionsCreated: 0 (0)

- PeakMemoryUsage: 67.00 KB (68608)

- RowsRepartitioned: 0 (0)

- RowsReturned: 1 (1)

- RowsReturnedRate: 0

- SpilledPartitions: 0 (0)

KUDU_SCAN_NODE (id=0):(Total: 2m33s, non-child: 2m33s, % non-child: 100.00%)

- BytesRead: 0

- KuduRemoteScanTokens: 0 (0)

- NumScannerThreadsStarted: 1 (1)

- PeakMemoryUsage: 151.50 KB (155136)

- RowsRead: 1.02M (1016810)

- RowsReturned: 1.02M (1016810)

- RowsReturnedRate: 88.06 K/sec

- ScanRangesComplete: 1 (1)

- ScannerThreadsInvoluntaryContextSwitches: 1 (1)

- ScannerThreadsTotalWallClockTime: 2m33s

- MaterializeTupleTime(*): 2m33s

- ScannerThreadsSysTime: 17.996ms

- ScannerThreadsUserTime: 145.810ms

- ScannerThreadsVoluntaryContextSwitches: 315 (315)

- TotalKuduScanRoundTrips: 293 (293)

- TotalReadThroughput: 0.00 /sec

Fragment F00:

Instance 2945a77ff619defe:b658730000000005 (host=rrs-hdp-dn04:22000):(Total: 14m8s, non-child: 2.048ms, % non-child: 0.00%)

Hdfs split stats (<volume id>:<# splits>/<split lengths>):

MemoryUsage(16s000ms): 83.61 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB

ThreadUsage(16s000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2

- AverageThreadTokens: 2.00

- BloomFilterBytes: 0

- PeakMemoryUsage: 339.18 KB (347323)

- PeakReservation: 0

- PeakUsedReservation: 0

- PerHostPeakMemUsage: 383.19 KB (392387)

- RowsProduced: 1 (1)

- TotalNetworkReceiveTime: 0.000ns

- TotalNetworkSendTime: 262.704us

- TotalStorageWaitTime: 14m7s

- TotalThreadsInvoluntaryContextSwitches: 1 (1)

- TotalThreadsTotalWallClockTime: 28m16s

- TotalThreadsSysTime: 55.991ms

- TotalThreadsUserTime: 364.945ms

- TotalThreadsVoluntaryContextSwitches: 2.66K (2658)

Fragment Instance Lifecycle Timings:

- ExecTime: 532.246us

- ExecTreeExecTime: 7.588us

- OpenTime: 14m8s

- ExecTreeOpenTime: 14m8s

- PrepareTime: 43.153ms

- ExecTreePrepareTime: 147.736us

DataStreamSender (dst_id=2):(Total: 492.978us, non-child: 492.978us, % non-child: 100.00%)

- BytesSent: 16.00 B (16)

- NetworkThroughput(*): 225.75 KB/sec

- OverallThroughput: 31.69 KB/sec

- PeakMemoryUsage: 16.00 KB (16384)

- RowsReturned: 1 (1)

- SerializeBatchTime: 13.202us

- TransmitDataRPCTime: 69.212us

- UncompressedRowBatchSize: 16.00 B (16)

CodeGen:(Total: 70.631ms, non-child: 70.631ms, % non-child: 100.00%)

- CodegenTime: 639.905us

- CompileTime: 6.523ms

- LoadTime: 0.000ns

- ModuleBitcodeSize: 1.95 MB (2041444)

- NumFunctions: 17 (17)

- NumInstructions: 223 (223)

- OptimizationTime: 21.608ms

- PeakMemoryUsage: 111.50 KB (114176)

- PrepareTime: 41.281ms

AGGREGATION_NODE (id=1):(Total: 14m8s, non-child: 5.205ms, % non-child: 0.00%)

ExecOption: Codegen Enabled

- BuildTime: 3.851ms

- GetResultsTime: 0.000ns

- HTResizeTime: 0.000ns

- HashBuckets: 0 (0)

- LargestPartitionPercent: 0 (0)

- MaxPartitionLevel: 0 (0)

- NumRepartitions: 0 (0)

- PartitionsCreated: 0 (0)

- PeakMemoryUsage: 67.00 KB (68608)

- RowsRepartitioned: 0 (0)

- RowsReturned: 1 (1)

- RowsReturnedRate: 0

- SpilledPartitions: 0 (0)

KUDU_SCAN_NODE (id=0):(Total: 14m8s, non-child: 14m8s, % non-child: 100.00%)

BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

- BytesRead: 0

- KuduRemoteScanTokens: 0 (0)

- NumScannerThreadsStarted: 1 (1)

- PeakMemoryUsage: 299.00 KB (306176)

- RowsRead: 1.02M (1016239)

- RowsReturned: 1.02M (1016239)

- RowsReturnedRate: 1.20 K/sec

- ScanRangesComplete: 1 (1)

- ScannerThreadsInvoluntaryContextSwitches: 1 (1)

- ScannerThreadsTotalWallClockTime: 14m8s

- MaterializeTupleTime(*): 14m8s

- ScannerThreadsSysTime: 49.992ms

- ScannerThreadsUserTime: 313.952ms

- ScannerThreadsVoluntaryContextSwitches: 1.67K (1669)

- TotalKuduScanRoundTrips: 1.56K (1560)

- TotalReadThroughput: 0.00 /sec

Instance 2945a77ff619defe:b658730000000002 (host=rrs-hdp-dn13:22000):(Total: 39s554ms, non-child: 3.704ms, % non-child: 0.01%)

Hdfs split stats (<volume id>:<# splits>/<split lengths>):

MemoryUsage(1s000ms): 44.18 KB, 73.68 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB

ThreadUsage(1s000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2

- AverageThreadTokens: 2.00

- BloomFilterBytes: 0

- PeakMemoryUsage: 213.18 KB (218299)

- PeakReservation: 0

- PeakUsedReservation: 0

- PerHostPeakMemUsage: 213.18 KB (218299)

- RowsProduced: 1 (1)

- TotalNetworkReceiveTime: 0.000ns

- TotalNetworkSendTime: 1.901ms

- TotalStorageWaitTime: 39s355ms

- TotalThreadsInvoluntaryContextSwitches: 3 (3)

- TotalThreadsTotalWallClockTime: 1m18s

- TotalThreadsSysTime: 10.998ms

- TotalThreadsUserTime: 183.972ms

- TotalThreadsVoluntaryContextSwitches: 1.09K (1094)

Fragment Instance Lifecycle Timings:

- ExecTime: 2.249ms

- ExecTreeExecTime: 10.310us

- OpenTime: 39s511ms

- ExecTreeOpenTime: 39s480ms

- PrepareTime: 41.037ms

- ExecTreePrepareTime: 139.531us

DataStreamSender (dst_id=2):(Total: 317.413us, non-child: 317.413us, % non-child: 100.00%)

- BytesSent: 16.00 B (16)

- NetworkThroughput(*): 179.50 KB/sec

- OverallThroughput: 49.23 KB/sec

- PeakMemoryUsage: 16.00 KB (16384)

- RowsReturned: 1 (1)

- SerializeBatchTime: 11.384us

- TransmitDataRPCTime: 87.047us

- UncompressedRowBatchSize: 16.00 B (16)

CodeGen:(Total: 70.153ms, non-child: 70.153ms, % non-child: 100.00%)

- CodegenTime: 761.527us

- CompileTime: 7.312ms

- LoadTime: 0.000ns

- ModuleBitcodeSize: 1.95 MB (2041444)

- NumFunctions: 17 (17)

- NumInstructions: 223 (223)

- OptimizationTime: 22.364ms

- PeakMemoryUsage: 111.50 KB (114176)

- PrepareTime: 39.409ms

AGGREGATION_NODE (id=1):(Total: 39s480ms, non-child: 2.999ms, % non-child: 0.01%)

ExecOption: Codegen Enabled

- BuildTime: 2.742ms

- GetResultsTime: 0.000ns

- HTResizeTime: 0.000ns

- HashBuckets: 0 (0)

- LargestPartitionPercent: 0 (0)

- MaxPartitionLevel: 0 (0)

- NumRepartitions: 0 (0)

- PartitionsCreated: 0 (0)

- PeakMemoryUsage: 67.00 KB (68608)

- RowsRepartitioned: 0 (0)

- RowsReturned: 1 (1)

- RowsReturnedRate: 0

- SpilledPartitions: 0 (0)

KUDU_SCAN_NODE (id=0):(Total: 39s477ms, non-child: 39s477ms, % non-child: 100.00%)

BytesRead(1s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

- BytesRead: 0

- KuduRemoteScanTokens: 0 (0)

- NumScannerThreadsStarted: 1 (1)

- PeakMemoryUsage: 122.00 KB (124928)

- RowsRead: 1.02M (1016532)

- RowsReturned: 1.02M (1016532)

- RowsReturnedRate: 25.75 K/sec

- ScanRangesComplete: 1 (1)

- ScannerThreadsInvoluntaryContextSwitches: 3 (3)

- ScannerThreadsTotalWallClockTime: 39s478ms

- MaterializeTupleTime(*): 39s474ms

- ScannerThreadsSysTime: 6.998ms

- ScannerThreadsUserTime: 137.979ms

- ScannerThreadsVoluntaryContextSwitches: 100 (100)

- TotalKuduScanRoundTrips: 89 (89)

- TotalReadThroughput: 0.00 /sec

Instance 2945a77ff619defe:b658730000000006 (host=rrs-hdp-dn05:22000):(Total: 10s311ms, non-child: 2.554ms, % non-child: 0.02%)

Hdfs split stats (<volume id>:<# splits>/<split lengths>):

MemoryUsage(500.000ms): 139.50 KB, 44.18 KB, 44.18 KB, 44.18 KB, 69.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB

ThreadUsage(500.000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2

- AverageThreadTokens: 1.95

- BloomFilterBytes: 0

- PeakMemoryUsage: 213.18 KB (218299)

- PeakReservation: 0

- PeakUsedReservation: 0

- PerHostPeakMemUsage: 213.18 KB (218299)

- RowsProduced: 1 (1)

- TotalNetworkReceiveTime: 0.000ns

- TotalNetworkSendTime: 451.701us

- TotalStorageWaitTime: 10s123ms

- TotalThreadsInvoluntaryContextSwitches: 2 (2)

- TotalThreadsTotalWallClockTime: 20s504ms

- TotalThreadsSysTime: 23.996ms

- TotalThreadsUserTime: 154.976ms

- TotalThreadsVoluntaryContextSwitches: 1.03K (1034)

Fragment Instance Lifecycle Timings:

- ExecTime: 760.590us

- ExecTreeExecTime: 8.396us

- OpenTime: 10s268ms

- ExecTreeOpenTime: 10s237ms

- PrepareTime: 42.298ms

- ExecTreePrepareTime: 126.204us

DataStreamSender (dst_id=2):(Total: 576.789us, non-child: 576.789us, % non-child: 100.00%)

- BytesSent: 16.00 B (16)

- NetworkThroughput(*): 202.79 KB/sec

- OverallThroughput: 27.09 KB/sec

- PeakMemoryUsage: 16.00 KB (16384)

- RowsReturned: 1 (1)

- SerializeBatchTime: 18.201us

- TransmitDataRPCTime: 77.049us

- UncompressedRowBatchSize: 16.00 B (16)

CodeGen:(Total: 70.739ms, non-child: 70.739ms, % non-child: 100.00%)

- CodegenTime: 686.682us

- CompileTime: 6.655ms

- LoadTime: 0.000ns

- ModuleBitcodeSize: 1.95 MB (2041444)

- NumFunctions: 17 (17)

- NumInstructions: 223 (223)

- OptimizationTime: 22.799ms

- PeakMemoryUsage: 111.50 KB (114176)

- PrepareTime: 40.260ms

AGGREGATION_NODE (id=1):(Total: 10s237ms, non-child: 2.163ms, % non-child: 0.02%)

ExecOption: Codegen Enabled

- BuildTime: 2.634ms

- GetResultsTime: 0.000ns

- HTResizeTime: 0.000ns

- HashBuckets: 0 (0)

- LargestPartitionPercent: 0 (0)

- MaxPartitionLevel: 0 (0)

- NumRepartitions: 0 (0)

- PartitionsCreated: 0 (0)

- PeakMemoryUsage: 67.00 KB (68608)

- RowsRepartitioned: 0 (0)

- RowsReturned: 1 (1)

- RowsReturnedRate: 0

- SpilledPartitions: 0 (0)

KUDU_SCAN_NODE (id=0):(Total: 10s235ms, non-child: 10s235ms, % non-child: 100.00%)

BytesRead(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

- BytesRead: 0

- KuduRemoteScanTokens: 0 (0)

- NumScannerThreadsStarted: 1 (1)

- PeakMemoryUsage: 122.00 KB (124928)

- RowsRead: 1.02M (1017341)

- RowsReturned: 1.02M (1017341)

- RowsReturnedRate: 99.39 K/sec

- ScanRangesComplete: 1 (1)

- ScannerThreadsInvoluntaryContextSwitches: 2 (2)

- ScannerThreadsTotalWallClockTime: 10s235ms

- MaterializeTupleTime(*): 10s231ms

- ScannerThreadsSysTime: 20.996ms

- ScannerThreadsUserTime: 107.983ms

- ScannerThreadsVoluntaryContextSwitches: 34 (34)

- TotalKuduScanRoundTrips: 31 (31)

- TotalReadThroughput: 0.00 /sec

Instance 2945a77ff619defe:b658730000000001 (host=rrs-hdp-dn03:22000):(Total: 9s700ms, non-child: 2.187ms, % non-child: 0.02%)

Hdfs split stats (<volume id>:<# splits>/<split lengths>):

MemoryUsage(500.000ms): 28.00 KB, 44.18 KB, 44.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB

ThreadUsage(500.000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2

- AverageThreadTokens: 1.95

- BloomFilterBytes: 0

- PeakMemoryUsage: 162.18 KB (166075)

- PeakReservation: 0

- PeakUsedReservation: 0

- PerHostPeakMemUsage: 162.18 KB (166075)

- RowsProduced: 1 (1)

- TotalNetworkReceiveTime: 0.000ns

- TotalNetworkSendTime: 403.863us

- TotalStorageWaitTime: 9s524ms

- TotalThreadsInvoluntaryContextSwitches: 1 (1)

- TotalThreadsTotalWallClockTime: 19s294ms

- TotalThreadsSysTime: 16.997ms

- TotalThreadsUserTime: 147.977ms

- TotalThreadsVoluntaryContextSwitches: 1.03K (1034)

Fragment Instance Lifecycle Timings:

- ExecTime: 834.047us

- ExecTreeExecTime: 7.554us

- OpenTime: 9s660ms

- ExecTreeOpenTime: 9s634ms

- PrepareTime: 38.618ms

- ExecTreePrepareTime: 109.881us

DataStreamSender (dst_id=2):(Total: 457.379us, non-child: 457.379us, % non-child: 100.00%)

- BytesSent: 16.00 B (16)

- NetworkThroughput(*): 260.60 KB/sec

- OverallThroughput: 34.16 KB/sec

- PeakMemoryUsage: 16.00 KB (16384)

- RowsReturned: 1 (1)

- SerializeBatchTime: 17.881us

- TransmitDataRPCTime: 59.957us

- UncompressedRowBatchSize: 16.00 B (16)

CodeGen:(Total: 63.064ms, non-child: 63.064ms, % non-child: 100.00%)

- CodegenTime: 627.375us

- CompileTime: 5.744ms

- LoadTime: 0.000ns

- ModuleBitcodeSize: 1.95 MB (2041444)

- NumFunctions: 17 (17)

- NumInstructions: 223 (223)

- OptimizationTime: 19.537ms

- PeakMemoryUsage: 111.50 KB (114176)

- PrepareTime: 36.839ms

AGGREGATION_NODE (id=1):(Total: 9s634ms, non-child: 1.772ms, % non-child: 0.02%)

ExecOption: Codegen Enabled

- BuildTime: 2.204ms

- GetResultsTime: 0.000ns

- HTResizeTime: 0.000ns

- HashBuckets: 0 (0)

- LargestPartitionPercent: 0 (0)

- MaxPartitionLevel: 0 (0)

- NumRepartitions: 0 (0)

- PartitionsCreated: 0 (0)

- PeakMemoryUsage: 67.00 KB (68608)

- RowsRepartitioned: 0 (0)

- RowsReturned: 1 (1)

- RowsReturnedRate: 0

- SpilledPartitions: 0 (0)

KUDU_SCAN_NODE (id=0):(Total: 9s632ms, non-child: 9s632ms, % non-child: 100.00%)

BytesRead(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

- BytesRead: 0

- KuduRemoteScanTokens: 0 (0)

- NumScannerThreadsStarted: 1 (1)

- PeakMemoryUsage: 122.00 KB (124928)

- RowsRead: 1.02M (1017071)

- RowsReturned: 1.02M (1017071)

- RowsReturnedRate: 105.58 K/sec

- ScanRangesComplete: 1 (1)

- ScannerThreadsInvoluntaryContextSwitches: 1 (1)

- ScannerThreadsTotalWallClockTime: 9s632ms

- MaterializeTupleTime(*): 9s628ms

- ScannerThreadsSysTime: 10.998ms

- ScannerThreadsUserTime: 110.983ms

- ScannerThreadsVoluntaryContextSwitches: 33 (33)

- TotalKuduScanRoundTrips: 31 (31)

- TotalReadThroughput: 0.00 /sec

Instance 2945a77ff619defe:b658730000000004 (host=rrs-hdp-dn11:22000):(Total: 8s165ms, non-child: 2.354ms, % non-child: 0.03%)

Hdfs split stats (<volume id>:<# splits>/<split lengths>):

MemoryUsage(500.000ms): 44.18 KB, 44.18 KB, 44.18 KB, 69.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 154.18 KB, 103.18 KB

ThreadUsage(500.000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2

- AverageThreadTokens: 2.00

- BloomFilterBytes: 0

- PeakMemoryUsage: 213.18 KB (218299)

- PeakReservation: 0

- PeakUsedReservation: 0

- PerHostPeakMemUsage: 213.18 KB (218299)

- RowsProduced: 1 (1)

- TotalNetworkReceiveTime: 0.000ns

- TotalNetworkSendTime: 385.908us

- TotalStorageWaitTime: 7s980ms

- TotalThreadsInvoluntaryContextSwitches: 2 (2)

- TotalThreadsTotalWallClockTime: 16s215ms

- TotalThreadsSysTime: 12.998ms

- TotalThreadsUserTime: 163.975ms

- TotalThreadsVoluntaryContextSwitches: 1.03K (1030)

Fragment Instance Lifecycle Timings:

- ExecTime: 861.953us

- ExecTreeExecTime: 7.334us

- OpenTime: 8s123ms

- ExecTreeOpenTime: 8s093ms

- PrepareTime: 40.960ms

- ExecTreePrepareTime: 145.679us

DataStreamSender (dst_id=2):(Total: 469.050us, non-child: 469.050us, % non-child: 100.00%)

- BytesSent: 16.00 B (16)

- NetworkThroughput(*): 328.30 KB/sec

- OverallThroughput: 33.31 KB/sec

- PeakMemoryUsage: 16.00 KB (16384)

- RowsReturned: 1 (1)

- SerializeBatchTime: 14.939us

- TransmitDataRPCTime: 47.593us

- UncompressedRowBatchSize: 16.00 B (16)

CodeGen:(Total: 69.192ms, non-child: 69.192ms, % non-child: 100.00%)

- CodegenTime: 761.541us

- CompileTime: 6.639ms

- LoadTime: 0.000ns

- ModuleBitcodeSize: 1.95 MB (2041444)

- NumFunctions: 17 (17)

- NumInstructions: 223 (223)

- OptimizationTime: 22.322ms

- PeakMemoryUsage: 111.50 KB (114176)

- PrepareTime: 39.278ms

AGGREGATION_NODE (id=1):(Total: 8s093ms, non-child: 3.025ms, % non-child: 0.04%)

ExecOption: Codegen Enabled

- BuildTime: 3.004ms

- GetResultsTime: 0.000ns

- HTResizeTime: 0.000ns

- HashBuckets: 0 (0)

- LargestPartitionPercent: 0 (0)

- MaxPartitionLevel: 0 (0)

- NumRepartitions: 0 (0)

- PartitionsCreated: 0 (0)

- PeakMemoryUsage: 67.00 KB (68608)

- RowsRepartitioned: 0 (0)

- RowsReturned: 1 (1)

- RowsReturnedRate: 0

- SpilledPartitions: 0 (0)

KUDU_SCAN_NODE (id=0):(Total: 8s090ms, non-child: 8s090ms, % non-child: 100.00%)

BytesRead(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

- BytesRead: 0

- KuduRemoteScanTokens: 0 (0)

- NumScannerThreadsStarted: 1 (1)

- PeakMemoryUsage: 122.00 KB (124928)

- RowsRead: 1.02M (1018748)

- RowsReturned: 1.02M (1018748)

- RowsReturnedRate: 125.92 K/sec

- ScanRangesComplete: 1 (1)

- ScannerThreadsInvoluntaryContextSwitches: 2 (2)

- ScannerThreadsTotalWallClockTime: 8s091ms

- MaterializeTupleTime(*): 8s085ms

- ScannerThreadsSysTime: 9.998ms

- ScannerThreadsUserTime: 114.982ms

- ScannerThreadsVoluntaryContextSwitches: 31 (31)

- TotalKuduScanRoundTrips: 28 (28)

- TotalReadThroughput: 0.00 /sec

Instance 2945a77ff619defe:b658730000000003 (host=rrs-hdp-dn08:22000):(Total: 6s024ms, non-child: 2.396ms, % non-child: 0.04%)

Hdfs split stats (<volume id>:<# splits>/<split lengths>):

MemoryUsage(500.000ms): 44.18 KB, 44.18 KB, 44.18 KB, 69.18 KB, 69.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB, 103.18 KB

ThreadUsage(500.000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2

- AverageThreadTokens: 2.00

- BloomFilterBytes: 0

- PeakMemoryUsage: 213.18 KB (218299)

- PeakReservation: 0

- PeakUsedReservation: 0

- PerHostPeakMemUsage: 213.18 KB (218299)

- RowsProduced: 1 (1)

- TotalNetworkReceiveTime: 0.000ns

- TotalNetworkSendTime: 406.573us

- TotalStorageWaitTime: 5s867ms

- TotalThreadsInvoluntaryContextSwitches: 0 (0)

- TotalThreadsTotalWallClockTime: 11s935ms

- TotalThreadsSysTime: 13.997ms

- TotalThreadsUserTime: 123.981ms

- TotalThreadsVoluntaryContextSwitches: 1.02K (1021)

Fragment Instance Lifecycle Timings:

- ExecTime: 839.894us

- ExecTreeExecTime: 6.566us

- OpenTime: 5s982ms

- ExecTreeOpenTime: 5s954ms

- PrepareTime: 40.479ms

- ExecTreePrepareTime: 132.040us

DataStreamSender (dst_id=2):(Total: 455.519us, non-child: 455.519us, % non-child: 100.00%)

- BytesSent: 16.00 B (16)

- NetworkThroughput(*): 398.54 KB/sec

- OverallThroughput: 34.30 KB/sec

- PeakMemoryUsage: 16.00 KB (16384)

- RowsReturned: 1 (1)

- SerializeBatchTime: 15.141us

- TransmitDataRPCTime: 39.206us

- UncompressedRowBatchSize: 16.00 B (16)

CodeGen:(Total: 67.243ms, non-child: 67.243ms, % non-child: 100.00%)

- CodegenTime: 684.171us

- CompileTime: 6.448ms

- LoadTime: 0.000ns

- ModuleBitcodeSize: 1.95 MB (2041444)

- NumFunctions: 17 (17)

- NumInstructions: 223 (223)

- OptimizationTime: 21.152ms

- PeakMemoryUsage: 111.50 KB (114176)

- PrepareTime: 38.632ms

AGGREGATION_NODE (id=1):(Total: 5s954ms, non-child: 1.312ms, % non-child: 0.02%)

ExecOption: Codegen Enabled

- BuildTime: 1.922ms

- GetResultsTime: 0.000ns

- HTResizeTime: 0.000ns

- HashBuckets: 0 (0)

- LargestPartitionPercent: 0 (0)

- MaxPartitionLevel: 0 (0)

- NumRepartitions: 0 (0)

- PartitionsCreated: 0 (0)

- PeakMemoryUsage: 67.00 KB (68608)

- RowsRepartitioned: 0 (0)

- RowsReturned: 1 (1)

- RowsReturnedRate: 0

- SpilledPartitions: 0 (0)

KUDU_SCAN_NODE (id=0):(Total: 5s952ms, non-child: 5s952ms, % non-child: 100.00%)

BytesRead(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

- BytesRead: 0

- KuduRemoteScanTokens: 0 (0)

- NumScannerThreadsStarted: 1 (1)

- PeakMemoryUsage: 122.00 KB (124928)

- RowsRead: 1.01M (1014934)

- RowsReturned: 1.01M (1014934)

- RowsReturnedRate: 170.50 K/sec

- ScanRangesComplete: 1 (1)

- ScannerThreadsInvoluntaryContextSwitches: 0 (0)

- ScannerThreadsTotalWallClockTime: 5s952ms

- MaterializeTupleTime(*): 5s948ms

- ScannerThreadsSysTime: 8.998ms

- ScannerThreadsUserTime: 88.986ms

- ScannerThreadsVoluntaryContextSwitches: 25 (25)

- TotalKuduScanRoundTrips: 22 (22)

- TotalReadThroughput: 0.00 /sec

[rrs-hdp-dn01:25003] >

也可以在CDH的Web页面（Cloudera Manager Web UI）上点击CM > Impala > Queries，找到刚刚运行的查询，然后点击Query Details：

接着向下滚动找到“Download Profile”按钮：

由于我只是开发报表，CDH的web页面用户名和密码甲方是不会透露给乙方的，这块就不操作了，也不是本系列的重点。

你也可以到执行查询的Impala Daemon（impalad节点，该节点在Impala集群中为coordinator角色）节点的Web界面来下载query profile信息：

https://{impala-daemon-url}:25000/queries

注：这里的乱码是中文注释，不会影响代码执行。

然后点击对应SQL后的“Details”链接转到“Profile” tab页：

注：由于这里只展现了最近25个操作，所以我操作的目前看不到了。

获取了Profile之后，接下来我们开始剖析Profile。

Profile解析

Query Runtime Profile:

Query (id=2945a77ff619defe:b658730000000000):

Summary:

Session ID: 24abae22c723db5:2c0a25a81814e8a8

Session Type: BEESWAX

Start Time: 2020-07-03 11:58:47.317039000

End Time: 2020-07-03 12:12:56.216137000

Query Type: QUERY

Query State: FINISHED

Query Status: OK

Impala Version: impalad version 2.10.0-cdh5.13.3 RELEASE (build 15a453e15865344e75ce0fc6c4c760696d50f626)

User: root

Connected User: root

Delegated User:

Network Address: 10.138.232.87:36237

Default Db: default

Coordinator: rrs-hdp-dn04:22000

Query Options (set by configuration):

Query Options (set by configuration and planner): MT_DOP=0

Plan:

----------------

我们将其分为几个部分来介绍，这里面有一些频繁用到的重要信息：

1、查询ID：

Query (id=2945a77ff619defe:b658730000000000):

该ID唯一标识在Impala中运行过的SQL，这对于从Impala Daemon日志中用ID查询相关的信息很有用，只需搜索此查询ID，就可以了解SQL运行细节和相关的错误信息。

2、Session类型：

Session Type: BEESWAX

这可以告诉我们连接来自哪里。BEESWAX意味着查询是从impala-shell客户端运行的。如果从Hue运行，类型将是HIVESERVER2，因为Hue是通过HIVESERVER2 thrift连接的。

3、查询起止时间：

Start Time: 2020-07-03 11:58:47.317039000

End Time: 2020-07-03 12:12:56.216137000

这可以告诉我们查询运行了多长时间，此时间包括会话空闲时间（session idle time）。因此，如果你看到在Hue运行一个简单的查询用了几秒钟返回结果，是由于Hue会保持会话打开直到会话关闭或用户运行另一个查询，因此此处的时间可能比正常显示的时间更长。如果通过impala-shell运行，则开始时间和结束时间应与运行时间完全匹配，因为impala-shell在查询完成后会立即关闭查询处理程序。

4、查询状态：

Query Status: OK

这表明查询是否成功完成。OK表示查询成功执行完成。如果有错误，通常会在此处显示，例如，被用户取消（cancelled by user）、会话超时（session timeout）、异常（Exceptions）等。

5、Impala版本：

Impala Version: impalad version 2.10.0-cdh5.13.3 RELEASE (build 15a453e15865344e75ce0fc6c4c760696d50f626)

该信息显示运行查询的Impala版本，如果你发现它与你安装的Impala版本不匹配，则说明未正确安装某些部分。

6、运行查询的用户信息：

User: root

Connected User: root

Delegated User:

该信息表示运行查询的用户。

7、查询所用的数据库：

Default Db: default

该信息表示查询所使用的数据库。

8、查询语句：

如果你帮助其他人排查问题，则需要此信息，因为需要了解查询的构造方式以及涉及的表，在许多情况下，对查询的简单重写将有助于解决问题或提高查询性能。

9、执行查询的impalad节点：

Coordinator: rrs-hdp-dn04:22000

该信息显示运行查询的Impala daemon服务的主机，通常被称为Coordinator，这有助于我们选择从哪个主机查找INFO、WARNING和ERROR级别的日志。

10、查询选项：

Query Options (set by configuration):

Query Options (set by configuration and planner): MT_DOP=0

该信息显示当前查询使用了哪些查询选项（QUERY OPTIONS），这有助于我们判断是否在用户级别（user level）或池级别（pool level）覆盖了查询选项。一个示例是将Impala Daemon的内存设置为120GB，但小查询仍然失败，并抛出OutOfMemory异常，通过该信息可以帮助我们验证用户是否在会话中将MEM_LIMIT参数设置了太小的值（可能导致OutOfMemory异常）。

总结

本系列的第1篇介绍了查询的摘要（Summary）部分以帮助我们了解Query Profile的基本信息，在本系列的下一部分，将详细介绍查询计划以及Profile的执行摘要（Execution Summary）部分。