flink图表:analyze流程图

1、analyzetable流程图以tpc-ds生成的表作为例子,先从短小的reason表开始。它一共有3个字段,分别是r_reason_sk,r_reason_id,r_reason_desc。

2、流程图为一条直线,如下图所示,分别A、B、C三个方块。

3、A方块表达了从外部注入表的相关信息,table名,table中的字段名,长度等等。

4、B方块readfields,首先读入了三个字段信息,然后table整个传入buffer缓冲中,table保存在default_catalog,default_database,以reason表明保存。接着,select: (1 AS $f0, r_reason_sk, r_reason_id, CAST(CHAR_LENGTH(r_reason_id)) AS $f3, CHAR_LENGTH(r_reason_id) AS $f4, r_reason_desc, CAST(CHAR_LENGTH(r_reason_desc)) AS $f6, CHAR_LENGTH(r_reason_desc) AS $f7),然后,select:(Partial_COUNT(*) AS count1$0, Partial_COUNT(APPROXIMATE DISTINCT r_reason_sk) AS $f1, Partial_COUNT(r_reason_sk) AS count$1, Partial_MAX(r_reason_sk) AS max$2, Partial_MIN(r_reason_sk) AS min$3, Partial_COUNT(APPROXIMATE DISTINCT r_reason_id) AS $f5, Partial_COUNT(r_reason_id) AS count$4, Partial_AVG($f3) AS (sum$5, count$6), Partial_MAX($f4) AS max$7, Partial_COUNT(APPROXIMATE DISTINCT r_reason_desc) AS $f9, Partial_COUNT(r_reason_desc) AS count$8, Partial_AVG($f6) AS (sum$9, count$10), Partial_MAX($f7) AS max$11),每个字段按照PARTIAL_COUNT(APPROXIMATE DISTINCT),PARTIAL_COUNT,PARTIAL_MAX,PARTIAL_MIN的顺序进行select。

5、C方块Final_COUNT(count1$0) AS $f0, Final_COUNT(APPROXIMATE DISTINCT $f1) AS $f1, Final_COUNT(count$1) AS $f2, Final_MAX(max$2) AS $f3, Final_MIN(min$3) AS $f4, Final_COUNT(APPROXIMATE DISTINCT $f5) AS $f5, Final_COUNT(count$4) AS $f6, Final_AVG(sum$5, count$6) AS $f7, Final_MAX(max$7) AS $f8, Final_COUNT(APPROXIMATE DISTINCT $f9) AS $f9, Final_COUNT(count$8) AS $f10, Final_AVG(sum$9, count$10) AS $f11, Final_MAX(max$11) AS $f12),传进来了一堆参数,唯一能识别出的就是$f开头的都是B方块中的APPROXIMATE DISTINCT参数。select: ($f0 AS EXPR$0, $f1 AS EXPR$1, ($f0 - $f2) AS EXPR$2, 8.0:DOUBLE AS EXPR$3, 8.0 AS EXPR$4, $f3 AS EXPR$5, $f4 AS EXPR$6, $f5 AS EXPR$7, ($f0 - $f6) AS EXPR$8, $f7 AS EXPR$9, $f8 AS EXPR$10, null:INTEGER AS EXPR$11, null:INTEGER AS EXPR$12, $f9 AS EXPR$13, ($f0 - $f10) AS EXPR$14, $f11 AS EXPR$15, $f12 AS EXPR$16, null:INTEGER AS EXPR$17, null:INTEGER AS EXPR$18))然后把他们分门别类的传入expr中,最后用sink collection传出。

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值