spark-sql的cube函数问题

在spark-sql的命令行可以跑通,使用thrift服务也可跑通

但是放在shell脚本中报如下问题

20/06/08 19:03:07 WARN util.Utils: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.
Error in query: grouping_id() can only be used with GroupingSets/Cube/Rollup;;
'InsertIntoTable 'UnresolvedRelation `ads_order_source_cube`, OverwriteOptions(true,Map()), false
+- 'GroupingSets [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, ... 8 more fields], [corcode_f2#34, institutionId#6, platfromFiledCode#7, orderSource#9, orderStatus#11], [2020-05-31, corcode_f2#34, institutionId#6, platfromFiledCode#7, orderSource#9, orderStatus#11, count(1), 'regexp_replace('regexp_replace('regexp_replace(lpad(bin(cast(grouping_id() as bigint)), 5, 0), 0, 'x), 1, 0), 'x, 1)]
   +- Filter (dt#4 = 2020-05-31)
      +- MetastoreRelation paascloud, dwd_order_basic

20/06/08 19:03:07 INFO server.ServerConnector: Stopped Spark@45eab322{HTTP/1.1}{0.0.0.0:4042}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5d66ae3a{/stages/stage/kill,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@30159886{/jobs/job/kill,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@33de7f3d{/api,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7f4e5a39{/,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1cee2e10{/static,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@38c761e9{/executors/threadDump/json,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2dac2e1b{/executors/threadDump,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5e67a490{/executors/json,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3a4e524{/executors,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@781dac73{/environment/json,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4df7d9ee{/environment,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@6826b70f{/storage/rdd/json,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5f6494c0{/storage/rdd,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@c472300{/storage/json,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5dcf0772{/storage,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@421ead7e{/stages/pool/json,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@56c0a61e{/stages/pool,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@35ac9ebd{/stages/stage/json,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@446b64b3{/stages/stage,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1ad1c363{/stages/json,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@57c69937{/stages,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@76d0ecd7{/jobs/job/json,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4ae280da{/jobs/job,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@100bba26{/jobs/json,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@26d0ece6{/jobs,null,UNAVAILABLE,@Spark}
20/06/08 19:03:07 INFO ui.SparkUI: Stopped Spark web UI at http://192.168.0.213:4042
20/06/08 19:03:07 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
20/06/08 19:03:07 INFO memory.MemoryStore: MemoryStore cleared
20/06/08 19:03:07 INFO storage.BlockManager: BlockManager stopped
20/06/08 19:03:07 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
20/06/08 19:03:07 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
20/06/08 19:03:07 INFO spark.SparkContext: Successfully stopped SparkContext
20/06/08 19:03:07 INFO util.ShutdownHookManager: Shutdown hook called
20/06/08 19:03:07 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-54d95bb7-000f-42ed-964e-0d5417ef346d
20/06/08 19:03:07 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-82edd7df-7588-4e7b-817c-9bc91b3c86ec
[root@hadoop213 paascloud]# 

 

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值