python使用spark sql查询impala_使用SparkSQL阅读Impala表

我试图执行一个查询，该函数的功能类似于lead .. over ..分区和Union。当我试图在impala上运行它时，此查询效果很好，但在Hive上失败。使用SparkSQL阅读Impala表

我需要编写一个执行此查询星火工作。它在SparkSQL中也失败了，我的假设是因为Spark 1.6在内部使用HiveQL来完成上述任务。

有一些不同的方式来读取SparkSQL黑斑羚表？因为在Hive中工作的基本查询和Both都可以很好地与SprkSQL一起工作。

FYR我想查询运行：

SELECT issue_id,

CASE WHEN COALESCE(lead(created, 1) OVER (PARTITION BY issue_id ORDER BY created ASC,

field_sequence ASC), '') = '' THEN 'to' ELSE LEAD('from', 1) OVER (PARTITION BY issue_id ORDER BY created ASC, field_sequence ASC) END Status,

created StartDate,

LEAD(created, 1) OVER (PARTITION BY issue_id ORDER BY created ASC, field_sequence ASC) EndDate

FROM (

SELECT issue_id, created, field, 'from', 'to', field_sequence FROM tab1 WHERE COALESCE(LOWER(field), '') = 'status'

UNION

SELECT issue