报错内容:
Make sure that term plans is in your classpath and check for conflicting dependencies with -Ylog-classpath
.
Error:(52, 15) Symbol 'term org.apache.spark.sql.catalyst.plans' is missing from the classpath.
This symbol is required by 'value org.apache.spark.sql.Dataset.logicalPlan'.
Make sure that term plans is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'Dataset.class' was compiled against an incompatible version of org.apache.spark.sql.catalyst.
.master(s"local[${threads}]")
Error:(52, 15) Symbol 'term org.apache.spark.sql.catalyst.plans' is missing from the classpath.
This symbol is required by 'class org.apache.spark.sql.execution.command.CreateViewCommand'.
Make sure that term plans is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'CreateViewCommand.class' was compiled against an incompatible version of org.apache.spark.sql.catalyst.
.master(s"local[${threads}]")
报错原因:因为用的apache的散装组件,有很多依赖冲突,编译失败。不能在sparksession中执行sql。
a,先排除冲突的依赖,最好冲突依赖都去掉
(我的spark受影响的依赖)
<exclusions>
<exclusion>
<groupId>org.apache.sppa