说三种情况,看大家有没有遇到类似的场景。
第一种情况:
这种情况也是我经常会遇到的一个场景,之前也有同学拿着sql来问,说这样写会不会影响运行效率:
select
tmp.A
from (select A,B from testdata2) tmp
结论是
不用担心,这样写完全可以被优化
== Analyzed Logical Plan ==
Project [A#3]
+- SubqueryAlias tmp
+- Project [A#3, B#4]
+- SubqueryAlias testdata2
+- View (`testData2`, [a#3,b#4])
+- SerializeFromObject [knownnotnull(assertnotnull(input[0, org.apache.spark.sql.test.SQLTestData$TestData2, true])).a AS a#3, knownnotnull(assertnotnull(input[0, org.apache.spark.sql.test.SQLTestData$TestData2, true])).b AS b#4]
+- ExternalRDD [obj#2]
== Optimized Logical Plan ==
Project [A#3]
+- SerializeFromObject [knownnotnull(assertnotnull(input[0, org.apache.spark.sql.test.SQLTestData$TestData2, true])).a AS a#3]
+- ExternalRDD [obj#2]
从执行计划上清晰的看到,最终被优化成
select A from testdata2
这样的效果,主要是 ColumnPruning(列裁剪&#