Spark源码--RuleExecutor

Spark SQL catalyst RuleExecutor的作用是分批次(batch)执行一系列的优化规则直到稳定点(fixpoint)或最大迭代次数,如图所示：

实现如下：

源码如下：   /** 优化规则执行器 */ abstract  class  RuleExecutor[TreeType < :  TreeNode[ _ ]]  extends  Logging {      /** 控制运行次数的策略，如果在达到最大迭代次数前到达稳定点就停止运行 */    abstract  class  Strategy {  def  maxIterations :  Int }      /** 只运行一次的策略 */    case  object  Once  extends  Strategy {  val  maxIterations  =  1  }      /** 运行到稳定点或者最大迭代次数的策略，2选1 */    case  class  FixedPoint(maxIterations :  Int)  extends  Strategy      /** 一批规则 */    protected  case  class  Batch(name :  String, strategy :  Strategy, rules :  Rule[TreeType]*)      /** Defines a sequence of rule batches, to be overridden by the implementation. */    protected  val  batches :  Seq[Batch]        /**     * 执行器入口     Executes the batches of rules defined by the subclass. The batches are executed serially     * using the defined execution strategy. Within each batch, rules are also executed serially.     */    def  execute(plan :  TreeType) :  TreeType  =  {      var  curPlan  =  plan        batches.foreach { batch  = >        // 用来对比执行规则前后，初始的plan有无变化        val  batchStartPlan  =  curPlan        var  iteration  =  1        var  lastPlan  =  curPlan        var  continue  =  true          // 执行直到达到稳定点或者最大迭代次数        while  (continue) {          curPlan  =  batch.rules.foldLeft(curPlan) {            case  (plan, rule)  = >              val  startTime  =  System.nanoTime()              val  result  =  rule(plan)              val  runTime  =  System.nanoTime() - startTime              RuleExecutor.timeMap.addAndGet(rule.ruleName, runTime)                if  (!result.fastEquals(plan)) {                logTrace(                  s "" "                    |=== Applying Rule ${rule.ruleName} ===                    |${sideBySide(plan.treeString, result.treeString).mkString(" \n ")}                  " "" .stripMargin)              }                result          }          iteration + =  1          // 到达最大迭代次数, 不再执行优化          if  (iteration > batch.strategy.maxIterations) {            // 只对最大迭代次数大于1的情况打log            if  (iteration ! =  2 ) {              logInfo(s "Max iterations (${iteration - 1}) reached for batch ${batch.name}" )            }            continue  =  false          }             // plan不变了，到达稳定点，不再执行优化          if  (curPlan.fastEquals(lastPlan)) {            logTrace(              s "Fixed point reached for batch ${batch.name} after ${iteration - 1} iterations." )            continue  =  false          }          lastPlan  =  curPlan        }          if  (!batchStartPlan.fastEquals(curPlan)) {          logDebug(            s "" "            |=== Result of Batch ${batch.name} ===            |${sideBySide(plan.treeString, curPlan.treeString).mkString(" \n ")}          " "" .stripMargin)        }  else  {          logTrace(s "Batch ${batch.name} has no effect." )        }      }        curPlan    } }