ExecutorAllocationManager

最新推荐文章于 2023-08-17 15:52:19 发布

宝哥大数据

最新推荐文章于 2023-08-17 15:52:19 发布

阅读量393

点赞数

分类专栏： # spark

本文链接：https://blog.csdn.net/wuxintdrh/article/details/89283371

版权

spark 专栏收录该内容

145 篇文章 14 订阅

订阅专栏

1.1、和启动Executor分配管理器ExecutorAllocationManager

		//声明
		private var _executorAllocationManager: Option[ExecutorAllocationManager] = None

		
		//初始化
        // Optionally scale number of executors dynamically based on workload. Exposed for testing.
        val dynamicAllocationEnabled = Utils.isDynamicAllocationEnabled(_conf)
        _executorAllocationManager =
                if (dynamicAllocationEnabled) {
                    schedulerBackend match {
                        case b: ExecutorAllocationClient =>
                            Some(new ExecutorAllocationManager(
                                schedulerBackend.asInstanceOf[ExecutorAllocationClient], listenerBus, _conf))
                        case _ =>
                            None
                    }
                } else {
                    None
                }
		//启动
        _executorAllocationManager.foreach(_.start())