运行Spark GraphX Pregel出现Issue communicating with driver in heartbeater异常

最近基于Spark GraphX Pregel开发了一个ETL任务，运行过程中会报Issue communicating with driver in heartbeater，然后就是Connection refused错误。

经分析，Spark GraphX Pregel在每次迭代都会产生新的job, 如果迭代的次数多，会产生大量的job, driver会被阻塞造成通讯失败，任务中断。

解决办法：调整driver-memory参数。

spark.driver.memory

1g

Amount of memory to use for the driver process, i.e. where SparkContext is initialized, in the same format as JVM memory strings with a size unit suffix ("k", "m", "g" or "t") (e.g. 512m, 2g).  Note: In client mode, this config must not be set through the SparkConf directly in your application, because the driver JVM has already started at that point. Instead, please set this through the --driver-memory command line option or in your default properties file.

1.1.1

driver-memory默认是1G，将参数调整到32G，任务正常结束。

但是如果数据非常大，这个不是最优解决方案，因为AM的内存分配是有上限的，上限值由配置项 yarn.scheduler.maximum-allocation-mb控制。

其它配置参数：

--num-executors：如果job很多，可考虑调大此参数，增加并行度，能降低运行时间，一般20-50

--executor-memory: 如果job的stage很多，可以调大此参数，默认1G，生产环境4G