Spark1.5的一个bug

>>> 16/10/15 20:07:35 INFO YarnClientSchedulerBackend: Requesting to kill executor(s) 1
16/10/15 20:07:35 INFO ExecutorAllocationManager: Removing executor 1 because it has been idle for 60 seconds (new desired total will be 0)
16/10/15 20:07:36 ERROR YarnScheduler: Lost executor 1 on hadoop05: remote Rpc client disassociated
16/10/15 20:07:36 INFO DAGScheduler: Executor lost: 1 (epoch 0)
16/10/15 20:07:36 INFO BlockManagerMasterEndpoint: Trying to remove executor 1 from BlockManagerMaster.
16/10/15 20:07:36 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(1, hadoop05, 41258)
16/10/15 20:07:36 INFO BlockManagerMaster: Removed 1 successfully in removeExecutor
16/10/15 20:07:36 INFO ExecutorAllocationManager: Existing executor 1 has been removed (new total is 0)

时不时就报ERROR YarnScheduler: Lost executor 1 on hadoop05: remote Rpc client disassociated的错误。

后查证该问题是spark1.5的bug由于启用了动态分配以及回收资源，当正确的回收资源后，会报出这个错误。
这个错误不会影响集群以及计算任务的结果。
Jira地址：https://issues.apache.org/jira/browse/SPARK-4134

最好的办法是将spark升级至1.6