报错:
ERROR server.TransportRequestHandler: Error while invoking RpcHandler#receive() for one-way message. org.apache.spark.SparkException: Could not find CoarseGrainedScheduler. at org.apache.spark.rpc.netty.Dispatcher.postMessage(Dispatcher.scala:160) at org.apache.spark.rpc.netty.Dispatcher.postOneWayMessage(Dispatcher.scala:140) at org.apache.spark.rpc.netty.NettyRpcHandler.receive(NettyRpcEnv.scala:655)
参考
解决办法:
提交Spark代码时,设置如下参数值改为一个较大值(单位:秒):
spark.dynamicAllocation.executorIdleTimeout
比如
--conf spark.dynamicAllocation.executorIdleTimeout=360000