在独立模式下运行Spark集群时，通过--deploy-mode cluster --supervise选项提交了一个Spark应用以实现容错。现在需要保持集群运行，但要停止应用程序。尝试过停止并重启集群、杀死DriverWrapper守护进程以及删除临时文件，但应用总会恢复运行。问题是如何在不改变代码的情况下停止应用，或者提供其他方法来停止应用并保持集群运行。解决方案是使用./bin/spark-class org.apache.spark.deploy.Client kill命令，可以在Master web UI找到driver ID。
I'm running a Spark cluster in standalone mode.
I've submitted a Spark application in cluster mode using options:
--deploy-mode cluster –supervise
So that the job is fault tolerant.
Now I need to keep the cluster running but stop the application from running.
Things I have tried:
Stopping the cluster and restarting it. But the application resumes
execution when I do that.
Used Kill -9 of a daemon named DriverWrapper but the job resumes again after that.
I've also removed temporary files and directories and restarted the cluster but the job resumes again.
So the running application is really fault tolerant.
Question:
Based on the above scenario can someone suggest how I can stop the job from running or what else I can try to stop the application from running but keep the cluster running.</
最低0.47元/天 解锁文章
1663
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
