spark提交任务到集群

2019独角兽企业重金招聘Python工程师标准>>>

参考：http://spark.apache.org/docs/latest/submitting-applications.html

 

 

#Run on a Spark standalone cluster in client deploy mode(运行集群模式，客户端发布模式)

spark-submit \
  --class org.apache.spark.examples.SparkPi \
  --master spark://207.184.161.138:7077 \
  --executor-memory 20G \
  --total-executor-cores 100 \
  /path/to/examples.jar \
  1000

如采用：Run on a Spark standalone cluster in cluster deploy mode with supervise（发布模式为集群，则jar包，每个worker都要有此jar 包）

 

转载于:https://my.oschina.net/u/160697/blog/627670