hadoop：三种环境运行spark PI

最新推荐文章于 2024-08-12 15:57:42 发布

花和尚也有春天

最新推荐文章于 2024-08-12 15:57:42 发布

阅读量435

点赞数

分类专栏： spark 文章标签： spark local模式 standalone模式 yarn yarn模式

本文链接：https://blog.csdn.net/weixin_38750084/article/details/104134904

版权

spark 专栏收录该内容

67 篇文章 5 订阅

订阅专栏


local:
bin/spark-submit \
--class org.apache.spark.examples.SparkPi \
--master local \
./lib/spark-examples-1.5.1-hadoop2.4.0.jar


bin/spark-submit \
--class org.apache.spark.examples.SparkPi \
--executor-memory 1G \
--total-executor-cores 2 \
./lib/spark-examples-1.5.1-hadoop2.4.0.jar \
100


standalone:
bin/spark-submit \
--class org.apache.spark.examples.SparkPi \
--master spark://hadoop102:7077 \
--executor-memory 1G \
--total-executor-cores 2 \
./examples/jars/spark-examples_2.11-2.1.1.jar \
100


yarn:
bin/spark-submit \
--class org.apache.spark.examples.SparkPi \
--master yarn \
--deploy-mode client \
./lib/spark-examples-1.5.1-hadoop2.4.0.jar \
100

bin/spark-submit \
--class org.apache.spark.examples.SparkPi \
--master yarn \
--deploy-mode cluster \
./lib/spark-examples-1.5.1-hadoop2.4.0.jar \
100