Spark HA模式访问Hadoop HA下的数据

首先是需要将hadoop的配置文件core-site,xml和hdfs-site.xml 拷贝到Spark conf目录下

然后启动提交即可

spark-submit \
--master spark://hadoop-namenode-01:7077,hadoop-namenode-02:7077 \
--class org.apache.spark.examples.SparkPi \
--deploy-mode cluster \
--executor-memory 4G \
--total-executor-cores 10 \
hdfs://bi/spark-examples-1.6.0-hadoop2.6.4.jar \
100

 

转载于:https://www.cnblogs.com/atomicbomb/p/7064326.html