spark config详解

1.spark-env.sh

测试环境配置客串

1. export SPARK_DIST_CLASSPATH=$(/usr/local/hadoop/bin/hadoop classpath)
2. export SCALA_HOME=/usr/local/scala
3. export JAVA_HOME=/opt/jdk1.8.0_65
4. export SPARK_MASTER=localhost
5. export SPARK_LOCAL_IP=localhost
6. export HADOOP_HOME=/usr/local/hadoop
7. export SPARK_HOME=/usr/local/spark
8. export SPARK_LIBARY_PATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$HADOOP_HOME/lib/native
9. export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop

2.slaves

1. # A Spark Worker will be started on each of the machines listed below.
2. mercury
3. venus
4. mars
5. saturn

来自为知笔记(Wiz)

转载于:https://www.cnblogs.com/iathena/p/dd3fd021f637c3e5ee7a256e2e07b985.html