Linux搭建spark单机操作环境

https://blog.csdn.net/king13127/article/details/86528416
https://www.cnblogs.com/duanxz/p/4437802.html

##spark、Hadoop、java、scala
https://blog.csdn.net/pucao_cug/article/details/72377219
首先分别下载spark、scala和java对应版本
接着编辑3个文件

1、在/etc/profile中加入
export JAVA_HOME=/app/jdk
export SPARK_HOME=/usr/local/spark-2.3.3
export SCALA_HOME=/app/scala
export CLASS_PATH=.:${JAVA_HOME}/lib:$CLASS_PATH
export PATH=.:${JAVA_HOME}/bin:${SPARK_HOME}/bin:${SCALA_HOME}/bin:/root/anaconda3/bin:$PATH



2、在../sbin/spark-env.sh中加入
export SCALA_HOME=/app/scala
export JAVA_HOME=/app/jdk
export  SPARK_HOME=/usr/local/spark-2.3.3


3、在../sbin/spark-config.sh中加入
export JAVA_HOME=/app/jdk


最后运行spark-shell测试代码

##设置日志告警级别
https://jingyan.baidu.com/article/948f5924c023b3d80ff5f9ed.html

注：
scala下载地址：https://www.scala-lang.org/download/2.12.8.html
java（jdk）下载地址：https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
spark（2.3.3版本）下载地址：http://spark.apache.org/downloads.html

转载于:https://my.oschina.net/kyo4321/blog/3076146