Spark独立应用程序 Scala

1. 安装sbt
2. 创建应用
3. 使用sbt打包Scala程序
4. spark-submit 运行程序
5. 安装idea

安装sbt

创建目录

mkdir /usr/local/sbt

sbt-launch.jar

cp ~/下载/sbt-launch.jar .
chmod u+x ./sbt
检查
hadoop@dhjvirtualmachine:/usr/local/sbt$ ./sbt sbt-version
[info] Set current project to sbt (in build file:/usr/local/sbt/)
[warn] The `-` command is deprecated in favor of `onFailure` and will be removed in 0.14.0
hadoop@dhjvirtualmachine:/usr/local/sbt$ ./sbt sbt-version
[info] Set current project to sbt (in build file:/usr/local/sbt/)
[info] 0.13.11

创建应用

终端中执行如下命令创建一个文件夹 sparkapp 作为应用程序根目录

cd ~           # 进入用户主文件夹
mkdir ./sparkapp        # 创建应用程序根目录
mkdir -p ./sparkapp/src/main/sca