初识scala(一)－－－使用zsh提交spark-submit时出现zsh no matches found local 2

情况描述：

写一个HelloWorld然后spark-submit时出现zsh no matches found local 2，看了下submit命令，

 

spark-submit --class "com.learning.scala.HelloWorld" --master local[2] target/test-1.0-SNAPSHOT.jar

感觉没问题啊，Ref[1]给到解决方法是zsh中提交spark需要在local加引号。

 

local后面加不加数字的区别是：

| local | run Spark locally with one worker thread (i.e. no parallelism at all) |
| local[K] | run Spark locally with K worker threads (ideally, set K to the number of cores on your machine) |

local默认一个worker执行。

但以为这样就结束了？Naive,　又来了Warnning和一个Error,　BUG....

2018-08-03 10:58:34 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2018-08-03 10:58:34 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 

java.lang.ClassNotFoundException: com.learning.scala.HelloWorld

别急，我们为BUG而生～

Warning解决方法参考Ref[2][3]：

           SPARK_LOCAL_IP解决办法：参考Ref[3]，查找spark-env.sh文件，sudo find / -name spark-env.sh，然后编辑该文件加入

SPARK_LOCAL_IP="<IP address>"

SPARK无法加载HADOOP本地库解决方法：参考Ref[2]，简单来说，

一个是，cp ${HADOOP_HOME}/lib/native/libhadoop.so ${HAVA_HOME}/jre/lib/amd64/

另一个是下载[4]解压，tar -zxvf snappy_1.1.0.orig.tar.gz，cd ./snappy-1.1.0，./configure，make，cd ./libs，

cp libsnappy.so ${HAVA_HOME}/jre/lib/amd64/

至于Error，scala版本需要和POM文件中版本一致

Ref:

[1] https://zpjiang.me/2015/10/17/zsh-no-match-found-local-spark/

[2] https://www.cnblogs.com/devilmaycry812839668/p/6932622.html

[3] https://support.datastax.com/hc/en-us/articles/204675669-Spark-hostname-resolving-to-loopback-address-warning-in-spark-worker-logs

[4] https://launchpad.net/ubuntu/+source/snappy/1.1.0-1