2、JAVA环境配置、scala环境
3、
tar -zxvf spark-1.6.1-bin-hadoop2.6.tgz
4、cd spark/bin
5、执行 错误异常
java.net.UnknownHostException: s-238: s-238: Name or service not known
把/etc/hostname文件中的主机名加入到/etc/hosts文件中,问题解决。
6、执行
/home/spark/spark-1.6.1-bin-hadoop2.6/sbin$ 下面的 /start-all.sh
cloudera
集群配置:
1、cp spark-env.sh.template spark-env.sh
2、
cp slaves.template slaves
vim slaves
s-238
s-239
vim
spark-env.sh
export SPARK_MASTER_IP=192.168.110.238
./bin/spark-submit --master spark://192.168.110.238:7077 --executor-memory 10G /home/spark/word.jar
添加mysql
./bin/spark-submit --master spark://192.168.110.238:7077 --jars /home/spark/mysql-connector-java-6.0.2.jar --executor-memory 20G /home/spark/category.jar