Spark1.01运行例子以及问题解决

1. 直接下载编译好的 

export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop

SPARK_JAR=./assembly/target/scala-2.10/spark-assembly_2.10-0.9.1-hadoop2.2.0.jar \
./bin/spark-class org.apache.spark.deploy.yarn.Client \
--jar ./examples/target/scala-2.10/spark-examples_2.10-assembly-0.9.1.jar \
--class org.apache.spark.examples.JavaSparkPi \
--args yarn-standalone \
--num-workers 1 \
--master-memory 512m \
--worker-memory 512m \
--worker-cores 1

改改对应的jar的位置

从上图可知 yarnAppstate: Finished运行失败

查看：http://resourcemanager:8088/cluster

可知运行失败，继续跟进查看日志

http://euca-clc:8042/node/containerlogs/container_1405925244858_0002_01_000001/hadoop/stderr/?start=-4096

Caused by: java.lang.NumberFormatException: For input string: "yarn-standalone"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
	at java.lang.Integer.parseInt(Integer.java:449)
	at java.lang.Integer.parseInt(Integer.java:499)
	at org.apache.spark.examples.JavaSparkPi.main(JavaSparkPi.java:40)

查看源代码可知 JavaSparkPi.java被修改了需要接收一个整型参数，不在是运行模式

所以用如下命令去掉 --args yarn-standalone

执行状态

yarnAppState: Succeeded成功了.

访问http://resourcemanager:8088/cluster并查看结果

运行成功。