本地运行spark程序配置

val  inputFile = "hdfs://node02:9000/input/Hamlet.txt"
val conf = new SparkConf().setAppName("WordCount")
  //.setMaster("local")
  .setMaster("spark://node01:7077")
  .setJars(List("D:\\code\\Test\\TestSpark\\out\\artifacts\\TestSpark_jar\\TestSpark.jar"))//.setMaster("spark://node01:7077")
  //.setExecutorEnv("spark.sql.warehouse.dir","file:///D://tmp//hadoop")
  //.config("spark.sql.warehouse.dir","file:///D://test")
val sc = new SparkContext(conf)
val textFile = sc.textFile(inputFile)

 

如果master设置为spark://node01:7077  则file需要设置为hdfs分布式的文件     setJars参数必须设置