如何设置和读取Spark配置

设置

在SaprkConf中设置，如

val conf = new SparkConf().setMaster("local").setAppName("Myapplication").set("spark.executor.memory","1g") 
val sc = new SparkContext(conf)

或者通过System.properties设置，如

System.setProperty("spark.executor.memory","14g") 
System.setProperty("spark.worker.memory","16g") 
val conf = new SparkConf().setAppName("Simple Application") 
val sc = new SparkContext(conf)

参考：Spark配置参数详解 - 知乎

读取

1.通过Web UI

2.spark-shell

scala> println(Sy