问题1:All master are unresponsive! Giving up.
问题1—提交代码的方式:spark-submit connectedComponentAnalysis.py --master yarn --deploy-mode cluster --executor-memory 3g --num-executors 10
问题1—代码环境配置:
conf = SparkConf()
conf.setAppName("cca")
sc = SparkContext(conf=conf) # 必须带参数名
解决方法:
在代码中配置 yarn
conf = SparkConf()
conf.setAppName("cca").setMaster("yarn")
sc = SparkContext(conf=conf) # 必须带参数名
原因:不太清楚