一.spark on yarn historyserver(运行完毕日志)
1.配置yarn日志参数:yarn-site.xml
<property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property> <property> <name>yarn.log.server.url</name> <value>http://hadoop-senior01.ibeifeng.com:19888/jobhistory/job/</value> </property> |
修改spark的spark-defaults.conf
spark.yarn.historyServer.address http://hadoop-senior01.ibeifeng.com:18080 |