EMR 配置纪录（不断更新）

日志路径 -> /var/log（软连接到 /mnt/var/log）

在 spark master 机器中，配置的路径为

/usr/lib/spark/conf
/usr/lib/hadoop-yarn/etc/hadoop/

　　

其中关键的配置

<property><name>yarn.nodemanager.local-dirs</name><value>/mnt/yarn</value></property>
<property><name>yarn.nodemanager.log-dirs</name><value>/var/log/hadoop-yarn/containers</value></property>

local-dirs: 应用的本地文件，官方解释为：

List of directories to store localized files in. An application's localized file directory will be found in: ${yarn.nodemanager.local-dirs}/usercache/${user}/appcache/application_${appid}. Individual containers' work directories, called container_${contid}, will be subdirectories of this.

log-dirs: 应用的日志路径，官方解释为：
Where to store container logs. An application's localized log directory will be found in ${yarn.nodemanager.log-dirs}/application_${appid}. Individual containers' log directories will be below this, in directories named container_{$contid}. Each container directory will contain the files stderr, stdin, and syslog generated by that container.

 

转载于:https://www.cnblogs.com/keepthinking/p/10386803.html