hadoop配置文件聚集功能（设置日志可以查看）

日志聚集概念：应用运行完成以后，将程序运行日志信息上传到HDFS系统上。
日志聚集功能好处：可以方便的查看到程序运行详情，方便开发调试。

没有配置之前history中logs没有内容


具体步骤：
1.关闭NodeManager 、ResourceManager和HistoryManager

[xuzhenchao@hadoop101 hadoop-2.7.5]$ jps
6217 NameNode
6329 DataNode
30985 ResourceManager
31418 JobHistoryServer
31245 NodeManager
31453 Jps
[xuzhenchao@hadoop101 hadoop-2.7.5]$ sbin/mr-jobhistory-daemon.sh stop historyserver
stopping historyserver
[xuzhenchao@hadoop101 hadoop-2.7.5]$ jps
31552 Jps
6217 NameNode
6329 DataNode
30985 ResourceManager
31245 NodeManager
[xuzhenchao@hadoop101 hadoop-2.7.5]$ sbin/yarn-daemon.sh stop nodemanager
stopping nodemanager
[xuzhenchao@hadoop101 hadoop-2.7.5]$ jps
31588 Jps
6217 NameNode
6329 DataNode
30985 ResourceManager
[xuzhenchao@hadoop101 hadoop-2.7.5]$ sbin/yarn-daemon.sh stop resourcemanager
stopping resourcemanager
[xuzhenchao@hadoop101 hadoop-2.7.5]$ jps
31632 Jps
6217 NameNode
6329 DataNode

2.1. 配置yarn-site.xml

[xuzhenchao@hadoop101 hadoop-2.7.5]$ cd etc/
[xuzhenchao@hadoop101 etc]$ ll
总用量 4
drwxr-xr-x. 4 xuzhenchao xuzhenchao 4096 4月  23 15:57 hadoop
[xuzhenchao@hadoop101 etc]$ cd hadoop/
[xuzhenchao@hadoop101 hadoop]$ vim yarn-site.xml 

在文件中增加如下配置信息

<!-- 日志聚集功能使能 -->
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>

<!-- 日志保留时间设置7天 -->
<property>
<name>yarn.log-aggregation.retain-seconds</name>
<value>604800</value>
</property>

3.重新启动NodeManager 、ResourceManager和HistoryManager

[xuzhenchao@hadoop101 hadoop-2.7.5]$ sbin/yarn-daemon.sh start resourcemanager
starting resourcemanager, logging to /opt/module/hadoop-2.7.5/logs/yarn-xuzhenchao-resourcemanager-hadoop101.out
[xuzhenchao@hadoop101 hadoop-2.7.5]$ jps
6217 NameNode
6329 DataNode
31739 ResourceManager
31773 Jps
[xuzhenchao@hadoop101 hadoop-2.7.5]$ sbin/yarn-daemon.sh start nodemanager
starting nodemanager, logging to /opt/module/hadoop-2.7.5/logs/yarn-xuzhenchao-nodemanager-hadoop101.out
[xuzhenchao@hadoop101 hadoop-2.7.5]$ jps
32081 Jps
6217 NameNode
6329 DataNode
31739 ResourceManager
32047 NodeManager
[xuzhenchao@hadoop101 hadoop-2.7.5]$ sbin/mr-jobhistory-daemon.sh start historyserver
starting historyserver, logging to /opt/module/hadoop-2.7.5/logs/mapred-xuzhenchao-historyserver-hadoop101.out
[xuzhenchao@hadoop101 hadoop-2.7.5]$ jps
32258 Jps
32213 JobHistoryServer
6217 NameNode
6329 DataNode
31739 ResourceManager
32047 NodeManager

4.删除HDFS上已经存在的输出文件

[xuzhenchao@hadoop101 hadoop-2.7.5]$ bin/hdfs dfs -rm -r /user/atguigu/output
21/04/26 13:44:49 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.
Deleted /user/atguigu/output

5. 执行WordCount程序

[xuzhenchao@hadoop101 hadoop-2.7.5]$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.5.jar wordcount /user/atguigu/input /user/atguigu/output

配置完成
此时再点击logs效果：