日志文件:
2018-08-07 15:24:32,767 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1533624199682_0095 transitioned from NEW to INITING
2018-08-07 15:24:32,767 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Adding container_1533624199682_0095_01_000001 to application application_1533624199682_0095
2018-08-07 15:24:32,768 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1533624199682_0095 transitioned from INITING to RUNNING
其中本人nodemanager部署在IP:99.48.1.2上 查看下表对应的端口:8042
那么我们可以get方式浏览器访问
http://99.48.1.2:8042/logLevel?log=org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl&level=WARN
将该类的日志级别设置为WARN。
以下为hadoop常用模块和端口:
组件 节点 默认端口 配置 用途说明
HDFS DataNode 50010 dfs.datanode.address "datanode服务端口,用于数据传输"
HDFS DataNode 50075 dfs.datanode.http.address http服务的端口
HDFS DataNode 50475 dfs.datanode.https.address https服务的端口
HDFS DataNode 50020 dfs.datanode.ipc.address ipc服务的端口
HDFS NameNode 50070 dfs.namenode.http-address http服务的端口
HDFS NameNode 50470 dfs.namenode.https-address https服务的端口
HDFS NameNode 8020 fs.defaultFS 接收Client连接的RPC端口,用于获取文件系统metadata信息。
HDFS journalnode 8485 dfs.journalnode.rpc-address RPC服务
HDFS journalnode 8480 dfs.journalnode.http-address HTTP服务
HDFS ZKFC 8019 dfs.ha.zkfc.port ZooKeeper FailoverController,用于NN HA
YARN ResourceManager 8032 yarn.resourcemanager.address RM的applications manager(ASM)端口
YARN ResourceManager 8030 yarn.resourcemanager.scheduler.address scheduler组件的IPC端口
YARN ResourceManager 8031 yarn.resourcemanager.resource-tracker.address IPC
YARN ResourceManager 8033 yarn.resourcemanager.admin.address IPC
YARN ResourceManager 8088 yarn.resourcemanager.webapp.address http服务端口
YARN NodeManager 8040 yarn.nodemanager.localizer.address localizer IPC
YARN NodeManager 8042 yarn.nodemanager.webapp.address http服务端口
YARN NodeManager 8041 yarn.nodemanager.address NM中container manager的端口
YARN JobHistory Server 10020 mapreduce.jobhistory.address IPC
YARN JobHistory Server 19888 mapreduce.jobhistory.webapp.address http服务端口
参考:
https://blog.csdn.net/sunwukong_hadoop/article/details/60877698