【YARN运维】Yarn常见任务shell运维命令：集群状态、查看任务、任务日志

一. 查看Yarn集群状态

1. 集群所有节点状态

yarn node -list -all

Total Nodes:4
         Node-Id             Node-State Node-Http-Address       Number-of-Running-Containers
    hadoop6:8842                RUNNING      hadoop6:8042                                  1
    hadoop3:8842                RUNNING      hadoop3:8042                                  0
    hadoop5:8842                RUNNING      hadoop5:8042                                  0
    hadoop4:8842                RUNNING      hadoop4:8042                                  0

 

2. 某个队列的情况

yarn queue -status default

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Queue Information :
Queue Name : root.default
        State : RUNNING
        Capacity : .0%
        Current Capacity : .0%
        Maximum Capacity : -100.0%
        Default Node Label expression : DEFAULT_LABEL
        Accessible Node Labels : DEFAULT_LABEL

 

curl http://yarn-node:8088/ws/v1/cluster/scheduler

 

二. 查看任务

1. 查看所有任务

yarn application -list

Total number of applications (application-types: [] and states: [SUBMITTED, ACCEPTED, RUNNING]):1
                Application-Id      Application-Name        Application-Type          User           Queue                   State             Final-State             Progress                        Tracking-URL
application_1581674307855_264715                     jdittoa            Apache Flink          work       root.work                 RUNNING               UNDEFINED                 100%                 http://hadoop6:44358

 

2. 查看处于某个状态的任务

状态有：ALL、NEW、NEW_SAVING、SUBMITTED、ACCEPTED、RUNNING、FINISHED、FAILED、KILLED。

yarn application -list -appStates running

Total number of applications (application-types: [] and states: [RUNNING]):1
                Application-Id      Application-Name        Application-Type          User           Queue                   State             Final-State             Progress                        Tracking-URL
application_1581674307855_264715                     jdittoa            Apache Flink          work       root.work                 RUNNING               UNDEFINED                 100%                 http://hadoop6:44358

 

3. 查看任务尝试情况

如下任务尝试了一次

yarn applicationattempt -list application_1581674307855_264715

Total number of application attempts :1
         ApplicationAttempt-Id                 State                        AM-Container-Id                            Tracking-URL
appattempt_1581674307855_264715_000001               RUNNING    container_e03_1581674307855_264715_01_000001    http://hadoop1:8888/proxy/application_1581674307855_264715/

 

查看尝试任务的状态，id由上获取

yarn applicationattempt -status appattempt_1581674307855_264715_000001

Application Attempt Report :
        ApplicationAttempt-Id : appattempt_1581674307855_264715_000001
        State : RUNNING
        AMContainer : container_e03_1581674307855_264715_01_000001
        Tracking-URL : http://hadoop1:8888/proxy/application_1581674307855_264715/
        RPC Port : 44358
        AM Host : hadoop6
        Diagnostics : Attempt recovered after RM restart

查看（正在运行任务的）container情况

yarn container -status container_e03_1581674307855_264715_01_000001

Container Report :
        Container-Id : container_e03_1581674307855_264715_01_000001
        Start-Time : 1611554602648
        Finish-Time : 0
        State : RUNNING
        LOG-URL : http://hadoop6:8042/node/containerlogs/container_e03_1581674307855_264715_01_000001/work
        Host : hadoop6:8842
        NodeHttpAddress : http://hadoop6:8042
        Diagnostics : null

 

4. 杀掉任务

 yarn application -kill app_id
 

 

三. 查看日志

1. 任务日志

yarn logs -applicationId application_1581674307855_264715

 

2. 查看某个container的日志

 yarn logs -applicationId application_1581674307855_264715 \
 -containerId container_e03_1581674307855_264715_01_000001 \
 -nodeAddress hadoop6