yarn application常用命令及显示示例

一般情况下，开发人员都能方便地访问web管理界面查看任务状态、日志等信息，但在有些无法直接访问管理界面情况下，就需要用到命令了，因此把常用的命令整理出来供参考：
1、-list 列出所有 application 信息
示例：yarn application -list

[ltemr@hbbdc-sp-dn-07 spark]$ yarn application -list
Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):61
                Application-Id      Application-Name        Application-Type          User           Queue                   State             Final-State             Progress                               Tracking-URL
application_1564480217518_0132  UEMRFingerPositionDriver-ltemr_uemr_distri_0                   SPARK         ltemr           ltemr                 RUNNING               UNDEFINED                 10%         http://hbbdc-sp-dn-52:35365

2、-appStates 跟 -list 一起使用，用来筛选不同状态的 application，多个用","分隔；
所有状态：ALL,NEW,NEW_SAVING,SUBMITTED,ACCEPTED,RUNNING,FINISHED,FAILED,KILLED
示例：yarn application -list -appStates RUNNING

  [ltemr@hbbdc-sp-dn-07 spark]$ yarn application -list -appStates RUNNING
Total number of applications (application-types: [], states: [RUNNING] and tags: []):61
                Application-Id      Application-Name        Application-Type          User           Queue                   State             Final-State             Progress                               Tracking-URL
application_1564480217518_0132  UEMRFingerPositionDriver-ltemr_uemr_distri_0                   SPARK         ltemr           ltemr                 RUNNING               UNDEFINED                 10%         http://hbbdc-sp-dn-52:35365

3、-appTypes 跟 -list 一起使用，用来筛选不同类型的 application，多个用","分隔；
如 SPARK、MAPREDUCE 示例：yarn application -list -appTypes SPARK

[ltemr@hbbdc-sp-dn-07 spark]$ yarn application -list -appTypes SPARK
Total number of applications (application-types: [SPARK], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):61
                Application-Id      Application-Name        Application-Type          User           Queue                   State             Final-State             Progress                               Tracking-URL
application_1564480217518_0132  UEMRFingerPositionDriver-ltemr_uemr_distri_0                   SPARK         ltemr           ltemr                 RUNNING               UNDEFINED                 10%         http://hbbdc-sp-dn-52:35365

4、-kill 杀死一个 application，需要指定一个 Application ID

示例：yarn  application -kill application_1526100291229_206393

5、-status 列出 某个application 的状态
示例：yarn application -status application_1526100291229_206393

[ltemr@hbbdc-sp-dn-07 spark]$ yarn application -status application_1564480217518_0170
19/08/02 09:22:29 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
19/08/02 09:22:29 INFO conf.Configuration: found resource resource-types.xml at file:/etc/hadoop/3.1.0.0-78/0/resource-types.xml
Application Report : 
        Application-Id : application_1564480217518_0170
        Application-Name : UEMRFingerPositionDriver-ltemr_uemr_distri_38
        Application-Type : SPARK
        User : ltemr
        Queue : ltemr
        Application Priority : 0
        Start-Time : 1564668893331
        Finish-Time : 0
        Progress : 10%
        State : RUNNING
        Final-State : UNDEFINED
        Tracking-URL : http://hbbdc-sp-dn-47:37580
        RPC Port : 44963
        AM Host : hbbdc-sp-dn-47
        Aggregate Resource Allocation : 1189356923 MB-seconds, 80105 vcore-seconds
        Aggregate Resource Preempted : 0 MB-seconds, 0 vcore-seconds
        Log Aggregation Status : NOT_START
        Diagnostics : 
        Unmanaged Application : false
        Application Node Label Expression : <Not set>
        AM container Node Label Expression : <DEFAULT_PARTITION>
        TimeoutType : LIFETIME  ExpiryTime : UNLIMITED  RemainingTime : -1seconds

[ltemr@hbbdc-sp-dn-07 spark]$ 

6、-movetoqueue 移动 application 到其他的 queue，不能单独使用，并需要权限
7、-queue 与 movetoqueue 命令一起使用，指定移动到哪个 queue

示例：yarn  application -movetoqueue application_1526100291229_206393 -queue ltemr

8、yarn 命令查看任务日志

示例：yarn logs -applicationId application_1526100291229_206393

9、yarn node -list 查看节点清单，显示数据节点Data Node列表

[ltemr@hbbdc-sp-dn-07 spark]$ yarn node -list
Total Nodes:43
         Node-Id             Node-State Node-Http-Address       Number-of-Running-Containers
hbbdc-sp-dn-53:45454            RUNNING hbbdc-sp-dn-53:8042                                1
hbbdc-sp-dn-62:45454            RUNNING hbbdc-sp-dn-62:8042                                4
hbbdc-sp-dn-44:45454            RUNNING hbbdc-sp-dn-44:8042                                3
hbbdc-sp-dn-37:45454            RUNNING hbbdc-sp-dn-37:8042                                0

10、查看队列使用情况 yarn queue -status 队列名称

示例：yarn queue -status ltemr
[ltemr@hbbdc-sp-dn-07 spark]$ yarn queue -status ltemr
19/08/02 09:32:43 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
Queue Information : 
Queue Name : ltemr
        State : RUNNING
        Capacity : 100.0%
        Current Capacity : 81.5%
        Maximum Capacity : 100.0%
        Default Node Label expression : <DEFAULT_PARTITION>
        Accessible Node Labels : *
        Preemption : enabled
        Intra-queue Preemption : enabled
[ltemr@hbbdc-sp-dn-07 spark]$ 

11、yarn top 查看集群资源使用信息

示例：yarn top
[ltemr@hbbdc-sp-dn-07 spark]$ yarn top
YARN top - 09:37:44, up 2d, 15:47, 0 active users, queue(s): root
NodeManager(s): 43 total, 43 active, 0 unhealthy, 0 decommissioned, 0 lost, 0 rebooted
Queue(s) Applications: 61 running, 228 submitted, 0 pending, 28 completed, 139 killed, 0 failed
Queue(s) Mem(GB): 414 available, 1818 allocated, 0 pending, 0 reserved
Queue(s) VCores: 852 available, 129 allocated, 0 pending, 0 reserved
Queue(s) Containers: 129 allocated, 0 pending, 0 reserved
                  APPLICATIONID USER             TYPE      QUEUE PRIOR   #CONT  #RCONT  VCORES RVCORES     MEM    RMEM  VCORESECS    MEMSECS %PROGR       TIME NAME   
 application_1564480217518_0192 ltemr           spark      ltemr     0       9       0       9       0     78G      0G     357183    3095662  10.00   00:11:01 UemrDis
 application_1564480217518_0132 ltemr           spark      ltemr     0       2       0       2       0     29G      0G      82308    1193423  10.00   00:11:26 UEMRFin