linux查看gpu使用情况脚本,GPU使用情况监控（CUDA）

以下函数将信息(例如PID，用户名，CPU使用率，内存使用率，GPU内存使用率，程序参数和正在GPU上运行的进程的运行时间)附加到以下输出nvidia-smi：

function better-nvidia-smi () {

nvidia-smi

join -1 1 -2 3 \

--format=csv \

| sed "s/ //g" | sed "s/,/ /g" \

| awk 'NR<=1 {print toupper($0)} NR>1 {print $0}' \

| sed "/\[NotSupported\]/d" \

| awk 'NR<=1{print $0;next}{print $0| "sort -k1"}') \

| awk 'NR<=1{print $0;next}{print $0| "sort -k3"}') \

| column -t

}

输出示例：

$ better-nvidia-smi

Fri Sep 29 16:52:58 2017

+-----------------------------------------------------------------------------+

| NVIDIA-SMI 378.13 Driver Version: 378.13 |

|-------------------------------+----------------------+----------------------+

| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |

| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |

|===============================+======================+======================|

| 0 GeForce GT 730 Off | 0000:01:00.0 N/A | N/A |

| 32% 49C P8 N/A / N/A | 872MiB / 976MiB | N/A Default |

+-------------------------------+----------------------+----------------------+

| 1 Graphics Device Off | 0000:06:00.0 Off | N/A |

| 23% 35C P8 17W / 250W | 199MiB / 11172MiB | 0% Default |

+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+

| Processes: GPU Memory |

| GPU PID Type Process name Usage |

|=============================================================================|

| 0 Not Supported |

| 1 5113 C python 187MiB |

+-----------------------------------------------------------------------------+

PID USED_GPU_MEMORY[MIB] USER PGRP %CPU %MEM TIME COMMAND

9178 187MiB tmborn 9175 129 2.6 04:32:19 ../path/to/python script.py args 42