VIRT, RES, %MEM, SHR, SWAP
$ top -M
top - 20:10:06 up 167 days, 10:00, 1 user, load average: 0.66, 0.81, 0.79
Tasks: 168 total, 2 running, 166 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.9%us, 0.7%sy, 0.0%ni, 97.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 11.633G total, 7000.922M used, 4911.324M free, 212.566M buffers
Swap: 8191.996M total, 0.000k used, 8191.996M free, 5432.844M cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ SWAP COMMAND
31281 gcloud 20 0 913m 222m 43m S 4.4 1.9 695:23.69 0 tconnd
31212 gcloud 20 0 913m 229m 43m S 4.1 1.9 694:21.23 0 tconnd
31021 gcloud 20 0 145m 64m 42m R 2.3 0.5 390:00.04 0 version_server
4787 gcloud 20 0 325m 46m 3780 S 1.2 0.4 190:46.46 0 tagent
31348 gcloud 20 0 328m 45m 3920 S 1.2 0.4 193:01.51 0 tagent
31415 gcloud 20 0 332m 41m 3256 S 1.2 0.4 193:14.74 0 tagent
31084 gcloud 20 0 289m 60m 42m S 0.6 0.5 99:16.58 0 dir_server
RES: 进程占用的物理内存。
RES stands for the resident size, which is an accurate representation of how much actual physical memory a process is consuming. (This also corresponds directly to the %MEM column.) This will virtually always be less than the VIRT size, since most programs depend on the C library.%MEM: RES/RAM.
SWAP: 进程占用的 SWAP 分区空间(磁盘空间)。
SHR: 进程占用的共享内存,包括通常所说的共享内存以及动态链接库(例如 glibc)。
SHR indicates how much of the VIRT size is actually sharable memory or libraries). In the case of libraries, it does not necessarily mean that the entire library is resident. For example, if a program only uses a few functions in a library, the whole library is mapped and will be counted in VIRT and SHR, but only the parts of the library file containing the functions being used will actually be loaded in and be counted under RES.VIRT: 进程占用的虚拟内存,包括 RES, SWAP, SHR, 以及分配但是又没有使用的空间(见下文“虚存分配”部分)。网上流传的 “VIRT = RES + SWAP” 严重错误的。
VIRT stands for the virtual size of a process, which is the sum of memory it is actually using, memory it has mapped into itself (for instance the video card’s RAM for the X server), files on disk that have been mapped into it (most notably shared libraries), and memory shared with other processes. VIRT represents how much memory the program is able to access at the present moment.
虚存(内存)分配
首先需要澄清的是,用户空间的内存分配并不会立即引起物理内存的分配,只要在访问它时才会分配物理内存。比如使用 malloc() 或 sbrk() 分配了 100M 内存,但时在访问它前,并不会分配 100M 物理内存,即使访问了这块内存,也不会一次就分配 100M 物理内存。sbrk() 只是调整了一下 program break 而已。
这块内存会在 VIRT 体现出来,在 RES 不会体现出来。
After the program break is increased, the program may access any address in the newly allocated area, but no physical memory pages are allocated yet. The kernel automatically allocates new physical pages on the first attempt by the process to access addresses in those pages. (The Linux Programming Interface - A Linux and UNIX System Programming Handbook, Section 6.4, 7.1)
buffer & cache
top 和 free 命令中 buffer 和 cache 的区别通过举例的方式更容易让人理解。
buffer: 如 socket 的发送和接收缓冲区。
cache: 如反复访问一个文件,这个文件可能会直接缓存在物理内存中以提供访问速度。
Alternatively, if your system has much more physical memory than required by your applications, Linux will cache recently used files in physical memory so that subsequent accesses to that file do not require an access to the hard drive. This can greatly speed up applications that access the hard drive frequently, which, obviously, can prove especially useful for frequently launched applications. The first time the application is launched, it needs to be read from the disk; if the application remains in the cache, however, it needs to be read from the much quicker physical memory. This disk cache differs from the processor cache mentioned in the previous chapter. Other than oprofile, valgrind, and kcachegrind, most tools that report statistics about “cache” are actually referring to disk cache.
In addition to cache, Linux also uses extra memory as buffers. To further optimize applications, Linux sets aside memory to use for data that needs to be written to disk. These set-asides are called buffers. If an application has to write something to the disk, which would usually take a long time, Linux lets the application continue immediately but saves the file data into a memory buffer. At some point in the future, the buffer is flushed to disk, but the application can continue immediately.
It can be discouraging to see very little free memory in a system because of the cache and buffer usage, but this is not necessarily a bad thing. By default, Linux tries to use as much of your memory as possible. This is good. If Linux detects any free memory, it caches applications and data in the free memory to speed up future accesses. Because it is usually a few orders of magnitude faster to access things from memory rather than disk, this can dramatically improve overall performance. When the system needs the cache memory for more important things, the cache memory is erased and given to the system. Subsequent access to the object that was previously cached has to go out to disk to be filled.
[Optimizing Linux Performance: A Hands-On Guide to Linux Performance Tools, 3.1.2.2 Buffers and Cache (Too Much Physical Memory)]
A buffer is a temporary location to store data for a particular application and this data is not used by any other application. This is similar to bandwidth concept. When you try to send burst of data through network, if your network card is capable of sending less data, it will keep these huge amounts of data in buffer so that it can send data constantly in lesser speeds.
In other hand Cache is a memory location to store frequently used data for faster access.
Other difference between a buffer and a cache is that cache can be used multiple times where as buffer is used single time. And both are temporary store for your data processing.
Understanding free command in Linux/Unix - The Linux Juggernaut
free 命令中 -/+ buffers/cache
的含义
$ free -h
total used free shared buffers cached
Mem: 11G 6.8G 4.8G 120M 212M 5.3G
-/+ buffers/cache: 1.3G 10G
Swap: 8.0G 0B 8.0G
In the used
column, this shows the amount of memory that would be used if buffers/cache were not counted as used memory. In the free
column, this shows the amount of memory that would be free if buffers/cache were counted as free memory.
在 -/+ buffers/cache
一行中:
used(1.3G) = Mem:used(6.8G) - Mem:buffers(212M) - Mem:cached(5.3G)
free(10G) = Mem:free(4.8G) + Mem:buffers(212M) + Mem:cached(5.3G)
如果物理内存充足,系统会尽可能多的去做 cache, -/+ buffers/cache
一行的 used
才是真实的应用程序占用的内存。很多情况下,可能发现 Mem
一行的 used
特别大,free
特别小,看到这个,先不要惊慌,瞎看看 -/+ buffers/cache
一行的 used
和 free
, 例如下面这个例子:
$ free -m
total used free shared buffers cached
Mem: 7863 7801 62 0 474 4139
-/+ buffers/cache: 3187 4676
Swap: 2039 1 2038
这个例子中, Mem:free
只剩 62M, 但是 -/+ buffers/cache:free
有 4676M, 所以不用担心。
案例
系统中空闲物理内存大于已使用的交换空间
按照通常的理解,如果系统有足够的物理内存,是不会使用交换分区的,但是为什么会看到这样的现象:
$ free -m
total used free shared buffers cached
Mem: 19824 19537 286 0 995 17012
-/+ buffers/cache: 1529 18294
Swap: 2055 64 1990
$ top -M
top - 20:53:39 up 45 days, 10:53, 7 users, load average: 0.01, 0.04, 0.05
Tasks: 173 total, 1 running, 172 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.2%us, 0.2%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 19.360G total, 19.081G used, 285.008M free, 995.980M buffers
Swap: 2055.184M total, 64.543M used, 1990.641M free, 16.614G cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ SWAP COMMAND
16617 root 20 0 7184 6252 584 S 1.3 0.0 307:05.00 12 sap1002
1838 root 20 0 178m 1920 1644 S 0.3 0.0 0:29.90 464 nmbd
9926 root 20 0 9996 1912 1188 S 0.3 0.0 18:42.29 76 safe_TsysAgent.
10127 root 20 0 1728m 7380 3156 S 0.3 0.0 40:13.34 5032 AgentWorker
24445 root 20 0 37032 14m 836 S 0.3 0.1 25:16.71 4092 secu-tcs-agent
1 root 20 0 19276 1504 1296 S 0.0 0.0 0:02.99 84 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 0 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:01.41 0 ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 0 kworker/0:0H
6 root 20 0 0 0 0 S 0.0 0.0 1:42.34 0 kworker/u16:0
这个场景中,空闲的物理内存有 286M, 已使用的交换分区有 64M, 看上去好像不符合逻辑。
我们动态地推导一下就解释的通了:在次之前的某个时刻,系统物理内存不够用了,内核交换出一些不常用的页,但是后来内存又释放,物理内存又足够了,但是交换出去的空间一直都没有被访问到,这种情况下系统也不会主动去把这些空间再放回物理内存。