| 下面是buffers与cached的区别。
buffers是指用来给块设备做的缓冲大小，他只记录文件系统的metadata以及 tracking in-flight pages.
cached是用来给文件做缓冲。
那就是说：buffers是用来存储，目录里面有什么内容，权限等等。
而cached直接用来记忆我们打开的文件，如果你想知道他是不是真的生效，你可以试一下，先后执行两次命令#man X ,你就可以明显的感觉到第二次的开打的速度快很多。
实验：在一台没有什么应用的机器上做会看得比较明显。记得实验只能做一次，如果想多做请换一个文件名。
#free
#man X
#free
#man X
#free
你可以先后比较一下free后显示buffers的大小。
另一个实验：
#free
#ls /dev
#free
你比较一下两个的大小，当然这个buffers随时都在增加，但你有ls过的话，增加的速度会变得快，这个就是buffers/chached的区别。 compnik 回复于：2005-10-05 21:44:13 LZ 从gentoo wiki上理解的cache 和 buffer 似乎欠妥
以下内容来自
http://c.ittoolbox.com/documents/document.asp?i=1957
A buffer is something that has yet to be "written" to disk. A cache is something that has been "read" from the disk and stored for later use.
More information provided from moderator
Buffers are allocated by various processes to use as input queues, etc. A simplistic explanation of buffers is that they allow processes to temporarily store input in memory until the process can deal with it.
Cache is typically frequently requested disk I/O. If multiple processes are accessing the same files, much of those files will be cached to improve performance (RAM being so much faster than hard drives).
| compnik 回复于：2005-10-05 22:03:45 | 我理解了一下应该是这样的:
buffer(缓冲): 缓冲是为了程序以后读取数据更快速而分配的，大部份时候应该是程序输出的内容，就是C里面程序的输出到文件的 那个文件的buffer (这个应该是文件相关的)
cache(缓存)： 指的是那些频繁求的IO,如果多个进程访问同一文件，它就会被缓存，以提高性能 (这个应该指是磁盘缓存)
大家可以打开两个终端
在一个终端下 free -m 查看内存情况，
在另一个终端下执行 find /usr/
在find 执行的过程中 会明显看到 buffer 的增加
到一定程序后kernel为增加buffer 的大小，会减少 cache 的大小
|
compnik 回复于：2005-10-05 22:12:29 | 呵呵，wiki上那一段应该有问题，偶把它改了，好像生效了说 ; )
|
bingosek 回复于：2005-10-06 00:12:46 | [quote:52bc9ee017="iwasasoldier"]我们的也是，sybase、4G内存 红旗DC4.1大概10多天内存就消耗完了。然后就死机了。[/quote:52bc9ee017]
那是你客户段程序写得有的问题,占住资源不放,多少资源都会耗光
|
bingosek 回复于：2005-10-06 01:16:14 | [quote:1c17b33417="compnik"]呵呵，wiki上那一段应该有问题，偶把它改了，好像生效了说 ; )[/quote:1c17b33417]
你的理解有问题,见:
http://www.redhat.com/advice/tips/meminfo.html
我们就当redhat的东西是比较权威的解释吧:
[quote:1c17b33417]Buffers: Memory in buffer cache. mostly useless as metric nowadays
Cached: Memory in the pagecache (diskcache) minus SwapCache
SwapCache: Memory that once was swapped out, is swapped back in but still also is in the swapfile (if memory is needed it doesn't need to be swapped out AGAIN because it is already in the swapfile. This saves I/O) [/quote:1c17b33417]
关于buffer/page cache:
http://www.linuxbyte.net/view.php?skin=art&ID=3386
[quote:1c17b33417]page cache、buffer cache和swap cache
page cache：读写文件时文件内容的cache，大小为一个页。不一定在磁盘上连续。
buffer cache：读写磁盘块的时候磁盘块内容的cache，buffer cache的内容对应磁盘上一个连续的区域，一个buffer cache大小可能从512(扇区大小)到一个页。
swap cache： 是page cache的子集。用于多个进程共享的页面被换出到交换区的情况。
page cache 和 buffer cache的关系
本质上是很不同的，buffer cache缓冲磁盘块内容，page cache缓冲文件的一页内容。page cache写回时会使用临时的buffer cache来写磁盘。
bdflush： 把dirty的buffer cache写回磁盘。通常只当dirty的buffer太多或者需要更多的buffer而内存开始不足时运行。page_lauder也可能唤醒它。
kupdate： 定时运行，把写回期限已经到了的dirty buffer写回磁盘。
2.4的改进：page cache和buffer cache耦合得更好了。在2.2里，磁盘文件的读使用page cache，而写绕过 page cache，直接使用buffer cache，因此带来了同步的问题：写完之后必须使用update_vm_cache()更新可能有的 page cache。2.4中page cache做了比较大的改进，文件可以通过page cache直接写了，page cache优先使用 high memory。而且，2.4引入了新的对象：file address space，它包含用来读写一整页数据的方法。这些方法考虑到了 inode的更新、page cache处理和临时buffer的使用。page cache和buffer cache的同步问题就消除了。原来使用 inode+offset查找page cache变成通过file address space+offset；原来struct page 中的 inode成员被address_space类型的mapping成员取代。这个改进还使得匿名内存的共享成为可能（这个在2.2很难实现，许多讨论 过）。[/quote:1c17b33417]
可以看到内核不同buffer和cache的使用也不同,不可以单纯地理解为读写
http://www.linuxforum.net/forum/showflat.php?Cat=&Board=linuxK&Number=106845&page=0&view=collapsed&sb=5&o=7∂=
[quote:1c17b33417]page cache是VFS的一部分，buffer cache是块设备驱动的一部分，或者说 page cache是面向用户IO的cache，buffer cache是面向块设备IO的cache，page cache按照文件的逻辑页进行缓 冲，buffer cache按照文件的物理块进行缓冲。page cache与buffer cache并不相互独立而是相互融合的，同一文件的 cache页即可存在于page cache中，又可存在于buffer cache中，它们在物理内存中只有一份拷贝。文件系统接口就处于 page cache和buffer cache之间，它完成page cache的逻辑页与buffer cache的物理块之间的相互转换，再交给统 一的块设备IO进行调度处理，文件的逻辑块与物理块的关系就表现为page cache与buffer cache的关 系。 [/quote:1c17b33417]
2.4与2.6的不同:
http://kerneltrap.org/node/5283
[quote:1c17b33417]The main difference is that right now, there aren't two distinct caches in the kernel. Meaning, the buffer cache maps on to the page cache. Earlier, in the 2.4 series, the page cache and buffer cache used to be two separate caches, which led to problems of maintaining coherency when a block was present in the buffer cache as well as the page cache. The other issue was that you would obviously use up too much memory if a lot of blocks were present in both caches. Now that the buffer cache entries (to put it very simply) "point" to parts in the page cache, you don't have to worry much about synchronizing the two caches.
The files you can look at are mm/filemap.c, mm/page-writeback.c. It's also worth looking at mm/pdflush.c and understand how the pdflush daemon works. This'd just be a subset of everything that uses the page cache, but a decent enough place to start
[/quote:1c17b33417]
|
compnik 回复于：2005-10-06 14:16:45 | [quote:796779cd79="bingosek"][/quote:796779cd79]
楼上的请指出我理解的哪一点有问题了？
和redhat上的比较权威的解释也没有冲突啊
以下内容来自
http://www.faqs.org/docs/linux_admin/buffer-cache.html
这个也应该比较权威吧
[code:1:796779cd79]
Reading from a disk [1] is very slow compared to accessing (real) memory. In addition, it is common to read the same part of a disk several times during relatively short periods of time. For example, one might first read an e-mail message, then read the letter into an editor when replying to it, then make the mail program read it again when copying it to a folder. Or, consider how often the command ls might be run on a system with many users. By reading the information from disk only once and then keeping it in memory until no longer needed, one can speed up all but the first read. This is called disk buffering, and the memory used for the purpose is called the buffer cache.
Since memory is, unfortunately, a finite, nay, scarce resource, the buffer cache usually cannot be big enough (it can't hold all the data one ever wants to use). When the cache fills up, the data that has been unused for the longest time is discarded and the memory thus freed is used for the new data.
Disk buffering works for writes as well. On the one hand, data that is written is often soon read again (e.g., a source code file is saved to a file, then read by the compiler), so putting data that is written in the cache is a good idea. On the other hand, by only putting the data into the cache, not writing it to disk at once, the program that writes runs quicker. The writes can then be done in the background, without slowing down the other programs.
Most operating systems have buffer caches (although they might be called something else), but not all of them work according to the above principles. Some are write-through: the data is written to disk at once (it is kept in the cache as well, of course). The cache is called write-back if the writes are done at a later time. Write-back is more efficient than write-through, but also a bit more prone to errors: if the machine crashes, or the power is cut at a bad moment, or the floppy is removed from the disk drive before the data in the cache waiting to be written gets written, the changes in the cache are usually lost. This might even mean that the filesystem (if there is one) is not in full working order, perhaps because the unwritten data held important changes to the bookkeeping information.
Because of this, you should never turn off the power without using a proper shutdown procedure (see Chapter 8), or remove a floppy from the disk drive until it has been unmounted (if it was mounted) or after whatever program is using it has signalled that it is finished and the floppy drive light doesn't shine anymore. The sync command flushes the buffer, i.e., forces all unwritten data to be written to disk, and can be used when one wants to be sure that everything is safely written. In traditional UNIX systems, there is a program called update running in the background which does a sync every 30 seconds, so it is usually not necessary to use sync. Linux has an additional daemon, bdflush, which does a more imperfect sync more frequently to avoid the sudden freeze due to heavy disk I/O that sync sometimes causes.
Under Linux, bdflush is started by update. There is usually no reason to worry about it, but if bdflush happens to die for some reason, the kernel will warn about this, and you should start it by hand (/sbin/update).
The cache does not actually buffer files, but blocks, which are the smallest units of disk I/O (under Linux, they are usually 1 kB). This way, also directories, super blocks, other filesystem bookkeeping data, and non-filesystem disks are cached.
The effectiveness of a cache is primarily decided by its size. A small cache is next to useless: it will hold so little data that all cached data is flushed from the cache before it is reused. The critical size depends on how much data is read and written, and how often the same data is accessed. The only way to know is to experiment.
If the cache is of a fixed size, it is not very good to have it too big, either, because that might make the free memory too small and cause swapping (which is also slow). To make the most efficient use of real memory, Linux automatically uses all free RAM for buffer cache, but also automatically makes the cache smaller when programs need more memory.
Under Linux, you do not need to do anything to make use of the cache, it happens completely automatically. Except for following the proper procedures for shutdown and removing floppies, you do not need to worry about it.
[/code:1:796779cd79]
|
bingosek 回复于：2005-10-06 15:16:56 | [quote:e9a78c9f4d]A buffer is something that has yet to be "written" to disk. A cache is something that has been "read" from the disk and stored for later use. [/quote:e9a78c9f4d]
你这一段有问题，程序的读和写是通过通过buffer cache还是page cache来进行内核版本不同这种行为也不同
|
bingosek 回复于：2005-10-06 15:20:09 | redhat比较权威的解释是告诉你，linux vm中buffer和cache到底是什么，因为你引用的所有资料和你自己写的东西里都没有指明buffer和cache是什么
我这里引用redhat的文档告诉大家：
buffer＝buffer cache
cache＝page cache
以及告诉大家buffer/page cache是什么东西，在2.2,2.4,2.6下有什么区别，程序读写的行为有什么区别
|
compnik 回复于：2005-10-06 16:22:09 | [quote:342d2f2b26="bingosek"]
你这一段有问题，程序的读和写是通过通过buffer cache还是page cache来进行内核版本不同这种行为也不同[/quote:342d2f2b26]
那段是引用那个地址里面的
具体是通过buffer cache 还是通过 page cache 2.4和2.6还是应该差不多的
只不过按你所引用的话来看应该是2.6把buffer cache链接入page cache
这样更易于管内核管理而已,但它仍然是 buffer cache ，个人感觉 faq.org上的那个还是比较形像的
再具体的东西可能比较权威的就该是Understanding the linux kernel 了
google print 里面有，可惜没有关于block, buffer介绍的那一章
http://print.google.com/print?id=9yIEji1UheIC&pg=PA474&lpg=PA474&dq=buffer+CACHE&prev=http://print.google.com/print%3Fq%3DUnderstanding%2Bthe%2Blinux%2Bkernel%26oi%3Dprint&sig=-Ajzx7W7UYmhuTAdKFo0-hYmteU&auth=DQAAAG4AAADygPctf4ZFpkCRC7-TlMOlLYbKzIl9kItE8zeIvxfvzHFz1XfMvadlC7I5s4i4pvnHf0g_XfxrxeviV1NHOXYmsZWck6OTaJJVLM6hIV-sNVMf94hyFg8ADasLXz5tZbx1FauIbbYoLOZXGCzRRLFw
|
nonameboy 回复于：2005-10-07 20:32:01 | [quote:72c6bb93c9="compnik"]呵呵，wiki上那一段应该有问题，偶把它改了，好像生效了说 ; )[/quote:72c6bb93c9]
改了那一部分生效的？
|
nonameboy 回复于：2005-10-07 20:57:12 | [quote:6da92053ea="compnik"]LZ 从gentoo wiki上理解的cache 和 buffer 似乎欠妥
以下内容来自
http://c.ittoolbox.com/documents/document.asp?i=1957
A buffer is something that has yet to be "written" to disk. A cache is somet..........[/quote:6da92053ea]
我刚刚做了几个小实验,不过我发现在,当我打一个文本进行编辑,
的时候,大量地粘贴文件,一边看free,
如果按上面的理解,应该是会产生大量buffer,而实验的结果的是chched的增长比buffer 大得多,而你下面的实验,用find 的命令,因为是对目录结构进行查询,所以
结果就像我贴子上说的buffer会增大. 我目前的理解是这样子的.
不过下面几个redhat的引入我还没有看,看完再做几个实验看看来验证.
我的实验是在RHEL4.0 kernel:2.6.9-5.完成的.
我之前的实验,同时也在2.4.21-4下做过.
希望你们也把实验结果发上来,大家计论一下.
|
iwasasoldier 回复于：2005-10-09 10:36:45 | [quote:ed9a66516b="bingosek"]
那是你客户段程序写得有的问题,占住资源不放,多少资源都会耗光[/quote:ed9a66516b]
你怎么知道我的服务器上有应用程序？！我的服务器只作数据库服务用没有任何客户自己编写的程序。问题就是这样的。因为数据库用的内存比较大，大约2.2G。看来Linux的内存管理上遇到大的应用程序（比如数据库）还是有问题的。
|
solaris小兵 回复于：2005-10-09 11:15:30 | 真的很精彩！！
前一段时间一直在关注linux内存管理方面的东东！！这个帖子真的很精辟！
|
jackylau 回复于：2005-10-09 12:25:29 | 引用lz的＂说到这里不得不骂MS，为什么在很多时候还有很大的物理内存的时候，却还是用到了pagefile. ＂
我认为linux也是一样，在系统默认的情况下，真实的内存没有用完的时候，也会用到pagefile
|
rain_over 回复于：2005-10-09 12:35:34 | 顶顶
|
nonameboy 回复于：2005-10-09 22:43:32 | [quote:8a19499011="jackylau"]引用lz的＂说到这里不得不骂MS，为什么在很多时候还有很大的物理内存的时候，却还是用到了pagefile. ＂
我认为linux也是一样，在系统默认的情况下，真实的内存没有用完的时候，也会用到pagefile[/quote:8a19499011]
Linux很少会有这种情况.最少我还是很少见过的.如果你有这样的例子,可以拿出上来分析一下.
|
nonameboy 回复于：2005-10-09 22:45:42 | [quote:4a47abfa80="iwasasoldier"]
你怎么知道我的服务器上有应用程序？！我的服务器只作数据库服务用没有任何客户自己编写的程序。问题就是这样的。因为数据库用的内存比较大，大约 2.2G。看来Linux的内存管理上遇到大的应用程序（比如数据库）还是有..........[/quote:4a47abfa80]
你把情况贴出来看看,另:你的数据库每天要处理多少条记录?
因为最近做了几夽RAC,对于oracle占用内存很有兴趣.
|
bingosek 回复于：2005-10-10 00:53:36 | [quote:98bc06698f="iwasasoldier"]
你怎么知道我的服务器上有应用程序？！我的服务器只作数据库服务用没有任何客户自己编写的程序。问题就是这样的。因为数据库用的内存比较大，大约 2.2G。看来Linux的内存管理上遇到大的应用程序（比如数据库）还是有..........[/quote:98bc06698f]
我是指你的前台程序,前台程序没写好就这个样
|
jackylau 回复于：2005-10-10 08:46:17 | [quote:613f3249e7="nonameboy"]
Linux很少会有这种情况.最少我还是很少见过的.如果你有这样的例子,可以拿出上来分析一下.[/quote:613f3249e7]
这种例子很多,现在没有实例了.你难道这种情况没有过么?
|
nonameboy 回复于：2005-10-10 09:42:42 | [quote:13c0e2a7e8="jackylau"]
这种例子很多,现在没有实例了.你难道这种情况没有过么?[/quote:13c0e2a7e8]
在我的笔记本上出现过,不过用很少量的swap ,但是物理内存只有384M.
而服务器一般有四个G,所以没有见过.
|
hongzjx 回复于：2005-10-10 10:55:12 | swap是需要的,不管做什么应用,其大小是要根据所做的应用而定的,怎么有人会认为swap没必要呢?
某某公司的内存有32G,其swap仍设置了64G
|
nonameboy 回复于：2005-10-10 13:19:11 | [quote:5e477f985a="hongzjx"]swap是需要的,不管做什么应用,其大小是要根据所做的应用而定的,怎么有人会认为swap没必要呢?
某某公司的内存有32G,其swap仍设置了64G[/quote:5e477f985a]
这个只能说明那个公司的IT对内存管理的认识有问题...
##对于linux我的理解是到物理内存已经底于限定的值,才开始分页.
实不知设那么大的swap是基于什么的一个目的.
##相对windows
MS的pagefile有时候说要大于内存,是为了做memory dump.
但是大家只知道其一,不知其二, full memory dump 的文件只能放在系统盘里面.
而在win2000下一个分区最大只能放4095M的pagefile.所以.pagefile放那么大根本是浪费硬盘空间.
|
hongzjx 回复于：2005-10-10 13:41:58 | [quote:7d358e506d]这个只能说明那个公司的IT对内存管理的认识有问题...
##对于linux我的理解是到物理内存已经底于限定的值,才开始分页.
实不知设那么大的swap是基于什么的一个目的.
[/quote:7d358e506d]
对于普通应用是可以,但在很多生产环境中,swap的设置都是很大的,比如移动公司的BOSS系统等,如果谁认为在大内存机器中swap没有必要的话,只能说他太......
|
wolfop 回复于：2005-10-10 15:05:53 | [quote:4e199bcf92="hongzjx"]
对于普通应用是可以,但在很多生产环境中,swap的设置都是很大的,比如移动公司的BOSS系统等,如果谁认为在大内存机器中swap没有必要的话,只能说他太......[/quote:4e199bcf92]
那个不是用LINUX的，一般用AIX或者HPUX，两者对内存管理是不同的。
比如在LINUX下面如果都是FILESYSTEM CACHE占用内存提升的话，一般不会启用SWAP，但是AIX就会。
|
hongzjx 回复于：2005-10-11 10:52:52 | 6.1. What is virtual memory?
Linux supports virtual memory, that is, using a disk as an extension of RAM so that the effective size of usable memory grows correspondingly. The kernel will write the contents of a currently unused block of memory to the hard disk so that the memory can be used for another purpose. When the original contents are needed again, they are read back into memory. This is all made completely transparent to the user; programs running under Linux only see the larger amount of memory available and don't notice that parts of them reside on the disk from time to time. Of course, reading and writing the hard disk is slower (on the order of a thousand times slower) than using real memory, so the programs don't run as fast. The part of the hard disk that is used as virtual memory is called the swap space.
Linux can use either a normal file in the filesystem or a separate partition for swap space. A swap partition is faster, but it is easier to change the size of a swap file (there's no need to repartition the whole hard disk, and possibly install everything from scratch). When you know how much swap space you need, you should go for a swap partition, but if you are uncertain, you can use a swap file first, use the system for a while so that you can get a feel for how much swap you need, and then make a swap partition when you're confident about its size.
You should also know that Linux allows one to use several swap partitions and/or swap files at the same time. This means that if you only occasionally need an unusual amount of swap space, you can set up an extra swap file at such times, instead of keeping the whole amount allocated all the time.
A note on operating system terminology: computer science usually distinguishes between swapping (writing the whole process out to swap space) and paging (writing only fixed size parts, usually a few kilobytes, at a time). Paging is usually more efficient, and that's what Linux does, but traditional Linux terminology talks about swapping anyway.
|
net_xiaobao 回复于：2005-10-12 13:19:10 | 这个问题一直使我关注的，永久关注中
|
天啸 回复于：2005-10-18 15:40:03 | 楼主是我学习的楷模,细致深入啊!
|
jackylau 回复于：2005-10-18 17:20:55 | [quote:06f42c623b="nonameboy"]
在我的笔记本上出现过,不过用很少量的swap ,但是物理内存只有384M.
而服务器一般有四个G,所以没有见过.[/quote:06f42c623b]
晕,我的服务器有很多都是1g,以前常用到swap(其实真正用到的只有50%左右),后来采用pagecache,才解决此问题.
|
bingosek 回复于：2005-10-18 20:56:58 | [quote:2c2f6501e8="jackylau"]
晕,我的服务器有很多都是1g,以前常用到swap(其实真正用到的只有50%左右),后来采用pagecache,才解决此问题.[/quote:2c2f6501e8]
怎么采用pagecache啊？
|
nonameboy 回复于：2005-10-18 22:45:42 | [quote:8c65b19b8d="bingosek"]
怎么采用pagecache啊？[/quote:8c65b19b8d]
我也有同样的问题..怎样做有没有文档以及测试记录?
|
jackylau 回复于：2005-10-20 09:57:39 | 看一下我的一个贴子吧,上面有!
|
bingosek 回复于：2005-10-20 11:25:02 | [quote:1145d08f85="jackylau"]看一下我的一个贴子吧,上面有![/quote:1145d08f85]
方便的话给个链接吧
|
jackylau 回复于：2005-10-21 09:25:05 | | http://bbs.chinaunix.net/forum/viewtopic.php?t=270123&highlight=jackylau |
|
6105
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
