《unix环境高级编程》是这么说的：
标准IO流操作读写普通文件是使用全缓冲的，默认缓冲区长度是该文件系统优先选用的IO长度（从stat结构得到的st_blksize值）
//读取st_blksize的例子：
int main(int argc, char* argv[])
{
if(argc!=2)
{
printf("2 argment\n");
return 0;
}
struct stat buf;
stat(argv[1], &buf);
printf("st_blksize=%d,st_blocks=%d\n", (int)buf.st_blksize, (int)buf.st_blocks);
return 0;
}
setvbuf 用法：
语法: #include <stdio.h>
int setvbuf( FILE *stream, char *buffer, int mode, size_t size );
Parameters
stream
Pointer to FILE structure.
buffer
User-allocated buffer.
mode
Mode of buffering.
size
Buffer size in bytes. Allowable range: 2 <= size <= INT_MAX (2147483647). Internally, the value supplied for size is rounded down to the nearest multiple of 2.
函数setvbuf()设置用于stream(流)的缓冲区到buffer(缓冲区),其大小为size(大小). mode(方式)可以是:
_IOFBF, 表示完全缓冲
_IOLBF, 表示线缓冲
_IONBF, 表示无缓存
程序例:
/* setvbuf example */
#include <stdio.h>
char sBuf[1024];
int main ()
{
FILE *pFile;
pFile=fopen ("myfile.txt","w");
setvbuf ( pFile , sBuf, _IOFBF , 1024 );
// File operations here
fclose (pFile);
return 0;
}
在这个例子中，每次写1024字节，所以只有缓冲区满了（有1024字节），才往文件写入数据，可以减少IO次数
清除和设置文件缓冲区
(1).清除文件缓冲区函数： int fflush(FILE *stream); int flushall(); fflush()函数将清除由stream指向的文件缓冲区里的内容，常用于写完一些数据后，立即用该函数清除缓冲区，以免误操作时，破坏原来的数据。 flushall()将清除所有打开文件所对应的文件缓冲区。
(2).设置文件缓冲区函数 void setbuf(FILE *stream,char *buf); void setvbuf(FILE *stream,char *buf,int type,unsigned size); 这两个函数将使得打开文件后，用户可建立自己的文件缓冲区，而不使用fopen()函数打开文件设定的默认缓冲区。 对于setbuf()函数，buf指出的缓冲区长度由头文件stdio.h中定义的宏BUFSIZE的值决定，缺省值为512字节。当选定buf为空时，setbuf函数将使的文件I/O不带缓冲。而对setvbuf函数，则由malloc函数来分配缓冲区。参数size指明了缓冲区的长度(必须大于0),而参数type则表示了缓冲的类型，其值可以取如下值： type 值 含义 _IOFBF 文件全部缓冲，即缓冲区装满后，才能对文件读写 _IOLBF 文件行缓冲，即缓冲区接收到一个换行符时，才能对文件读写 _IONBF 文件不缓冲，此时忽略buf,size的值，直接读写文件，不再经过文件缓冲区缓冲
//
// sync_file_range
//
/
我们在做数据库程序或者IO密集型的程序的时候，通常在更新的时候，比如说数据库程序，希望更新有一定的安全性，我们会在更新操作结束的时候调用fsync或者fdatasync来flush数据到持久设备去。而且通常是以页面为单位，16K一次或者4K一次。 安全性保证了，但是性能就有很大的损害。而且我们更新的时候，通常是更新文件的某一个页面，那么由于是更新覆盖操作，对文件系统的元数据来讲的话，无需变更，所以我们通常不大关心元数据是否写入。 当更新非常频繁的时候，我们时候能够有其他方法减少性能损失。sync_file_range同学出场了。
Linux下系统调用sync_file_range只在内核2.6.17及更高版本是可用的, 我们常用的RHEL 5U4是支持的。
下面是这篇文章的介绍：
The 2.6.17 kernel will include two new system calls which expand the capabilities available to user-space programs in some interesting ways. This article contains a look at the current form of these new interfaces.
splice()
The splice() system call has a long history. First proposed by Larry McVoy in 1998; it was seen as a way of improving I/O performance on server systems. Despite being often mentioned in the following years, no splice() implementation was ever created for the mainline Linux kernel. That situation changed, however, just before the 2.6.17 merge window was closed when Jens Axboe's splice() patch, along with a number of modifications, was merged.
As of this writing, the splice() interface looks like this:
long splice(int fdin, int fdout, size_t len, unsigned int flags);
A call to splice() will cause the kernel to move up to len bytes from the data source fdin to fdout. The data will move through kernel space only, with a minimum of copying. In the current implementation, at least one of the two file descriptors must refer to a pipe device. That requirement is a limitation of the current code, and it could be removed at some future time.
The flags argument modifies how the copy is done. Currently implemented flags are:
SPLICE_F_NONBLOCK: makes the splice() operations non-blocking. A call to splice() could still block, however, especially if either of the file descriptors has not been set for non-blocking I/O.
SPLICE_F_MORE: a hint to the kernel that more data will come in a subsequent splice() call.
SPLICE_F_MOVE: if the output is a file, this flag will cause the kernel to attempt to move pages directly from the input pipe buffer into the output address space, avoiding a copy operation.
Internally, splice() works using the pipe buffer mechanism added by Linus in early 2005 - that is why one side of the operation is required to be a pipe for now. There are two additions to the ever-growing file_operations structure for devices and filesystems which wish to support splice():
ssize_t (*splice_write)(struct inode *pipe, struct file *out,
size_t len, unsigned int flags);
ssize_t (*splice_read)(struct file *in, struct inode *pipe,
size_t len, unsigned int flags);
The new operations should move len bytes between pipe and either in or out, respecting the given flags. For filesystems, there are generic implementations of these operations which can be used; there is also a generic_splice_sendpage() which is used to enable splicing to a socket. As of this writing, there are no splice() implementations for device drivers, but there is nothing preventing such implementations in the future, for char devices at least.
Discussions on the linux-kernel suggest that the splice() interface could change before it is set in stone with the 2.6.17 release. Andrew Tridgell has requested that an offset argument be added to specify where copying should begin - either that, or a separate psplice() should be added. There is also some concern about error handling; if a splice() call returns an error, how does the application tell whether the problem is with the input or the output? Resolving those issues may require some interface changes over the next month or so.
sync_file_range()
Early in the 2.6.17 process, some changes to the posix_fadvise() system call were merged. The new, Linux-specific options were meant to give applications better control over how data written to files is flushed to the physical media. The capabilities provided are needed, but there were concerns about extending a POSIX-defined function in a Linux-specific way. So, after some discussions, Andrew Morton pulled that patch back out and replaced it with a new system call:
long sync_file_range(int fd, loff_t offset, loff_t nbytes, int flags);
This call will synchronize a file's data to disk, starting at the given offset and proceeding for nbytes bytes (or to the end of the file if nbytes is zero). How the synchronization is done is controlled by flags:
SYNC_FILE_RANGE_WAIT_BEFORE blocks the calling process until any already in-progress writeout of pages (in the given range) completes.
SYNC_FILE_RANGE_WRITE starts writeout of any dirty pages in the given range which are not already under I/O.
SYNC_FILE_RANGE_WAIT_AFTER blocks the calling process until the newly-initiated writes complete.
An application which wants to initiate writeback of all dirty pages should provide the first two flags. Providing all three flags guarantees that those pages are actually on disk when the call returns.
The new implementation avoids distorting the posix_fadvise() system call. It also allows synchronization operations to be performed with a single call, instead of the multiple calls required by the previous attempt. In the future, it may also be possible to add other operations to the flags list; the ability to request metadata synchronization seems to be high on the list.
(Thanks to Michael Kerrisk - who agitated for this change - for providing some of the background information).
也可以man sync_file_range下他的具体作用, 但是请注意，sync_file_range是不可移植的。
sync_file_range – sync a file segment with disk
sync_file_range() permits fine control when synchronising the open file referred to by the file descriptor fd with disk.
offset is the starting byte of the file range to be synchronised. nbytes specifies the length of the range to be synchronised, in bytes; if nbytes is zero, then all bytes from offset through to the end of file are synchronised. Synchronisation is in units of the system page size: offset is rounded down to a page boundary; (offset+nbytes-1) is rounded up to a page boundary.
sync_file_range可以让我们在做多个更新后，一次性的刷数据，这样大大提高IO的性能。 具体的实现在fs/sync.c里面，有兴趣的同学可以围观下。
著名的fio测试工具支持sync_file_range来做sync操作，我们在决定在我们的应用中使用该syscall之前不妨先fio测试一把。
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
