1、Unix IPC分类：
2、PIPE内存
缓冲区大小为65536B。
The only difference between pipes and FIFOs is the manner in which
they are created and opened. Once these tasks have been
accomplished, I/O on pipes and FIFOs has exactly the same semantics.
3、IPC标识符
4、IPC访问性和持久性
5、字节流分割消息
6、文件操作
Each process has its own set of file descriptors.
When open a file, ifpathnameis a symbolic link, it is dereferenced.
Permissions actually placed on a new file depend not just on themodeargument, but also on the process umask (Section 15.4.6) and the (optionally present) default access control list (Section 17.6) of the parent directory.
SUSv3 specifies that if open() succeeds, it is guaranteed to use the lowest-numbered unused file descriptor for the process.
Since kernel 2.6.22, the Linux-specific files in the directory/proc/PID/fdinfo can be read to obtain information about the file descriptors of any process on the system. There is one file in this directory for each of the process’s open file
descriptors, with a name that matches the number of the descriptor. The pos field in this file shows the current file offset (Section 4.7). Theflagsfield is an octal number that shows the file access mode flags and open file status flags. (To decode this number, we need to look at the numeric values of these flags in the C library header files.)
A successful call toread()returns the number of bytes actually read, or 0 if end-of-file is encountered.
When performing I/O on a disk file, a successful return fromwrite()doesn’t guarantee that the data has been transferred to disk, because the kernel performs buffering of disk I/O in order to reduce disk activity and expeditewrite()calls.
When a process terminates, all of its open file descriptors are automatically closed.
Applying lseek() to a pipe, FIFO, socket, or terminal is not permitted; lseek() fails, with errno set to ESPIPE.
It is possible to write bytes at an arbitrary point past the end of the file.
The space in between the previous end of the file and the newly written bytes is referred to as afile hole. From a programming point of view, the bytes in a hole exist, and reading from the hole returns a buffer of bytes containing 0 (null bytes).
File holes don’t, however, take up any disk space. The file system doesn’t allocate any disk blocks for a hole until, at some later point, data is written into it.
Writing bytes into the middle of the file hole will decrease the amount of free disk space as the kernel allocates blocks to fill the hole, even though the file’s size doesn’t change.
All system calls are executed atomically. By this, we mean that the kernel guarantees that all of the steps in a system call are completed as a single operation, without being interrupted by another process or thread.
Avoiding this problem requires that the seek to the next byte past the end of the file and the write operation happen atomically. This is what opening a file with the O_APPEND flag guarantees.
Usingfcntl()to modify open file status flags is particularly useful in the following cases:
> The file was not opened by the calling program, so that it had no control over the flags used in theopen()call (e.g., the file may be one of the three standard descriptors that are opened before the program is started).
> The file descriptor was obtained from a system call other thanopen(). Examples of such system calls arepipe(), which creates a pipe and returns two file descriptors referring to either end of the pipe, andsocket(), which creates a socket and returns a file descriptor referring to the socket.
To modify the open file status flags, we usefcntl()to retrieve a copy of the existing flags, then modify the bits we wish to change, and finally make a further call tofcntl() to update the flags. Thus, to enable theO_APPENDflag, we would write the following:
int flags;
flags = fcntl(fd, F_GETFL);
if (flags == -1)
errExit("fcntl");
flags |= O_APPEND;
if (fcntl(fd, F_SETFL, flags) == -1)
errExit("fcntl");
> the per-process file descriptor table;
> the system-wide table of open file descriptions; and
> the file system i-node table.
For each process, the kernel maintains a table of open file descriptors . Each entry in this table records information about a single file descriptor, including:
> a set of flags controlling the operation of the file descriptor (there is just one such flag, the close-on-exec flag, which we describe in Section 27.4); and
> a reference to the open file description.
The kernel maintains a system-wide table of all open file descriptions . (This table is sometimes referred to as the open file table , and its entries are sometimes called open file handles .) An open file description stores all information relating to an open file, including:
> the current file offset (as updated by read() and write() , or explicitly modifiedusing lseek() );
> status flags specified when opening the file (i.e., the flags argument to open() );
> the file access mode (read-only, write-only, or read-write, as specified in open() );
> settings relating to signal-driven I/O (Section 63.3); and
> a reference to the i-node object for this file.
Each file system has a table of i-nodes for all files residing in the file system.
For now, we note that the i-node for each file includes the following information:
> file type (e.g., regular file, socket, or FIFO) and permissions;
> a pointer to a list of locks held on this file; and
> various properties of the file, including its size and timestamps relating to different types of file operations.
The pread() and pwrite()system calls operate just likeread()andwrite(), except that the file I/O is performed at the location specified byoffset, rather than at the current file offset. The file offset is left unchanged by these calls.These system calls can be particularly useful in multithreaded applications.
The off_t data type used to hold a file offset is typically implemented as a signed long integer. (A signed data type is required because the value –1 is used for representing error conditions.) On 32-bit architectures (such as x86-32) this would limit the size of files to 231–1 bytes (i.e., 2 GB).
7、进程
Historically, two widely used formats for UNIX executable files were the originala.out(“assembler output”) format
and the later, more sophisticated COFF(Common Object File Format). Nowadays, most UNIX implementations (including Linux) employ theExecutable and Linking Format (ELF), which provides a number of advantages over the older formats.
The Linux kernel limits process IDs to being less than or equal to 32,767. Once it has reached 32,767, the process ID counter is reset to 300, rather than 1.While the default upper limit for process IDs remains 32,767, this limit is adjustable via the value in the Linux-specific/proc/sys/kernel/pid_maxfile (which is one greater than the maximum process ID). On 32-bit platforms, the maximum value for this file is 32,768, but on 64-bit platforms, it can be adjusted to any value up to 2^22 (approximately 4 million), making it possible to accommodate very large numbers of processes.
If a child process becomes orphaned because its “birth” parent terminates, then the child is adopted by theinitprocess.
The memory allocated to each process is composed of a number of parts, usually referred to assegments.
>The text segment contains the machine-language instructions of the program run by the process.
>The initialized data segmentcontains global and static variables that are explicitly initialized.
>The uninitialized data segmentcontains global and static variables that are not explicitly initialized.
>The stack is a dynamically growing and shrinking segment containing stack frames.
>The heap is an area from which memory (for variables) can be dynamically allocated at run time.
Anapplication binary interface(ABI) is a set of rules specifying how a binary executable should exchange information with some service (e.g., the kernel or a library) at run time. Among other things, an ABI specifies which registers and stack locations are used to exchange this information, and what meaning is attached to the exchanged values. Once compiled for a particular ABI, a binary executable should be able to run on any system presenting the same ABI. This contrasts with a standardized API (such as SUSv3), which guarantees portability only for applications compiled from source code.
The kernel stack is a per-process memory region maintained in kernel memory that is used as the stack for execution of the functions called internally during the execution of a system call.
Each (user) stack frame contains the following information:
>Function arguments and local variables
>Call linkage information
When a new process is created, it inherits a copy of its parent’s environment.
One restriction of C’sgotois that it is not possible to jump out of the current function into another function.
Like the functions in themallocpackage,alloca()allocates memory dynamically.
However, instead of obtaining memory from the heap, alloca()obtains memory
from the stack by increasing the size of the stack frame. We need not—indeed,
must not—call free() to deallocate memory allocated with alloca(). Likewise, it is
not possible to use realloc() to resize a block of memory allocated byalloca().
Another advantage of alloca() is that the memory that it allocates is automatically
freed when the stack frame is removed.
8、系统限制
It is usually preferable to determine the limit on a particular system using<limits.h>,sysconf(), or pathconf().
If a limit can’t be determined,sysconf()returns –1. It may also return –1 if an error occurred. (The only specified error isEINVAL, indicating thatnameis not valid.) To distinguish the case of an indeterminate limit from an error, we must set errnoto 0 before the call; if the call returns –1 anderrnois set after the call, then an error occurred.
From the shell, we can use thegetconfcommand to obtain the limits and options implemented by a particular UNIX implementation.
9、系统和进程信息
In order to provide easier access to kernel information, many modern UNIX implementations provide a/procvirtual file system. This file system resides under the/procdirectory and contains various files that expose kernel information, allowing processes to conveniently read that information, and change it in some cases,
using normal file I/O system calls. The /proc file system is said to be virtual because the files and subdirectories that it contains don’t reside on a disk. Instead, the kernel creates them “on the fly” as processes access them.
As a convenience, any process can access its own/proc/PIDdirectory using the symbolic link /proc/self.
10.文件IO缓存
When working with disk files, theread()andwrite()system calls don’t directly initiate disk access. Instead, they simply copy data between a user-space buffer and a buffer in the kernelbuffer cache.At some later point, the kernel writes (flushes) its buffer to the disk. (Hence, we say that the system call is notsynchronizedwith the disk operation.) If, in the interim, another process attempts to read these bytes of the file, then the kernel automatically supplies the data from the buffer cache, rather than from (the outdated contents of) the file.
Correspondingly, for input, the kernel reads data from the disk and stores it in a kernel buffer. Calls to read() fetch data from this buffer until it is exhausted, at which point the kernel reads the next segment of the file into the buffer cache. (This is a simplification; for sequential file access, the kernel typically performs read-ahead to try to ensure that the next blocks of a file are read into the buffer cache before the reading process requires them. We say a bit more about readahead in Section 13.5.)
The aim of this design is to allowread()andwrite()to be fast, since they don’t need to wait on a (slow) disk operation. This design is also efficient, since it reduces the number of disk transfers that the kernel must perform.
The Linux kernel imposes no fixed upper limit on the size of the buffer cache. The kernel will allocate as many buffer cache pages as are required, limited only by the amount of available physical memory and the demands for physical memory for other purposes (e.g., holding the text and data pages required by running processes). If available memory is scarce, then the kernel flushes some modified buffer cache pages to disk, in order to free those pages for reuse.
The C99 standard makes two requirements if a stream is opened for both input and output. First, an output operation can’t be directly followed by an input operation without an intervening call tofflush()or one of the file-positioning functions (fseek(),fsetpos(), orrewind()). Second, an input operation can’t be directly followed by an output operation without an intervening call to one of the file-positioning functions, unless the input operation encountered end-of-file.
The fsync()system call causes the buffered data and all metadata associated with the open file descriptorfdto be flushed to disk.
the file read-ahead window to the default size (128 kB) .
I/O system calls transfer data directly to the kernel buffer cache, while thestdiolibrary waits until the stream’s user-space buffer is full before callingwrite()to transfer that buffer to the kernel buffer cache. Consider the following code used to write to standard output:
printf("To man the world is twofold, ");
write(STDOUT_FILENO, "in accordance with his twofold attitude.\n", 41);in accordance with his twofold attitude.
To man the world is twofold,
11、IO模型
During the course of this chapter, we’ll consider the reasons we may choose one of
these techniques rather than another. In the meantime, we summarize a few points:
> Theselect()andpoll()system calls are long-standing interfaces that have been
present on UNIX systems for many years. Compared to the other techniques,
their primary advantage is portability. Their main disadvantage is that they
don’t scale well when monitoring large numbers (hundreds or thousands) of
file descriptors.
> The key advantage of theepollAPI is that it allows an application to efficiently
monitor large numbers of file descriptors. Its primary disadvantage is that it is
a Linux-specific API.
Some other UNIX implementations provide (nonstandard) mechanisms similar toepoll. For example, Solaris provides the special/dev/pollfile (described in the Solarispoll(7d)manual page), and some of the BSDs provide thekqueue
API (which provides a more general-purpose monitoring facility thanepoll).
[Stevens et al., 2004] briefly describes these two mechanisms; a longer discussion ofkqueuecan be found in [Lemon, 2001].
> Likeepoll, signal-driven I/O allows an application to efficiently monitor large
numbers of file descriptors. However, epollprovides a number of advantages
over signal-driven I/O:
– We avoid the complexities of dealing with signals.
– We can specify the kind of monitoring that we want to perform (e.g., ready
for reading or ready for writing).
– We can select either level-triggered or edge-triggered notification (described
in Section 63.1.1).
Furthermore, taking full advantage of signal-driven I/O requires the use of
nonportable, Linux-specific features, and if we do this, signal-driven I/O is no
more portable than epoll.
Because, on the one hand,select()andpoll()are more portable, while signal-driven
I/O and epoll deliver better performance, for some applications, it can be worthwhile writing an abstract software layer for monitoring file descriptor events. With such a layer, portable programs can employ epoll(or a similar API) on systems that provide it, and fall back to the use ofselect()orpoll()on other systems.
Thelibeventlibrary is a software layer that provides an abstraction for monitoring file descriptor events. It has been ported to a number of UNIX systems. As its underlying mechanism,libeventcan (transparently) employ any of the techniques described in this chapter:select(),poll(), signal-driven I/O, orepoll, as well as the Solaris specific/dev/pollinterface or the BSDkqueueinterface. (Thus, libeventalso serves as an example of how to use each of these techniques.) Written by Niels Provos,libeventis available athttp://monkey.org/~provos/libevent/.
The select()system call counts a file descriptor multiple times if it occurs in more than one returned file descriptor set. Thepoll()system call returns a count of ready file descriptors, and a file descriptor is counted only once, even
if multiple bits are set in the corresponding reventsfield.
Within the Linux kernel,select()andpoll()both employ the same set of kernelinternalpollroutines. Thesepollroutines are distinct from thepoll()system call itself. Each routine returns information about the readiness of a single file descriptor. This readiness information takes the form of a bit mask whose values correspond to the bits returned in thereventsfield by thepoll()system call (Table 63-2).
The implementation of the poll() system call involves calling the kernelpollroutine for each file descriptor and placing the resulting information in the corresponding reventsfield.
fdcan’t be a file descriptor for a regular file or a directory (the errorEPERM results)
In a multithreaded program, it is possible for one thread to useepoll_ctl()to add file descriptors to the interest list of anepollinstance that is already being monitored byepoll_wait()in another thread.These changes to the interest list will be taken into account immediately, and theepoll_wait()call will return readiness information about the newly added file descriptors.
When we create an epoll instance using epoll_create(), the kernel creates a new in-memory i-node and open file description, and allocates a new file descriptor in the calling process that refers to the open file description. The interest list for an epoll instance is associated with the open file description, not with the epoll file descriptor.
For the purpose of epoll_wait()calls, the kernel monitors the open file description. This means that we must refine our earlier statement that when a file descriptor is closed, it is automatically removed from anyepollinterest lists of which it is a member. The refinement is this: an open file description is removed from theepollinterest list once all file descriptors that refer to it have been closed. This means that if we create duplicate descriptors referring to an open file—usingdup()(or similar) orfork()—then the open file will be removed only after the original descriptor and all of the duplicates have been closed。
12、文件系统
A file system contains the following parts:
>Boot block: This is always the first block in a file system. The boot block is not
used by the file system; rather, it contains information used to boot the operating system. Although only one boot block is needed by the operating system,
all file systems have a boot block (most of which are unused).
> Superblock: This is a single block, immediately following the boot block, which
contains parameter information about the file system, including:
– the size of the i-node table;
– the size of logical blocks in this file system; and
– the size of the file system in logical blocks.
Different file systems residing on the same physical device can be of different
types and sizes, and have different parameter settings (e.g., block size). This is
one of the reasons for splitting a disk into multiple partitions.
> I-node table: Each file or directory in the file system has a unique entry in the i-node
table. This entry records various information about the file. I-nodes are discussed in greater detail in the next section. The i-node table is sometimes also
called the i-list.
> Data blocks: The great majority of space in a file system is used for the blocks of
data that form the files and directories residing in the file system.
13、信号处理函数
If a disk I/O is required, the kernel puts the process to sleep until the I/O completes.
14、监控文件事件
15、进程
The following actions are performed byexit():
> Exit handlers (functions registered withatexit()andon_exit()) are called, in
reverse order of their registration (Section 25.3).
> The stdiostream buffers are flushed.
> The _exit()system call is invoked, using the value supplied instatus
During both normal and abnormal termination of a process, the following actions
occur:
> Open file descriptors, directory streams (Section 18.8), message catalog descriptors (see thecatopen(3)andcatgets(3)manual pages), and conversion descriptors
(see the iconv_open(3) manual page) are closed.
> As a consequence of closing file descriptors, any file locks (Chapter 55) held by
this process are released.
> Any attached System V shared memory segments are detached, and the
shm_nattch counter corresponding to each segment is decremented by one.
(Refer to Section 48.8.)
> For each System V semaphore for which asemadjvalue has been set by the process,
that semadj value is added to the semaphore value. (Refer to Section 47.8.)
> If this is the controlling process for a controlling terminal, then theSIGHUPsignal is sent to each process in the controlling terminal’s foreground process
group, and the terminal is disassociated from the session. We consider this point
further in Section 34.6.
> Any POSIX named semaphores that are open in the calling process are closed
as though sem_close() were called.
> Any POSIX message queues that are open in the calling process are closed as
though mq_close() were called.
> If, as a consequence of this process exiting, a process group becomes orphaned
and there are any stopped processes in that group, then all processes in the
group are sent a SIGHUP signal followed by aSIGCONTsignal. We consider this
point further in Section 34.7.4.
> Any memory locks established by this process usingmlock()ormlockall()(Section 50.2) are removed.
> Any memory mappings established by this process usingmmap()are unmapped.
The statusvalue returned bywait()andwaitpid()allows us to distinguish the following events for the child:
> The child terminated by calling_exit()(orexit()), specifying an integerexit status.
> The child was terminated by the delivery of an unhandled signal.
> The child was stopped by a signal, andwaitpid()was called with theWUNTRACEDflag.
> The child was resumed by aSIGCONTsignal, andwaitpid()was called with the
WCONTINUED flag.
A further point to consider is the issue of reentrancy. In Section 21.1.2, we noted that using a system call (e.g.,waitpid()) from within a signal handler may change the value of the global variableerrno. Such a change could interfere with attempts by the main program to explicitly seterrno(see, for example, the discussion ofgetpriority()in Section 35.1) or check its value after a failed system call. For this reason, it is sometimes necessary to code aSIGCHLDhandler to saveerrnoin a local variable on entry to the handler, and then restore theerrnovalue just prior to returning.
At this point, it is worth remarking that, to some extent, we are playing with words when trying to draw a distinction between the termsthreadandprocess. It helps a little to introduce the termkernel scheduling entity(KSE), which is used in some texts to refer to the objects that are dealt with by the kernel scheduler. Really, threads and processes are simply KSEs that provide for greater and lesser degrees of sharing of attributes (virtual memory, open file descriptors, signal dispositions, process ID, and so on) with other KSEs. The POSIX threads specification provides
just one out of various possible definitions of which attributes should be shared between threads.
If any of the threads in a thread group performs anexec(), then all threads other than the thread group leader are terminated (this behavior corresponds to the semantics required for POSIX threads), and the new program is execed in the thread group leader. In other words, in the new program,gettid()will return the thread ID of
the thread group leader. During an exec(), the termination signal that this process should send to its parent is reset toSIGCHLD.
Roughly, we can say that afork()corresponds to aclone()call withflags specified as
just SIGCHLD, while avfork()corresponds to aclone()call specifyingflagsas follows:
CLONE_VM | CLONE_VFORK | SIGCHLD
The LinuxThreads threading implementation uses clone() (with just the first four
arguments) to create threads by specifying flags as follows:
CLONE_VM | CLONE_FILES | CLONE_FS | CLONE_SIGHAND
The NPTL threading implementation uses clone()(with all seven arguments) to create
threads by specifying flags as follows:
CLONE_VM | CLONE_FILES | CLONE_FS | CLONE_SIGHAND | CLONE_THREAD |
CLONE_SETTLS | CLONE_PARENT_SETTID | CLONE_CHILD_CLEARTID | CLONE_SYSVSEM
one process attribute, thenice value, allows a process to indirectly
influence the kernel’s scheduling algorithm. Each process has a nice value in the
range –20 (high priority) to +19 (low priority); the default is 0
Processes are not scheduled in a strict hierarchy by nice value; rather, the nice
value acts as weighting factor that causes the kernel scheduler to favor processes
with higher priorities.
On recent 2.6 kernels, the realtime round-robin time slice is 0.1 seconds.
16、线程
If a thread is not detached , then we must join with it using pthread_join(). If we fail to do this, then, when the thread terminates, it produces the thread equivalent of a zombie process. Aside from wasting system resources, if enough thread zombies accumulate, we won’t be able to create additional threads.
Once a thread has been detached, it is no longer possible to usepthread_join()to
obtain its return status, and the thread can’t be made joinable again.
Each thread has its own stack whose size is fixed when the thread is created. On
Linux/x86-32, for all threads other than the main thread, the default size of the
per-thread stack is 2 MB. (On some 64-bit architectures, the default size is higher;
for example, it is 32 MB on IA-64.) The main thread has a much larger space for
stack growth 。
17、mutex
According to SUSv3, applying the operations that we describe in the remainder of this section to acopyof a mutex yields results that are undefined. Mutex operations should always be performed only on the original mutex that has
been statically initialized using PTHREAD_MUTEX_INITIALIZERor dynamically initialized usingpthread_mutex_init()。
If the calling thread itself has already locked the mutex given to pthread_mutex_lock(), then, for the default type of mutex, one of two implementationdefined possibilities may result: the thread deadlocks, blocked trying to lock a
mutex that it already owns, or the call fails, returning the error EDEADLK. On Linux, the thread deadlocks by default.
In most well-designed applications, a thread should hold a mutex for only a short time, so that other threads are not prevented from executing in parallel. This guarantees that other threads that are blocked on the mutex will soon be granted a lock on the mutex.
On Linux, mutexes are implemented usingfutexes(an acronym derived from fast user space mutexes), and lock contentions are dealt with using thefutex()system call.
A condition variable is always used in conjunction with a mutex. The mutex provides mutual exclusion for accessing the shared variable, while the condition variable is used to signal changes in the variable’s state.
According to SUSv3, applying the operations that we describe in the remainder of this section to acopyof a condition variable yields results that are undefined. Operations should always be performed only on the original condition variable that has been statically initialized usingPTHREAD_COND_INITIALIZERor dynamically initialized usingpthread_cond_init()
pthread_cond_signal()should be used only if just one of the waiting threads needs to be woken up to handle the
change in state of the shared variable, and it doesn’t matter which one of the waiting threads is woken up. This scenario typically applies when all of the waiting threads are designed to perform the exactly same task.
By contrast,pthread_cond_broadcast()handles the case where the waiting threads are designed to perform different tasks (in which case they probably have different predicates associated with the condition variable).
pthread_cond_wait() performs the following steps:
> unlock the mutex specified bymutex;
> block the calling thread until another thread signals the condition variable cond; and
> relock mutex.
We can’t make any assumptions about the state of the predicate upon return
from pthread_cond_wait(), for the following reasons:
> Other threads may be woken up first. Perhaps several threads were waiting to
acquire the mutex associated with the condition variable. Even if the thread
that signaled the mutex set the predicate to the desired state, it is still possible
that another thread might acquire the mutex first and change the state of the
associated shared variable(s), and thus the state of the predicate.
> Designing for “loose” predicates may be simpler. Sometimes, it is easier to design
applications based on condition variables that indicate possibilityrather than
certainty. In other words, signaling a condition variable would mean “theremay
be something” for the signaled thread to do, rather than “thereissomething” to
do. Using this approach, the condition variable can be signaled based on
approximations of the predicate’s state, and the signaled thread can ascertain
if there really is something to do by rechecking the predicate.
> Spurious wake-ups can occur. On some implementations, a thread waiting on a
condition variable may be woken up even though no other thread actually signaled the condition variable. Such spurious wake-ups are a (rare) consequence
of the techniques required for efficient implementation on some multiprocessor
systems, and are explicitly permitted by SUSv3.
18、daemon
After having closed file descriptors 0, 1, and 2, a daemon normally opens /dev/null
and uses dup2() (or similar) to make all those descriptors refer to this device.
This is done for two reasons:
– It ensures that if the daemon calls library functions that perform I/O on
these descriptors, those functions won’t unexpectedly fail.
– It prevents the possibility that the daemon later opens a file using descriptor
1 or 2, which is then written to—and thus corrupted—by a library function
that expects to treat these descriptors as standard output and standard error.
/dev/null is a virtual device that always discards the data written to it. When we
want to eliminate the standard output or error of a shell command, we can
redirect it to this file. Reads from this device always return end-of-file.
1560
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
