java io 高级

In the main body of this book, it’s important to understand the following topics:
• Buffer handling
• Kernel versus user space
• Virtual memory
• Paging
• File-oriented versus stream I/O
• Multiplexed I/O (readiness selection)
Buffer Handling
Figure 1-1 shows a simplified logical diagram of how block data moves from an external
source, such as a disk, to a memory area inside a running process. The process requests that
its buffer be filled by making the read( ) system call. This results in the kernel issuing
a command to the disk controller hardware to fetch the data from disk. The disk controller
writes the data directly into a kernel memory buffer by DMA without further assistance from
the main CPU. Once the disk controller finishes filling the buffer, the kernel copies the data
from the temporary buffer in kernel space to the buffer specified by the process when it
requested the read( ) operation.

User space is where regular processes live. The JVM is a regular process and dwells in user space. User space is a nonprivileged area: code executing there cannot directly access hardware devices, for example. Kernel space is where the operating system lives. Kernel code has special privileges: it can communicate with device controllers, manipulate the state of processes in user space, etc. Most importantly, all I/O flows through kernel space, either directly (as decsribed here) or indirectly.
When a process requests an I/O operation, it performs a system call, sometimes known as a trap, which transfers control into the kernel. The low-level open( ), read( ), write( ), and close( ) functions so familiar to C/C++ coders do nothing more than set up and perform the appropriate system calls. When the kernel is called in this way, it takes whatever steps are necessary to find the data the process is requesting and transfer it into the specified buffer in user space. The kernel tries to cache and/or prefetch data, so the data being requested by the process may already be available in kernel space. If so, the data requested by the process is copied out. If the data isn’t available, the process is suspended while the kernel goes about bringing the data into memory.
Looking at Figure 1-1, it’s probably occurred to you that copying from kernel space to the final user buffer seems like extra work. Why not tell the disk controller to send it directly to the buffer in user space? There are a couple of problems with this. First, hardware is usually not able to access user space directly.2 Second, block-oriented hardware devices such as disk controllers operate on fixed-size data blocks. The user process may be requesting an oddly sized or misaligned chunk of data. The kernel plays the role of intermediary, breaking down and reassembling data as it moves between user space and storage devices.
1.4.1.1 Scatter/gather
Many operating systems can make the assembly/disassembly process even more efficient. The notion of scatter/gather allows a process to pass a list of buffer addresses to the operating system in one system call. The kernel can then fill or drain the multiple buffers in sequence, scattering the data to multiple user space buffers on a read, or gathering from several buffers on a write.

This saves the user process from making several system calls (which can be expensive) and allows the kernel to optimize handling of the data because it has information about the total transfer. If multiple CPUs are available, it may even be possible to fill or drain several buffers simultaneously.
1.4.2 Virtual Memory
All modern operating systems make use of virtual memory（像Linux操作系统的交换空间）. Virtual memory means that artificial, or virtual, addresses are used in place of physical (hardware RAM) memory addresses. This provides many advantages, which fall into two basic categories:
1. More than one virtual address can refer to the same physical memory location.
2. A virtual memory space can be larger than the actual hardware memory available.
The previous section said that device controllers cannot do DMA directly into user space, but the same effect is achievable by exploiting item 1 above. By mapping a kernel space address to the same physical address as a virtual address in user space, the DMA hardware (which can access only physical memory addresses) can fill a buffer that is simultaneously visible to both the kernel and a user space process. (See Figure 1-3.)

This is great because it eliminates copies between kernel and user space, but requires the kernel and user buffers to share the same page alignment. Buffers must also be a multiple of the block size used by the disk controller (usually 512 byte disk sectors). Operating systems divide their memory address spaces into pages, which are fixed-size groups of bytes. These memory pages are always multiples of the disk block size and are usually powers of 2 (which simplifies addressing). Typical memory page sizes are 1,024, 2,048, and 4,096 bytes. The virtual and physical memory page sizes are always the same. Figure 1-4 shows how virtual memory pages from multiple virtual address spaces can be mapped to physical memory.

1.4.3 Memory Paging
To support the second attribute of virtual memory (having an addressable space larger than physical memory), it’s necessary to do virtual memory paging (often referred to as swapping, though true swapping is done at the process level, not the page level). This is a scheme whereby the pages of a virtual memory space can be persisted to external disk storage to make room in physical memory for other virtual pages. Essentially, physical memory acts as a cache for a paging area, which is the space on disk where the content of memory pages is stored when forced out of physical memory.
Figure 1-5 shows virtual pages belonging to four processes, each with its own virtual memory space. Two of the five pages for Process A are loaded into memory; the others are stored on disk.

Aligning memory page sizes as multiples of the disk block size allows the kernel to issue direct commands to the disk controller hardware to write memory pages to disk or reload them when needed. It turns out that all disk I/O is done at the page level. This is the only way data ever moves between disk and physical memory in modern, paged operating systems.
Modern CPUs contain a subsystem known as the Memory Management Unit (MMU). This device logically sits between the CPU and physical memory. It contains the mapping information needed to translate virtual addresses to physical memory addresses. When the CPU references a memory location, the MMU determines which page the location resides in (usually by shifting or masking the bits of the address value) and translates that virtual page number to a physical page number (this is done in hardware and is extremely fast). If there is no mapping currently in effect between that virtual page and a physical memory page, the MMU raises a page fault to the CPU.
A page fault results in a trap, similar to a system call, which vectors control into the kernel along with information about which virtual address caused the fault. The kernel then takes steps to validate the page. The kernel will schedule a pagein operation to read the content of the missing page back into physical memory. This often results in another page being stolen to make room for the incoming page. In such a case, if the stolen page is dirty (changed sinceits creation or last pagein) a pageout must first be done to copy the stolen page content to the paging area on disk.
This dynamic shuffling of memory pages based on usage is known as demand paging. Some sophisticated algorithms exist in the kernel to optimize this process and to prevent thrashing, a pathological condition in which paging demands become so great that nothing else can get done.
1.4.4 File I/O
File I/O occurs within the context of a filesystem. A filesystem is a very different thing from a disk. Disks store data in sectors(扇区), which are usually 512 bytes each. They are hardware devices that know nothing about the semantics（语义）of files. They simply provide a number of slots（槽）where data can be stored. In this respect, the sectors of a disk are similar to memory pages（内存页）; all are of uniform size and are addressable as a large array.
A filesystem is a higher level of abstraction. Filesystems are a particular method of arranging and interpreting data stored on a disk (or some other random-access, block-oriented device). The code you write almost always interacts with a filesystem, not with the disks directly. It is the filesystem that defines the abstractions of filenames, paths, files, file attributes, etc.
The previous section mentioned that all I/O is done via demand paging. You’ll recall that paging is very low level and always happens as direct transfers of disk sectors into and out of memory pages. So how does this low-level paging translate to file I/O, which can be performed in arbitrary sizes and alignments?
A filesystem organizes a sequence of uniformly sized data blocks. Some blocks store meta information such as maps of free blocks, directories, indexes, etc. Other blocks contain file data. The meta information about individual files describes which blocks contain the file data, where the data ends, when it was last updated, etc.
When a request is made by a user process to read file data, the filesystem implementation determines exactly where on disk that data lives. It then takes action to bring those disk sectors into memory. In older operating systems, this usually meant issuing a command directly to the disk driver to read the needed disk sectors. But in modern, paged operating systems, the filesystem takes advantage of demand paging to bring data into memory.
Filesystems also have a notion of pages(文件系统也有页的概念), which may be the same size as a basic memory page or a multiple of it. Typical filesystem page sizes range from 2,048 to 8,192 bytes and will always be a multiple of the basic memory page size.
How a paged filesystem performs I/O boils down to the following：

1. Determine which filesystem page(s) (group of disk sectors) the request spans. The file content and/or metadata on disk may be spread across multiple filesystem pages, and those pages may be noncontiguous.
2. Allocate enough memory pages in kernel space to hold the identified filesystem pages.
3. Establish mappings between those memory pages and the filesystem pages on disk.
4. Generate page faults for each of those memory pages.
5. The virtual memory system traps the page faults and schedules pageins to validate those pages by reading their contents from disk.
6. Once the pageins have completed, the filesystem breaks down the raw data to extract the requested file content or attribute information.
   Most filesystems also prefetch extra filesystem pages on the assumption that the process will be reading the rest of the file. such as a grep of several files. It seems to run much faster the second time around.
   1.4.4.1 Memory-mapped files
   a special type of I/O operation supported by most operating systems that allows user processes to take maximum advantage of the page-oriented nature of system I/O and completely avoid buffer copies. This is memory-mapped I/O, which is illustrated in Figure 1-6.
   

内存映射到文件系统页有几点优势：

1. 用户进程把文件数据当做内存，所以不再引发读写系统调用。
2. 当用户线程涉及到映射的内存空间，页错误将会被自动生成，将导致文件数据从磁盘加载。如果用户修改了映射的内存空间，受影响的页将自动标记为脏页随后刷新到磁盘并更新文件。
3. 操作系统的虚拟内存系统将会智能完成页的缓存。根据文件系统加载自动管理内存。
4. 数据总是页对其，不需要经常进行缓存拷贝。
5. 可以映射非常大的文件，而不需要占用大量内存来复制数据。
   Virtual memory and disk I/O are intimately linked and, in many respects, are simply two aspects of the same thing. Keep this in mind when handling large amounts of data. Most operating systems are far more effecient when handling data buffers that are page-aligned and are multiples of the native page size.
   1.4.4.2 File locking
   File locking is a scheme by which one process can prevent others from accessing a file or restrict how other processes access that file. Sophisticated applications, such as databases, rely heavily on file locking.