Linux提供的零拷贝技术 Java并不是全支持,支持2种(内存映射mmap、sendfile)
一、mmap内存映射
DMA加载磁盘数据到kernel buffer后,应用程序缓冲区(application buffers)和内核缓冲区(kernel buffer)进行映射,数据再应用缓冲区和内核缓存区的改变就能省略。
mmap内存映射将会经历:3次拷贝: 1次cpu copy,2次DMA copy;
以及4次上下文切换
NIO提供的内存映射 MappedByteBuffer
public final FileChannel getChannel() {
synchronized (this) {
if (channel == null) {
channel = FileChannelImpl.open(fd, path, true, rw, this);
}
return channel;
}
}
public static class MapMode {
/**
* Mode for a read-only mapping.
*/
public static final MapMode READ_ONLY
= new MapMode("READ_ONLY");
/**
* Mode for a read/write mapping.
*/
public static final MapMode READ_WRITE
= new MapMode("READ_WRITE");
/**
* Mode for a private (copy-on-write) mapping.
*/
public static final MapMode PRIVATE
= new MapMode("PRIVATE");
private final String name;
private MapMode(String name) {
this.name = name;
}
/**
* Returns a string describing this file-mapping mode.
*
* @return A descriptive string
*/
public String toString() {
return name;
}
}
MapMode.READ_ONLY:只读,试图修改得到的缓冲区将导致抛出异常。
MapMode.READ_WRITE:读/写,对得到的缓冲区的更改最终将写入文件;但该更改对映射到同一文件的其他程序不一定是可见的。
MapMode.PRIVATE:私用,可读可写,但是修改的内容不会写入文件,只是buffer自身的改变,这种能力称之为”copy on write”。
public MappedByteBuffer map(FileChannel.MapMode var1, long var2, long var4) throws IOException {
// 省略.............
try {
var7 = this.map0(var6, var13, var15);
} catch (OutOfMemoryError var30) {
System.gc();
try {
Thread.sleep(100L);
} catch (InterruptedException var29) {
Thread.currentThread().interrupt();
}
try {
var7 = this.map0(var6, var13, var15);
} catch (OutOfMemoryError var28) {
throw new IOException("Map failed", var28);
}
}
int var18 = (int)var4;
Unmapper var19 = new Unmapper(var7, var15, var18, var17);
// 判断Mode是否是可读
if (this.writable && var6 != 0) {
var20 = Util.newMappedByteBuffer(var18, var7 + (long)var12, var17, var19);
return var20;
}
var20 = Util.newMappedByteBufferR(var18, var7 + (long)var12, var17, var19);
return var20;
}
上述代码说明 map通过native函数map0把文件映射到虚拟内存,并返回逻辑地址address
- 如果第一次文件映射导致OOM,则手动触发垃圾回收,休眠100ms后再次尝试映射,如果失败,则抛出异常。
- 通过newMappedByteBuffer方法初始化MappedByteBuffer实例
static MappedByteBuffer newMappedByteBuffer(int var0, long var1, FileDescriptor var3, Runnable var4) {
if (directByteBufferConstructor == null) {
initDBBConstructor();
}
try {
MappedByteBuffer var5 = (MappedByteBuffer)directByteBufferConstructor.newInstance(new Integer(var0), new Long(var1), var3, var4);
return var5;
} catch (IllegalAccessException | InvocationTargetException | InstantiationException var7) {
throw new InternalError(var7);
}
}
private static void initDBBConstructor() {
AccessController.doPrivileged(new PrivilegedAction<Void>() {
public Void run() {
try {
Class var1 = Class.forName("java.nio.DirectByteBuffer");
Constructor var2 = var1.getDeclaredConstructor(Integer.TYPE, Long.TYPE, FileDescriptor.class, Runnable.class);
var2.setAccessible(true);
Util.directByteBufferConstructor = var2;
return null;
} catch (NoSuchMethodException | IllegalArgumentException | ClassCastException | ClassNotFoundException var3) {
throw new InternalError(var3);
}
}
});
}
由于FileChannelImpl和DirectByteBuffer不在同一个包中,所以有权限访问问题,通过AccessController
(在源码内较常见,之后可以多看看)类获取DirectByteBuffer的构造器进行实例化。
DirectByteBuffer 存储了对内存的直接操作。map0()函数返回一个地址address,通过address就能够操作文件。底层采用unsafe.getByte方法,通过(address + 偏移量)获取指定内存中数据。
private final Cleaner cleaner;
public Cleaner cleaner() { return cleaner; }
public byte get() {
return ((unsafe.getByte(ix(nextGetIndex()))));
}
public byte get(int i) {
return ((unsafe.getByte(ix(checkIndex(i)))));
}
private long ix(int i) {
return address + ((long)i << 0);
}
清除mmap映射
((DirectBuffer) map).cleaner().clean();
二、sendfile
当调用sendfile()时,DMA将磁盘数据复制到kernel buffer,然后将内核中的kernel buffer直接拷贝到socket buffer;
一旦数据全都拷贝到socket buffer,sendfile()系统调用将会return、代表数据转化的完成。
socket buffer里的数据就能在网络传输了。
sendfile会经历:3次拷贝,1次CPU copy 2次DMA copy;
以及2次上下文切换