Netty之ByteBuffer
目录
一、ByteBuffer的介绍
1.ByteBuffer的使用步骤
- 向buffer写入数据，例如调用channel.read(buffer)
- 调用filp()切换至读模式
- 从buffer读取数据，例如调用buffer.get()
- 调用clear()或compact()切换至写模式
- 重复步骤1~4
代码如下（示例）：
public static void main(String[] args) {
// FileChannel 代表数据的读写通道
// 1.输入输出流获取 2.RandomAccessFile获取
try (FileChannel channel = new FileInputStream("data.txt").getChannel()) {
// 1、准备缓存区
ByteBuffer buffer = ByteBuffer.allocate(10);
while (true) {
// 2、读取channl的数据 向buffer写入
int readLen = channel.read(buffer);
log.info("读取的字节：{}", readLen);
if (readLen == -1) {
break;
}
// 3、打印buffer内容
buffer.flip(); // 切换到读模式
while (buffer.hasRemaining()) {
byte b = buffer.get();//读一个字节
log.info("实际字节：{}", (char) b);
}
buffer.clear();// 切换为写模式
}
} catch (IOException e) {
}
}
打印结果如下：
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 读取的字节：10
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：1
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：2
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：3
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：4
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：5
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：6
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：7
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：8
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：9
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：0
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 读取的字节：3
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：a
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：b
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 实际字节：c
13:42:30 [INFO ] [main] c.i.m.TestByteBuffer - 读取的字节：-1
2.ByteBuffer的结构
ByteBuffer的父类有以下几个属性
- capaciity —— 缓冲区容量
- position —— 读写指针（索引下标）
- limit —— 读写限制
- mark —— 当前position的值
mark <= position <= limit <= capacity
ByteBuffer创建初始
写模式下，position是写入位置，limit等于容量，下图表示写入4个字节的状态。
读模式下，position切换为读取位置，limit切换为读取限制
读取四个字节，状态变化
切换写模式后，状态变化
compact()方法是把未读完的部分向前压缩，然后切换至写模式（比clear()方法更耗性能）
代码如下（示例）：
public class TestByteBuffer {
public static void main(String[] args) {
ByteBuffer buffer = ByteBuffer.allocate(10);
// 向buffer中写入1个字节的数据
buffer.put((byte)97);
// 使用工具类，查看buffer状态
ByteBufferUtil.debugAll(buffer);
// 向buffer中写入4个字节的数据
buffer.put(new byte[]{98, 99, 100, 101});
ByteBufferUtil.debugAll(buffer);
// 获取数据
buffer.flip();
ByteBufferUtil.debugAll(buffer);
System.out.println(buffer.get());
System.out.println(buffer.get());
ByteBufferUtil.debugAll(buffer);
// 使用compact切换模式
buffer.compact();
ByteBufferUtil.debugAll(buffer);
// 再次写入
buffer.put((byte)102);
buffer.put((byte)103);
ByteBufferUtil.debugAll(buffer);
}
}
打印结果如下：
// 向缓冲区写入了一个字节的数据，此时postition为1
+--------+-------------------- all ------------------------+----------------+
position: [1], limit: [10]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 61 00 00 00 00 00 00 00 00 00 |a......... |
+--------+-------------------------------------------------+----------------+
// 向缓冲区写入四个字节的数据，此时position为5
+--------+-------------------- all ------------------------+----------------+
position: [5], limit: [10]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 61 62 63 64 65 00 00 00 00 00 |abcde..... |
+--------+-------------------------------------------------+----------------+
// 调用flip切换模式，此时position为0，表示从第0个数据开始读取
+--------+-------------------- all ------------------------+----------------+
position: [0], limit: [5]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 61 62 63 64 65 00 00 00 00 00 |abcde..... |
+--------+-------------------------------------------------+----------------+
// 读取两个字节的数据
97
98
// position变为2
+--------+-------------------- all ------------------------+----------------+
position: [2], limit: [5]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 61 62 63 64 65 00 00 00 00 00 |abcde..... |
+--------+-------------------------------------------------+----------------+
// 调用compact切换模式，此时position及其后面的数据被压缩到ByteBuffer前面去了
// 此时position为3，会覆盖之前的数据
+--------+-------------------- all ------------------------+----------------+
position: [3], limit: [10]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 63 64 65 64 65 00 00 00 00 00 |cdede..... |
+--------+-------------------------------------------------+----------------+
// 再次写入两个字节的数据，之前的 0x64 0x65 被覆盖
+--------+-------------------- all ------------------------+----------------+
position: [5], limit: [10]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 63 64 65 66 67 00 00 00 00 00 |cdefg..... |
+--------+-------------------------------------------------+----------------+
3.ByteBuffer常见方法
3.1 分配空间
ByteBuffer.allocate(16); // HeapByteBuffer -java堆内存，读写效率低，受GC影响
ByteBuffer.allocateDirect(16); //DirectByteBuffer -系统内存，因为少一次拷贝，读写效率高，不受GC影响，但是分配内存的效率低，，使用不当会造成内存泄漏
3.2 向buffer写入数据
调用channel的read方法
int readBytes = channel.read(buf);
调用buffer的put方法
buf.put((byte)127);
3.3 从buffer读取数据
调用channel的write方法
int readBytes = channel.write(buf);
调用buffer的get方法
byte b = buf.get();
get方法会让position读指针向后走，若是重读数据可调用rewind()方法将position重置为0或者调用get(i)方法获取索引i的内容，它不会移动读指针
rewind() 从头开始读
mark() 对当前position的位置索引做标记
reset() 将position重置到mark位置
get(i) 不会改变读索引的位置
3.4 字符串与ByteBuffer相互转换
3.4.1 字符串转ByteBuffer
代码如下（示例）：
String str = "hello";
// 1.字符串转ByteBuffer
ByteBuffer buffer = ByteBuffer.allocate(16);
// 通过字符串的getByte方法获得字节数组，放入缓冲区中
buffer.put(str.getBytes());
ByteBufferUtil.debugAll(buffer);
// 2.Charset
ByteBuffer buffer2 = StandardCharsets.UTF_8.encode(str);
ByteBufferUtil.debugAll(buffer2);
// 3.wrap
ByteBuffer buffer3 = ByteBuffer.wrap(str.getBytes());
ByteBufferUtil.debugAll(buffer3);
3.4.2 ByteBuffer转字符串
代码如下（示例）：
String str = "hello";
ByteBuffer buffer = ByteBuffer.allocate(16);
ByteBuffer buffer2 = StandardCharsets.UTF_8.encode(str);
String str1 = StandardCharsets.UTF_8.decode(buffer2).toString();
buffer.flip();
String str2 = StandardCharsets.UTF_8.decode(buffer).toString();
3.5 Scattering Reads——分散读取
代码如下（示例）：
public static void main(String[] args) {
try (FileChannel fileChannel = new RandomAccessFile("test.txt", "r").getChannel()){
ByteBuffer b1 = ByteBuffer.allocate(3);
ByteBuffer b2 = ByteBuffer.allocate(3);
ByteBuffer b3 = ByteBuffer.allocate(3);
fileChannel.read(new ByteBuffer[]{b1,b2,b3});
b1.flip();
b2.flip();
b3.flip();
debugAll(b1);
debugAll(b2);
debugAll(b3);
} catch (IOException e) {
e.printStackTrace();
}
}
3.6 Gathering Writes——集中写入
代码如下（示例）：
public static void main(String[] args) {
ByteBuffer b1 = StandardCharsets.UTF_8.encode("hello");
ByteBuffer b2 = StandardCharsets.UTF_8.encode("world");
ByteBuffer b3 = StandardCharsets.UTF_8.encode("你好");
try (FileChannel fileChannel = new RandomAccessFile("test.txt", "r").getChannel()){
fileChannel.write(new ByteBuffer[]{b1,b2,b3});
} catch (IOException e) {
e.printStackTrace();
}
}
4. 粘包与半包
- 粘包
发送方在发送数据时，并不是一条一条地发送数据，而是将数据整合在一起，当数据达到一定的数量后再一起发送。这就会导致多条信息被放在一个缓冲区中被一起发送出去。
- 半包
接收方的缓冲区的大小是有限的，当接收方的缓冲区满了以后，就需要将信息截断，等缓冲区空了以后再继续放入数据。这就会发生一段完整的数据最后被截断的现象
代码如下（示例）：
public class ByteBufferDemo {
public static void main(String[] args) {
ByteBuffer buffer = ByteBuffer.allocate(32);
// 模拟粘包+半包
buffer.put("Hello,world\nI'm Nyima\nHo".getBytes());
// 调用split函数处理
split(buffer);
buffer.put("w are you?\n".getBytes());
split(buffer);
}
private static void split(ByteBuffer buffer) {
// 切换为读模式
buffer.flip();
for(int i = 0; i < buffer.limit(); i++) {
// 遍历寻找分隔符
// get(i)不会移动position
if (buffer.get(i) == '\n') {
// 缓冲区长度
int length = i+1-buffer.position();
ByteBuffer target = ByteBuffer.allocate(length);
// 将前面的内容写入target缓冲区
for(int j = 0; j < length; j++) {
// 将buffer中的数据写入target中
target.put(buffer.get());
}
// 打印查看结果
ByteBufferUtil.debugAll(target);
}
}
// 切换为写模式，但是缓冲区可能未读完，这里需要使用compact
buffer.compact();
}
}
打印结果如下：
+--------+-------------------- all ------------------------+----------------+
position: [12], limit: [12]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 65 6c 6c 6f 2c 77 6f 72 6c 64 0a |Hello,world. |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [10], limit: [10]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 49 27 6d 20 4e 79 69 6d 61 0a |I'm Nyima. |
+--------+-------------------------------------------------+----------------+
+--------+-------------------- all ------------------------+----------------+
position: [13], limit: [13]
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 6f 77 20 61 72 65 20 79 6f 75 3f 0a |How are you?. |
+--------+-------------------------------------------------+----------------+
二、代码示例
1.引入依赖
代码如下（示例）：
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.36.Final</version>
</dependency>
2.ByteBuffer工具类
代码如下（示例）：
import java.nio.ByteBuffer;
import io.netty.util.internal.MathUtil;
import io.netty.util.internal.StringUtil;
import io.netty.util.internal.MathUtil.*;
public class ByteBufferUtil {
private static final char[] BYTE2CHAR = new char[256];
private static final char[] HEXDUMP_TABLE = new char[256 * 4];
private static final String[] HEXPADDING = new String[16];
private static final String[] HEXDUMP_ROWPREFIXES = new String[65536 >>> 4];
private static final String[] BYTE2HEX = new String[256];
private static final String[] BYTEPADDING = new String[16];
static {
final char[] DIGITS = "0123456789abcdef".toCharArray();
for (int i = 0; i < 256; i++) {
HEXDUMP_TABLE[i << 1] = DIGITS[i >>> 4 & 0x0F];
HEXDUMP_TABLE[(i << 1) + 1] = DIGITS[i & 0x0F];
}
int i;
// Generate the lookup table for hex dump paddings
for (i = 0; i < HEXPADDING.length; i++) {
int padding = HEXPADDING.length - i;
StringBuilder buf = new StringBuilder(padding * 3);
for (int j = 0; j < padding; j++) {
buf.append(" ");
}
HEXPADDING[i] = buf.toString();
}
// Generate the lookup table for the start-offset header in each row (up to 64KiB).
for (i = 0; i < HEXDUMP_ROWPREFIXES.length; i++) {
StringBuilder buf = new StringBuilder(12);
buf.append(StringUtil.NEWLINE);
buf.append(Long.toHexString(i << 4 & 0xFFFFFFFFL | 0x100000000L));
buf.setCharAt(buf.length() - 9, '|');
buf.append('|');
HEXDUMP_ROWPREFIXES[i] = buf.toString();
}
// Generate the lookup table for byte-to-hex-dump conversion
for (i = 0; i < BYTE2HEX.length; i++) {
BYTE2HEX[i] = ' ' + StringUtil.byteToHexStringPadded(i);
}
// Generate the lookup table for byte dump paddings
for (i = 0; i < BYTEPADDING.length; i++) {
int padding = BYTEPADDING.length - i;
StringBuilder buf = new StringBuilder(padding);
for (int j = 0; j < padding; j++) {
buf.append(' ');
}
BYTEPADDING[i] = buf.toString();
}
// Generate the lookup table for byte-to-char conversion
for (i = 0; i < BYTE2CHAR.length; i++) {
if (i <= 0x1f || i >= 0x7f) {
BYTE2CHAR[i] = '.';
} else {
BYTE2CHAR[i] = (char) i;
}
}
}
/**
* 打印所有内容
* @param buffer
*/
public static void debugAll(ByteBuffer buffer) {
int oldlimit = buffer.limit();
buffer.limit(buffer.capacity());
StringBuilder origin = new StringBuilder(256);
appendPrettyHexDump(origin, buffer, 0, buffer.capacity());
System.out.println("+--------+-------------------- all ------------------------+----------------+");
System.out.printf("position: [%d], limit: [%d]\n", buffer.position(), oldlimit);
System.out.println(origin);
buffer.limit(oldlimit);
}
/**
* 打印可读取内容
* @param buffer
*/
public static void debugRead(ByteBuffer buffer) {
StringBuilder builder = new StringBuilder(256);
appendPrettyHexDump(builder, buffer, buffer.position(), buffer.limit() - buffer.position());
System.out.println("+--------+-------------------- read -----------------------+----------------+");
System.out.printf("position: [%d], limit: [%d]\n", buffer.position(), buffer.limit());
System.out.println(builder);
}
private static void appendPrettyHexDump(StringBuilder dump, ByteBuffer buf, int offset, int length) {
if (MathUtil.isOutOfBounds(offset, length, buf.capacity())) {
throw new IndexOutOfBoundsException(
"expected: " + "0 <= offset(" + offset + ") <= offset + length(" + length
+ ") <= " + "buf.capacity(" + buf.capacity() + ')');
}
if (length == 0) {
return;
}
dump.append(
" +-------------------------------------------------+" +
StringUtil.NEWLINE + " | 0 1 2 3 4 5 6 7 8 9 a b c d e f |" +
StringUtil.NEWLINE + "+--------+-------------------------------------------------+----------------+");
final int startIndex = offset;
final int fullRows = length >>> 4;
final int remainder = length & 0xF;
// Dump the rows which have 16 bytes.
for (int row = 0; row < fullRows; row++) {
int rowStartIndex = (row << 4) + startIndex;
// Per-row prefix.
appendHexDumpRowPrefix(dump, row, rowStartIndex);
// Hex dump
int rowEndIndex = rowStartIndex + 16;
for (int j = rowStartIndex; j < rowEndIndex; j++) {
dump.append(BYTE2HEX[getUnsignedByte(buf, j)]);
}
dump.append(" |");
// ASCII dump
for (int j = rowStartIndex; j < rowEndIndex; j++) {
dump.append(BYTE2CHAR[getUnsignedByte(buf, j)]);
}
dump.append('|');
}
// Dump the last row which has less than 16 bytes.
if (remainder != 0) {
int rowStartIndex = (fullRows << 4) + startIndex;
appendHexDumpRowPrefix(dump, fullRows, rowStartIndex);
// Hex dump
int rowEndIndex = rowStartIndex + remainder;
for (int j = rowStartIndex; j < rowEndIndex; j++) {
dump.append(BYTE2HEX[getUnsignedByte(buf, j)]);
}
dump.append(HEXPADDING[remainder]);
dump.append(" |");
// Ascii dump
for (int j = rowStartIndex; j < rowEndIndex; j++) {
dump.append(BYTE2CHAR[getUnsignedByte(buf, j)]);
}
dump.append(BYTEPADDING[remainder]);
dump.append('|');
}
dump.append(StringUtil.NEWLINE +
"+--------+-------------------------------------------------+----------------+");
}
private static void appendHexDumpRowPrefix(StringBuilder dump, int row, int rowStartIndex) {
if (row < HEXDUMP_ROWPREFIXES.length) {
dump.append(HEXDUMP_ROWPREFIXES[row]);
} else {
dump.append(StringUtil.NEWLINE);
dump.append(Long.toHexString(rowStartIndex & 0xFFFFFFFFL | 0x100000000L));
dump.setCharAt(dump.length() - 9, '|');
dump.append('|');
}
}
public static short getUnsignedByte(ByteBuffer buffer, int index) {
return (short) (buffer.get(index) & 0xFF);
}
}
总结
介绍了ByteBuffer的结构、使用步骤、常见方法和粘包、半包的概念。
1166
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
