为什么使用TCP通信?
因为在进行对接机器C语言开发的直播主机)的时候,TCP是一种更安全更高效率的连接,它是一种会确认通信对方,保持通信状态,并且能检查报文完整性的连接,而HTTP则是在TCP服务层上面的应用层,当然TCP通信稍微要复杂一点。
为什么要使用Netty?
先说一下BIO,NIO,AIO的关系:
-
BIO:是一种同步阻塞的io,当io建立连接等待应答的时候,当前的线程就被挂起不能做其他的事情。
-
NIO:是一种同步非阻塞的io,当io建立连接的时候,当前线程不会被挂起,可以去做其他的事情,它会不断去询问内核处理完消息没,当内核返回消息的时候他就可以拿着消息进行读写,其间线程并没有北挂起。
-
AIO:是一种异步非阻塞的io,当io建立连接的时候,当前线程不会被挂起,可以去做其他的事情,然而它也不会去询问消息是否出来完成,内核消息处理完成后会自动通知它,这才叫异步。
举个栗子:小明去食堂排队打饭的时候,他只能排在那里不能干其他的事情这就是BIO,当食堂实行叫号取餐的时候,小明这个时候完全不用一直等在那里,今天先去外面买瓶可乐打一把王者,他只需要时不时去看看取餐到他没有,这就是NIO,小明在寝室打吃鸡,于是叫了个外卖,餐好了会有外卖小哥送到寝室,他都不用去食堂了也不需要时不时去看是否做好了餐食,这就是AIO。
阻塞与非阻塞是看线程是否被挂起,同步与异步是看是否需要主动去询问消息已准备好
如果使用BIO这样的方式,一个线程只能应答一个请求同时还会被阻塞,不过可以采用线程池的方式来使用BIO,但是这样会造成线程的极大开销,导致系统资源匮乏,请求量过大会引起系统宕机,这时候基于NIO的Netty出来了,它首先不会北阻塞挂起,其次Netty是使用的是IO多路复用的技术,通过selector这个轮询器去轮询当前的请求,只需要一个线程就可以处理多个连接,这样能极大的提高效率
直接上需求把!!!
1,私有协议栈需求
私有协议为40字节的消息头+Json格式的消息体,如下:
2,消息封装
NettyMessage.class用于消息头+消息体的封装
public final class NettyMessage {
private Header header;
private Object body;
/**
* @return the header
*/
public final Header getHeader() {
return header;
}
/**
* @param header
* the header to set
*/
public final void setHeader(Header header) {
this.header = header;
}
/**
* @return the body
*/
public final Object getBody() {
return body;
}
/**
* @param body
* the body to set
*/
public final void setBody(Object body) {
this.body = body;
}
@Override
public String toString() {
return "NettyMessage{" +
"header=" + header +
", body=" + body +
'}';
}
}
Header.class用于消息头的封装,因为消息头里面里面数据是占固定字节的,可以使用Java的不同数据类型定义来占用固定字节
public final class Header {
/**
* 版本号占2个字节
*/
private short sVersion;
/**
* 命令类型占2个字节
*/
private short sCommand;
/**
* 包体长度占4个字节
*/
private int nPacketLen;
/**
* 命令标识,建议获取当前时间的毫秒时间+4位随机数
* 占8个字节
*/
private long uIdentity;
/**
* 具体命令,详见具体命令章节
* 占2个字节
*/
private short pCmd;
/**
* 账户名要求64位的数字
* 占8个字节
*/
private long account;
/**
* 密码要求64位的数字
* 占8个字节
*/
private long password;
/**
* szSession占六个字节
*/
private byte[] szSession;
/**
* 本地命令为0,集控命令为1
*/
private int flag;
public short getsVersion() {
return sVersion;
}
public void setsVersion(short sVersion) {
this.sVersion = sVersion;
}
public short getsCommand() {
return sCommand;
}
public void setsCommand(short sCommand) {
this.sCommand = sCommand;
}
public int getnPacketLen() {
return nPacketLen;
}
public void setnPacketLen(int nPacketLen) {
this.nPacketLen = nPacketLen;
}
public long getuIdentity() {
return uIdentity;
}
public void setuIdentity(long uIdentity) {
this.uIdentity = uIdentity;
}
public short getpCmd() {
return pCmd;
}
public void setpCmd(short pCmd) {
this.pCmd = pCmd;
}
public long getAccount() {
return account;
}
public void setAccount(long account) {
this.account = account;
}
public long getPassword() {
return password;
}
public void setPassword(long password) {
this.password = password;
}
public byte[] getSzSession() {
return szSession;
}
public void setSzSession(byte[] szSession) {
this.szSession = szSession;
}
public int getFlag() {
return flag;
}
public void setFlag(int flag) {
this.flag = flag;
}
@Override
public String toString() {
return "Header{" +
"sVersion=" + sVersion +
", sCommand=" + sCommand +
", nPacketLen=" + nPacketLen +
", uIdentity=" + uIdentity +
", pCmd=" + pCmd +
", account=" + account +
", password=" + password +
", szSession=" + Arrays.toString(szSession) +
'}';
}
}
3,序列化数据
Java默认提供的序列化机制,需要序列化的Java对象只需要实现 Serializable / Externalizable 接口并生成序列化ID,这个类就能够通过 ObjectInput 和 ObjectOutput 序列化和反序列化,不过Java的序列化效率低,而且不支持跨语言比如C,C++,目前主流的可跨语言的序列化方式有谷歌的Protobuf和Jboss的Marshalling,这里我们使用Marshalling
(1)MarshallingCodecFactory.class
public final class MarshallingCodecFactory {
/**
* 创建Jboss Marshaller
*
* @return
* @throws IOException
*/
protected static Marshaller buildMarshalling() throws IOException {
final MarshallerFactory marshallerFactory = Marshalling
.getProvidedMarshallerFactory("serial");
final MarshallingConfiguration configuration = new MarshallingConfiguration();
configuration.setVersion(5);
Marshaller marshaller = marshallerFactory
.createMarshaller(configuration);
return marshaller;
}
/**
* 创建Jboss Unmarshaller
*
* @return
* @throws IOException
*/
protected static Unmarshaller buildUnMarshalling() throws IOException {
final MarshallerFactory marshallerFactory = Marshalling
.getProvidedMarshallerFactory("serial");
final MarshallingConfiguration configuration = new MarshallingConfiguration();
configuration.setVersion(5);
final Unmarshaller unmarshaller = marshallerFactory
.createUnmarshaller(configuration);
return unmarshaller;
}
}
(2)ByteBuf的数据读写封装类
class ChannelBufferByteInput implements ByteInput {
private final ByteBuf buffer;
public ChannelBufferByteInput(ByteBuf buffer) {
this.buffer = buffer;
}
@Override
public void close() throws IOException {
// nothing to do
}
@Override
public int available() throws IOException {
return buffer.readableBytes();
}
@Override
public int read() throws IOException {
if (buffer.isReadable()) {
return buffer.readByte() & 0xff;
}
return -1;
}
@Override
public int read(byte[] array) throws IOException {
return read(array, 0, array.length);
}
@Override
public int read(byte[] dst, int dstIndex, int length) throws IOException {
int available = available();
if (available == 0) {
return -1;
}
length = Math.min(available, length);
buffer.readBytes(dst, dstIndex, length);
return length;
}
@Override
public long skip(long bytes) throws IOException {
int readable = buffer.readableBytes();
if (readable < bytes) {
bytes = readable;
}
buffer.readerIndex((int) (buffer.readerIndex() + bytes));
return bytes;
}
}
class ChannelBufferByteOutput implements ByteOutput {
private final ByteBuf buffer;
/**
* Create a new instance which use the given {@link ByteBuf}
*/
public ChannelBufferByteOutput(ByteBuf buffer) {
this.buffer = buffer;
}
@Override
public void close() throws IOException {
// Nothing to do
}
@Override
public void flush() throws IOException {
// nothing to do
}
@Override
public void write(int b) throws IOException {
buffer.writeByte(b);
}
@Override
public void write(byte[] bytes) throws IOException {
buffer.writeBytes(bytes);
}
@Override
public void write(byte[] bytes, int srcIndex, int length) throws IOException {
buffer.writeBytes(bytes, srcIndex, length);
}
/**
* Return the {@link ByteBuf} which contains the written content
*
*/
ByteBuf getBuffer() {
return buffer;
}
}
(3)MarshallingEncoder.class序列化消息体
@Sharable
public class MarshallingEncoder {
private static final byte[] LENGTH_PLACEHOLDER = new byte[4];
Marshaller marshaller;
public MarshallingEncoder() throws IOException {
marshaller = MarshallingCodecFactory.buildMarshalling();
}
protected void encode(Object msg, ByteBuf out) throws Exception {
System.out.println("-----------------编码时进行序列化--------------------");
try {
int lengthPos = out.writerIndex();
out.writeBytes(LENGTH_PLACEHOLDER);
ChannelBufferByteOutput output = new ChannelBufferByteOutput(out);
marshaller.start(output);
marshaller.writeObject(msg);
marshaller.finish();
out.setInt(lengthPos, out.writerIndex() - lengthPos - 4);
} finally {
marshaller.close();
}
}
}
(4)MarshallingDncoder.class反序列化消息体
public class MarshallingDecoder {
private final Unmarshaller unmarshaller;
/**
* Creates a new decoder whose maximum object size is {@code 1048576} bytes.
* If the size of the received object is greater than {@code 1048576} bytes,
* a {@link StreamCorruptedException} will be raised.
*
* @throws IOException
*
*/
public MarshallingDecoder() throws IOException {
unmarshaller = MarshallingCodecFactory.buildUnMarshalling();
}
protected Object decode(ByteBuf in) throws Exception {
System.out.println("-----------------解码时进行反序列化--------------------");
int objectSize = in.readInt();
ByteBuf buf = in.slice(in.readerIndex(), objectSize);
ByteInput input = new ChannelBufferByteInput(buf);
try {
unmarshaller.start(input);
Object obj = unmarshaller.readObject();
unmarshaller.finish();
in.readerIndex(in.readerIndex() + objectSize);
return obj;
} finally {
unmarshaller.close();
}
}
}
4,编码解码器
(1)NettyMessageEncoder.class消息编码器,对消息头每个参数固定字节,对消息体进行序列化
public final class NettyMessageEncoder extends MessageToByteEncoder<NettyMessage> {
MarshallingEncoder marshallingEncoder;
public NettyMessageEncoder() throws IOException {
this.marshallingEncoder = new MarshallingEncoder();
}
@Override
protected void encode(ChannelHandlerContext ctx, NettyMessage msg,ByteBuf sendBuf) throws Exception {
System.out.println("-----------------开始进行编码--------------------");
if (msg == null || msg.getHeader() == null){
throw new Exception("The encode message is null");
}else {
// 信息写入到流里面并且固定字节
sendBuf.writeShort(msg.getHeader().getsVersion());
sendBuf.writeShort(msg.getHeader().getsCommand());
sendBuf.writeInt(NettyUtil.getByteNum(msg.getBody()));
sendBuf.writeLong(msg.getHeader().getuIdentity());
sendBuf.writeShort(msg.getHeader().getpCmd());
if (msg.getHeader().getFlag()==0) {
// 本地tcp方式
msg.getHeader().setSzSession(new byte[22]) ;
sendBuf.writeBytes(msg.getHeader().getSzSession());
}else {
// 集控tcp方式
sendBuf.writeLong(msg.getHeader().getAccount());
sendBuf.writeLong(msg.getHeader().getPassword());
msg.getHeader().setSzSession(new byte[6]);
sendBuf.writeBytes(msg.getHeader().getSzSession());
}
}
if (msg.getBody() != null) {
Object body = msg.getBody();
String jsonString = JSON.toJSONString(body);
marshallingEncoder.encode(jsonString, sendBuf);
}else {
sendBuf.writeInt(0);
}
}
(2)NettyMessageDecoder.class消息编码器,对消息头每个参数进行读取,对消息体进行反序列化
public class NettyMessageDecoder extends LengthFieldBasedFrameDecoder {
MarshallingDecoder marshallingDecoder;
public NettyMessageDecoder(int maxFrameLength, int lengthFieldOffset,
int lengthFieldLength) throws IOException {
super(maxFrameLength, lengthFieldOffset, lengthFieldLength);
marshallingDecoder = new MarshallingDecoder();
}
@Override
protected Object decode(ChannelHandlerContext ctx, ByteBuf frame) throws Exception {
System.out.println("-----------------开始进行解码--------------------");
System.err.println("frame.readableBytes():"+frame.readableBytes());
NettyMessage message = new NettyMessage();
Header header = new Header();
header.setsVersion(frame.readShort());
header.setsCommand(frame.readShort());
header.setnPacketLen(frame.readInt());
header.setuIdentity(frame.readLong());
header.setpCmd(frame.readShort());
// 获取22字节的szSession
frame.readLong();
frame.readLong();
frame.readInt();
frame.readShort();
// 开始对消息包进行解码
if (frame.readableBytes() > 4) {
message.setBody(marshallingDecoder.decode(frame));
}
message.setHeader(header);
return message;
}
}
5,服务端代码
NettyServer.class用于监听端口,处理客户端请求
public class NettyServer {
public void bind(int port) throws Exception {
//bossGroup就是parentGroup,是负责处理TCP/IP连接的
EventLoopGroup bossGroup = new NioEventLoopGroup();
//workerGroup就是childGroup,是负责处理Channel(通道)的I/O事件
EventLoopGroup workerGroup = new NioEventLoopGroup();
ServerBootstrap sb = new ServerBootstrap();
sb.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
//初始化服务端可连接队列,指定了队列的大小128
.option(ChannelOption.SO_BACKLOG, 128)
//保持长连接
.childOption(ChannelOption.SO_KEEPALIVE, true)
//channel通道的处理器
.childHandler(new ServerChannelInitializer());
//绑定监听端口,调用sync同步阻塞方法等待绑定操作完
ChannelFuture future = sb.bind(port).sync();
if (future.isSuccess()) {
System.out.println("服务端启动成功");
} else {
System.out.println("服务端启动失败");
future.cause().printStackTrace();
bossGroup.shutdownGracefully(); //关闭线程组
workerGroup.shutdownGracefully();
}
//成功绑定到端口之后,给channel增加一个 管道关闭的监听器并同步阻塞,直到channel关闭,线程才会往下执行,结束进程。
future.channel().closeFuture().sync();
}
}
ServerChannelInitializer.class用于初始化channel通道,包括编码器,解码器,处理器等等
/**
* @author David
* @className ServerChannelInitializer
* @date 2020/3/23 21:36
*/
public class ServerChannelInitializer extends ChannelInitializer<SocketChannel> {
@Override
protected void initChannel(SocketChannel channel) throws Exception {
// 传输数据的解码器
channel.pipeline().addLast(new NettyMessageDecoder(1024 * 1024, 4, 4));
// 传输数据的编码器
channel.pipeline().addLast( new NettyMessageEncoder());
// 服务端收据接收处理器
channel.pipeline().addLast(new ServerHandler());
}
}
ServerHandler.class用于接收处理客户端的消息
public class ServerHandler extends ChannelInboundHandlerAdapter{
/**
* 接受client发送的消息
* @param ctx
* @param msg
* @throws Exception
*/
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println("-----------------服务器开始输出消息--------------------");
NettyMessage nettyMessage = (NettyMessage) msg;
System.out.println("接收到客户端信息:" + nettyMessage.toString());
//返回的数据结构
JsonBody jsonBody = new JsonBody();
jsonBody.setUuid(UUID.randomUUID().toString());
jsonBody.setData("hello nettyclient,这是给你的回复!");
// nettymessage
NettyMessage sendNettyMessage = new NettyMessage();
Header header = new Header();
header.setsVersion((short) 0x6635);
header.setsCommand((short) 0x2802);;
header.setuIdentity((long)1232221);
header.setnPacketLen(0);
header.setpCmd((short)0x2901);
header.setSzSession(null);
header.setAccount((long) 42343244);
header.setPassword((long) 31231213);
header.setFlag(1);
sendNettyMessage.setHeader(header);
sendNettyMessage.setBody(jsonBody);
ctx.writeAndFlush(sendNettyMessage);
}
/**
* 通知处理器最后的channelRead()是当前批处理中的最后一条消息时调用
* @param ctx
* @throws Exception
*/
@Override
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
System.out.println("服务端接收数据完毕..");
ctx.flush();
}
/**
* 读操作时捕获到异常时调用
* @param ctx
* @param cause
*/
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
ctx.close();
}
/**
* 客户端去和服务端连接成功时触发
* @param ctx
* @throws Exception
*/
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
// ctx.writeAndFlush("hello client");
}
}
6,客户端代码
NettyClient.class用于建立连接,发送数据,配置处理器
public class NettyClient {
private final String host;
private final int port;
private Channel channel;
//连接服务端的端口号地址和端口号
public NettyClient(String host, int port) {
this.host = host;
this.port = port;
}
public void start() throws Exception {
final EventLoopGroup group = new NioEventLoopGroup();
Bootstrap b = new Bootstrap();
// 使用NioSocketChannel来作为连接用的channel类
b.group(group).channel(NioSocketChannel.class)
.handler(new ClientChannelInitializer());
//发起异步连接请求,绑定连接端口和host信息
final ChannelFuture future = b.connect(host, port).sync();
future.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture arg0) throws Exception {
if (future.isSuccess()) {
System.out.println("连接服务器成功");
} else {
System.out.println("连接服务器失败");
future.cause().printStackTrace();
group.shutdownGracefully(); //关闭线程组
}
}
});
this.channel = future.channel();
}
public Channel getChannel() {
return channel;
}
ClientChannelInitializer.class用于初始化channel通道,包括编码器,解码器,处理器等等
public class ClientChannelInitializer extends ChannelInitializer<SocketChannel> {
@Override
protected void initChannel(SocketChannel channel) throws Exception {
// 解码器
channel.pipeline().addLast(new NettyMessageDecoder(1024 * 1024, 4, 4));
// 编码器
channel.pipeline().addLast( new NettyMessageEncoder());
// 处理器
channel.pipeline().addLast(new ClientHandler());
}
}
ClientHandler .class用于客户端消息处理,处理服务端返回的数据
/**
* @author david
* client消息处理类
*/
public class ClientHandler extends SimpleChannelInboundHandler<NettyMessage> {
/**
* 处理服务端返回的数据
* @param ctx
* @param msg
* @throws Exception
*/
@Override
protected void channelRead0(ChannelHandlerContext ctx, NettyMessage msg) throws Exception {
System.out.println("接受到server响应数据: " + msg.toString());
}
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
super.channelActive(ctx);
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
ctx.close();
}
}
7,测试
启动服务端
public class NettyServerStart {
public static void main(String[] args) throws Exception {
new NettyServer().bind(8080);
}
}
启动客户端
public class NettyClientStart {
public static void main(String[] args) throws Exception {
NettyClient client = new NettyClient(NettyConstant.LOCAL_IP, NettyConstant.LOCAL_PORT);
//启动client服务
client.start();
Channel channel = client.getChannel();
JsonBody jsonBody = new JsonBody();
jsonBody.setUuid("qwertyuiopuyiyiyiyiyit沪深 三分毒");
jsonBody.setData("hello nettyserver!");
// nettymessage
NettyMessage nettyMessage = new NettyMessage();
Header header = new Header();
header.setsVersion((short) 0x6635);
header.setsCommand((short) 0x2802);;
header.setuIdentity((long)1232221);
header.setpCmd((short)0x2901);
header.setSzSession(null);
header.setAccount((long) 121212151);
header.setPassword((long) 784514544);
header.setFlag(1);
nettyMessage.setHeader(header);
nettyMessage.setBody(jsonBody);
//channel对象可保存在map中,供其它地方发送消息
channel.writeAndFlush(nettyMessage);
}
}
服务端启动成功接收到消息,并返回消息,40字节的消息头没有错位
客户端发送消息,并接收服务器消息成功
总结:如上结束基于Netty的私有协议栈开发,其中还有登录模块,心跳模块,超时模块会在后面给出