web编程(5) Netty

sjzwangxufeng

已于 2024-03-28 17:06:01 修改

阅读量114

点赞数

分类专栏： web编程文章标签： java tcp nio spring

于 2023-11-11 19:47:33 首次发布

本文链接：https://blog.csdn.net/sjzwangxufeng/article/details/134326713

版权

web编程专栏收录该内容

7 篇文章 0 订阅

订阅专栏

1 概述

Netty是目前最流行的NIO框架之一，不管是功能、性能、健壮性，还是可定制、可扩展方面，表现都很卓越，尤其在高性能通讯方面，已逐渐成为业界主流，被广泛应用，例如阿里的分布式服务框架Dubbo，RocketMQ、Hadoop的Avro，以及spring 5.0 引入的全新技术框架WebFlux等，可以说，它基本上已是Java NIO 编程的首选框架。

通过个人实践、观察，netty之所以这么流行，除本身过硬的品质外，整个设计让人感触最深的是，它把复杂自己封装起来，把简单留给用户。

2 流程图

通过配置不同的启动参数，它可以支持Reactor单线程、多线程，以及主从多线程模型，以下为官方推荐的主从线程模型处理流程框图。

补充说明:

a. NioEventLoopGroup的线程池默认为CPU内核数的2倍。
b. 当一个TCP连接握手成功后，会在Work线程池，随机选择一个线程进行读写事件注册处理，也就是说SocketChannel会一直从属于一个I/O线程，不会出现多个线程处理一个SocketChannel的情况。
c. 每个SocketChannel连接，都会有一个的ChannelPipeline实例与其一一对应。
d. ChannelPipeline是ChannelHandler容器，它通过维护ChannelHandlerContext链表，来管理ChannelHandler。
f. 每个ChannelHandler，都会有一个的ChannelHandlerContext实例与其一一对应。

3 核心类

主要分4个方面: I/O线程、Channel（Socket）、Handler、ByteBuf(内存)

3.1 I/O线程

NioEventLoopGroup类

服务端启动时，通常创建两个NioEventLoopGroup，它们实际是两独立的线程池，也就是常说的boss、work两个线程池，一个用于负责接收客户端的TCP连接，处理OP_ACCEPT事件，另一个负责处理I/O相关的读写操作，也就是处理OP_READ、OP_WRITE事件，以及执行各种task任务。

3.2 Channel（Socket）

Channel是Netty自己抽象的网络I/O处理接口，不同于JDK原生的Channel，更通俗说，就是非阻塞的Socket，主要有NioServerSocketChannel、NioSocketChannel实现类。

NioServerSocketChannel 用于服务端，监听网络端口，处理客户端连接请求。

NioSocketChannel 一旦服务端accept客户端请求，就会形成一个独立的通讯通道，由该实例来处理。后端所有的 handler 都是围绕着该通道，进行I/O 读写处理。

3.3 Handler

Handler是一组类，包含ChannelPipeline、ChannelHandler、ChannelHandlerContext等，其中ChannelHandler又分为ChannelInboundHandler、ChannelOutboundHandler两类。

ChannelPipeline是ChannelHandler的容器，管理ChannelHandler处理的前后关系。

ChannelInboundHandler 负责I/O读相关处理，对请求数据进行解码，提交给业务处理层。

ChannelOutboundHandler 负责I/O写相关处理，对响应数据按协议编码，最终响应给客户端。

ChannelHandlerContext 辅助类，跟ChannelHandler是一一对应的关系，Handler的处理结果，都是通过它进行协调处理。

3.4 ByteBuf(内存)

JDK 原生NIO的数据读写操作，都是通过缓存区ByteBuffer类处理，ByteBuf是Netty自己的缓存区实现，其中很多接口方法跟ByteBuffer类似，依据使用场景，分类如下:

UnpooledHeapByteBuf 基于堆内存进行分配的缓存区，每次I/O读写，都会创建新的实例，相比堆外内存的申请、释放，它的成本较低，因此，在满足性能的情况，它是推荐的处理缓存区方式。

PooledHeapByteBuf 带有池的堆内存，在某些方面可以节约内存的创建开销，但它内部的内存管理也更复杂。

UnPooledDirectByteBuf 基于直接内存，也就是非堆内存，它的内存分配、回收慢一些，但对I/O的读写少了一次内存复制，速度比堆内存快。

PooledDirectByteBuf 基于内存池实现，跟UnPooledDirectByteBuf功能一样，只是内存分配策略不同。

4 功能类

4.1 TCP粘包、拆包

TCP是个基于字节流的协议，在网络中按IP分组包进行数据传递，依据TCP发送缓存区的实际情况，进行包的划分，在业务层认为的的一个完整数据包，可能会分多个包传递，也可能多个小包被合并成一大的数据包传递，这就是所谓的TCP粘包、拆包问题。

业界主流解决方案，大致可归纳为3种，Netty都已封装实现，如下:

(1) DelimiterBasedFrameDecoder、LineBasedFrameDecoder 基于分割符的数据帧格式，数据包间以约定的分割负划分，例如"$-$"、“@%@”等，其中LineBasedFrameDecoder 是以"\r\n"固定封装好的解码类。

(2) FixedLengthFrameDecoder 是固定长数据帧格式解码类，例如100字节，如果不足，用空位补齐。

(3) LengthFieldBasedFrameDecoder自定义长度帧解码器，将数据包分数据头、数据体两部分，

数据头包含数据长度等信息，类似于http-post协议中的header部分的 Content-Length。该种方案更灵活，通常自定义协议采用该种方式。

4.2 编解码

netty内置了丰富的编解码器，例如http、websocket、Protobuf、Marshaller等，在"6 案例"部分，有演示的小例子。

5 配置

由ServerBootstrap服务引导类处理，主要涉及线程池、TCP参数、ChannelHandler，以下只介绍前两项。

5.1 线程池

		// 配置服务端的NIO线程组
		EventLoopGroup bossGroup = new NioEventLoopGroup();
		EventLoopGroup workerGroup = new NioEventLoopGroup();

		// 服务引导类
		ServerBootstrap b = new ServerBootstrap();
		
		// 单线程池
		b.group(bossGroup);
		// 主从线程池(推荐)
		b.group(bossGroup, workerGroup);

5.2 TCP参数

作为服务端，主要是BACKLOG参数的设置，其他还包括读写缓存区、NODELAY、超时等设置，ChannelOption类已内置定义这些参数（以下仅部分举例）:

    public static final ChannelOption<Integer> SO_SNDBUF = valueOf("SO_SNDBUF");
    public static final ChannelOption<Integer> SO_RCVBUF = valueOf("SO_RCVBUF");

    public static final ChannelOption<Integer> SO_BACKLOG = valueOf("SO_BACKLOG");
    public static final ChannelOption<Integer> SO_TIMEOUT = valueOf("SO_TIMEOUT");

    public static final ChannelOption<Boolean> TCP_NODELAY = valueOf("TCP_NODELAY");

通常只配置BACKLOG、NODELAY就可以，如下:

			ServerBootstrap b = new ServerBootstrap();
			
			b.group(bossGroup, workerGroup)
			.channel(NioServerSocketChannel.class)
			.option(ChannelOption.SO_BACKLOG, 100)
			.option(ChannelOption.TCP_NODELAY, false)

BACKLOG 是内核中socket排队的最大连接数，依据TCP的3次握手过程，分为两队列未完成队列（收到客户端SYN时）、已完成队列(收到客户端SYN-ACK时，成功握手)，BACKLOG为两个队列之和最大值。

NODELAY 是否延时，通常选false，由Socket依据发送缓存区实际情况，进行分组发送数据。

5.3 依赖

		<dependency>
		    <groupId>io.netty</groupId>
		    <artifactId>netty-all</artifactId>
		    <version>4.0.56.Final</version>
		</dependency>

6 案例

6.1 简单例子（客户、服务端）

下面是一个包含Netty客户端、服务端的小例子，通讯协议采用最简单的回车换行格式，可以看到例子采用LineBasedFrameDecoder来解决tcp粘包、拆包问题。

netty服务端：启动类

public class DemoServer {

	public static void main(String[] args) throws Exception {
		
		new DemoServer().bind(8080);
	}
	
	public void bind(int port) throws Exception {

		EventLoopGroup bossGroup = new NioEventLoopGroup();
		EventLoopGroup workerGroup = new NioEventLoopGroup();
		try {
			ServerBootstrap b = new ServerBootstrap();
			
			b.group(bossGroup, workerGroup)
			.channel(NioServerSocketChannel.class)
			.option(ChannelOption.SO_BACKLOG, 100)
			
			.childHandler(new ChildChannelHandler());
			
			// 同步等待成功
			ChannelFuture f = b.bind(port).sync();			
			System.out.println("DemoServer start success, port: " + port);

			// 监听端口关闭
			f.channel().closeFuture().sync();
		} finally {
			// 释放线程池资源
			bossGroup.shutdownGracefully();
			workerGroup.shutdownGracefully();
		}
	}

	private class ChildChannelHandler extends ChannelInitializer<SocketChannel> {
		@Override
		protected void initChannel(SocketChannel arg0) throws Exception {
			arg0.pipeline().addLast(new LineBasedFrameDecoder(1024));
			arg0.pipeline().addLast(new StringDecoder());
			arg0.pipeline().addLast(new DemoServerHandler());
			
			System.out.println("initChannel:pipeline");
		}
	}	
}

netty服务端：Handler类

public class DemoServerHandler extends ChannelInboundHandlerAdapter {
	
	private static final Logger logger = Logger.getLogger(DemoServerHandler.class.getName());
	
	private static final AtomicInteger counter = new AtomicInteger();
		
	@Override
	public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
		String reqMsg = (String) msg;
		logger.info("netty-server request : " + reqMsg);
		
		String resMsg = reqMsg + " pong " + counter.addAndGet(1) + System.getProperty("line.separator");
		ByteBuf resp = Unpooled.copiedBuffer(resMsg.getBytes());
		ctx.writeAndFlush(resp);
	}

	@Override
	public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
		logger.warning("netty-server exceptionCaught : " + cause.getMessage());
		ctx.close();
	}	

}

netty客户端：启动类

public class DemoClient {

	public static void main(String[] args) throws Exception {
		new DemoClient().connect(8080, "127.0.0.1");
	}
	
	public void connect(int port, String host) throws Exception {

		EventLoopGroup group = new NioEventLoopGroup();
		try {
			Bootstrap b = new Bootstrap();
			b.group(group)
			.channel(NioSocketChannel.class)
			.option(ChannelOption.TCP_NODELAY, false)
			.handler(new ChildChannelHandler());

			// 同步等待成功
			ChannelFuture f = b.connect(host, port).sync();
			System.out.println(String.format("DemoClient start success, host: %s, port: %s", host, port));

			// 链路关闭
			f.channel().closeFuture().sync();
		} finally {
			// 释放线程池资源
			group.shutdownGracefully();
		}
	}
	
	private class ChildChannelHandler extends ChannelInitializer<SocketChannel> {
		@Override
		protected void initChannel(SocketChannel arg0) throws Exception {
			arg0.pipeline().addLast(new LineBasedFrameDecoder(1024));
			arg0.pipeline().addLast(new StringDecoder());
			arg0.pipeline().addLast(new DemoClientHandler());
			
			System.out.println("initChannel:pipeline");
		}
	}
}

netty客户端：Handler类

public class DemoClientHandler extends ChannelInboundHandlerAdapter {
	
	private static final Logger logger = Logger.getLogger(DemoClientHandler.class.getName());
	
	@Override
	public void channelActive(ChannelHandlerContext ctx) {
		
		byte[] msg = ("hello-ping" + System.getProperty("line.separator")).getBytes();
		
		ByteBuf buf = Unpooled.buffer(msg.length);;
		buf.writeBytes(msg);
		ctx.writeAndFlush(buf);
	}
	
	@Override
	public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
		String body = (String) msg;
		logger.info("netty-client response : " + body);
	}

	@Override
	public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
		
		logger.warning("netty-client exceptionCaught : " + cause.getMessage());
		ctx.close();
	}

}

6.2 http服务

一个http服务小例子，测试链接：http://localhost:8080/demo/hello

netty服务端：启动类

public class HttpServer {

	public static void main(String[] args) throws Exception {
		
		new HttpServer().bind(8080);
	}
	
	public void bind(int port) throws Exception {

		EventLoopGroup bossGroup = new NioEventLoopGroup();
		EventLoopGroup workerGroup = new NioEventLoopGroup();
		try {
			ServerBootstrap b = new ServerBootstrap();
			
			b.group(bossGroup, workerGroup)
			.channel(NioServerSocketChannel.class)
			.option(ChannelOption.SO_BACKLOG, 100)
			
			.childHandler(new ChildChannelHandler());
			
			// 同步等待成功
			ChannelFuture f = b.bind(port).sync();			
			System.out.println("HttpServer start success, port: " + port);

			// 监听端口关闭
			f.channel().closeFuture().sync();
		} finally {
			// 释放线程池资源
			bossGroup.shutdownGracefully();
			workerGroup.shutdownGracefully();
		}
	}

	private class ChildChannelHandler extends ChannelInitializer<SocketChannel> {
		@Override
		protected void initChannel(SocketChannel sc) throws Exception {
			sc.pipeline().addLast("http-decoder", new HttpRequestDecoder());
			// 目的是将多个消息转换为单一的request或者response对象
			sc.pipeline().addLast("http-aggregator", new HttpObjectAggregator(65536));
			sc.pipeline().addLast("http-encoder", new HttpResponseEncoder());
			// 目的是支持异步大文件传输（）
			sc.pipeline().addLast("http-chunked", new ChunkedWriteHandler());
			sc.pipeline().addLast("hello-handler", new HelloChannelHandler());			
			
			System.out.println("initChannel:pipeline");
		}
	}	
}

netty服务端：Handler类

public class HelloChannelHandler extends SimpleChannelInboundHandler<FullHttpRequest> {
	
	private final String GET_URL_HELLO = "/demo/hello"; 

	@Override
	protected void channelRead0(ChannelHandlerContext ctx, FullHttpRequest request) throws Exception {
		
		if (!request.getDecoderResult().isSuccess()) {
			sendError(ctx, BAD_REQUEST);
			return;
		}
		if (request.getMethod() != GET) {
			sendError(ctx, METHOD_NOT_ALLOWED);
			return;
		}
		final String uri = request.getUri();
		if (!GET_URL_HELLO.equals(uri)) {
			sendError(ctx, FORBIDDEN);
			return;			
		}
		
		sendHello(ctx, "hello, baby!");		
	}
	
	@Override
	public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
		cause.printStackTrace();
		if (ctx.channel().isActive()) {
			sendError(ctx, INTERNAL_SERVER_ERROR);
		}
	}	
	
	private static void sendError(ChannelHandlerContext ctx, HttpResponseStatus status) {
		FullHttpResponse response = new DefaultFullHttpResponse(HTTP_1_1, status,
				Unpooled.copiedBuffer("Failure: " + status.toString() + "\r\n", CharsetUtil.UTF_8));
		response.headers().set(CONTENT_TYPE, "text/plain; charset=UTF-8");
		ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE);
	}
	
	private static void sendHello(ChannelHandlerContext ctx, String msg) {
		FullHttpResponse response = new DefaultFullHttpResponse(HTTP_1_1, OK);
		response.headers().set(CONTENT_TYPE, "text/plain; charset=UTF-8");
				
		ByteBuf buffer = Unpooled.copiedBuffer(msg + "\r\n", CharsetUtil.UTF_8);
		response.content().writeBytes(buffer);
		buffer.release();
		ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE);
	}	

}

6.3 模拟MVC（Protobuf）

实施思路，跟spring mvc 的DispatcherServlet是一样的，不同之处就是，mvc 通过请求url与Controller的@RequestMapping方法对应，而Protobuf协议，通过数据包 reqCode 与Controller的自定义注解@PbRequestMapping方法对应，启动时HandlerMethodMapping扫描所有HandlerMethod实例方法，维护reqCode与HandlerMethod匹配关系，以下是源码：

6.3.1 pom依赖

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.mutouren</groupId>
  <artifactId>mtr-demo-base-blog-netty-mvc</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>mtr-demo-base-blog-netty-mvc</name>
  
	<properties>
		<jdk.version>1.8</jdk.version>
	</properties>

	<dependencies>
	
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter</artifactId>
		</dependency>		
	
 		<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-lang3</artifactId>
		</dependency>
				
		<dependency>
		    <groupId>io.netty</groupId>
		    <artifactId>netty-all</artifactId>
		    <version>4.0.56.Final</version>
		</dependency>		
        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>2.6.1</version>
        </dependency>			

	</dependencies>
	
	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-dependencies</artifactId>
				<version>2.3.2.RELEASE</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>	

	<build>
		<plugins>
			<plugin>
				<artifactId>maven-compiler-plugin</artifactId>
				<version>3.5.1</version>
				<configuration>
					<source>${jdk.version}</source>
					<target>${jdk.version}</target>
					<encoding>UTF-8</encoding>
				</configuration>
			</plugin>
		</plugins>
	</build>
</project>

6.3.2 Protobuf文件

Request.proto

option java_package = "mtr.demo.netty.model";
option java_outer_classname = "RequestProto";

message Request {

 	required int32 pd_type = 1; 			// Message类型
 	required int32 pd_req_code = 2; 		// 请求编号

	required int64 pd_timestamp = 3;		// 时间戳
 	required bytes pd_data = 4; 			// 内容
}

Response.proto

option java_package = "mtr.demo.netty.model";
option java_outer_classname = "ResponseProto";

message Response {

 	required int32 pd_type = 1; 			// Message类型
 	required int32 pd_res_code = 2; 		// 响应编号
 	
 	required int32 pd_state = 3; 			// 响应状态
 	optional string pd_message = 4; 		// 响应信息

	optional int64 pd_timestamp = 5;		// 时间戳
 	optional bytes pd_data = 6; 			// 内容
}

ReqMessgeA.proto

option java_package = "mtr.demo.netty.model";
option java_outer_classname = "ReqMessageAProto";

message ReqMessgeA {

 	required int32 pd_id = 1; 				// 消息id
 	optional string pd_desc = 2; 			// 描述
}

ReqMessgeB.proto

option java_package = "mtr.demo.netty.model";
option java_outer_classname = "ReqMessageBProto";

message ReqMessgeB {

 	required int32 pd_id = 1; 				// 消息id
	required string pd_name = 2; 			// 名称
 	optional string pd_desc = 3; 			// 描述
}

ResMessgeAB.proto

option java_package = "mtr.demo.netty.model";
option java_outer_classname = "ResMessageABProto";

message ResMessgeAB {

 	required int32 pd_id = 1; 				// 消息id
	required string pd_content = 2; 		// 内容
 	optional string pd_desc = 3; 			// 描述
}

6.3.3 服务端

（1）启动类

@SpringBootApplication
public class Application {

	public static void main(String[] args) throws Exception {
		
        SpringApplication springApplication = new SpringApplication(Application.class);
        springApplication.setWebApplicationType(WebApplicationType.NONE);
        ApplicationContext ctx = springApplication.run(args);

        NettyServer.bind(ctx, 8080);
	}
}

public class NettyServer {
	
	private static final Logger logger = LoggerFactory.getLogger(NettyServer.class);

	public static void bind(ApplicationContext context, int port) throws Exception {

		EventLoopGroup bossGroup = new NioEventLoopGroup();
		EventLoopGroup workerGroup = new NioEventLoopGroup();
		try {
			ServerBootstrap b = new ServerBootstrap();
			
			b.group(bossGroup, workerGroup)
			.channel(NioServerSocketChannel.class)
			.option(ChannelOption.SO_BACKLOG, 100)
			
			.childHandler(new ChildChannelHandler(context));
			
			// 同步等待成功
			ChannelFuture f = b.bind(port).sync();			
			logger.info("NettyServer start success, port: {}", port);

			// 监听端口关闭
			f.channel().closeFuture().sync();
		} finally {
			// 释放线程池资源
			bossGroup.shutdownGracefully();
			workerGroup.shutdownGracefully();
		}
	}

	private static class ChildChannelHandler extends ChannelInitializer<SocketChannel> {
		
		private final ApplicationContext context;
		
		public ChildChannelHandler(ApplicationContext context) {
			this.context = context;
		}
		
		@Override
		protected void initChannel(SocketChannel ch) throws Exception {
			ch.pipeline().addLast(new ProtobufVarint32FrameDecoder());							
			ch.pipeline().addLast(new ProtobufDecoder(RequestProto.Request.getDefaultInstance()));
			ch.pipeline().addLast(new ProtobufVarint32LengthFieldPrepender());
			ch.pipeline().addLast(new ProtobufEncoder());						
			ch.pipeline().addLast(new DispatcherChannelHandler(context));
			
			System.out.println("initChannel:pipeline");
		}
	}
}

（2）Controller类

@Controller
public class DemoController {
	
	private static final Logger logger = LoggerFactory.getLogger(DemoController.class);
	
	@PbRequestMapping(reqCode=Constant.REQ_MESSAGE_A)
	public Response processA(Request request, ReqMessgeA reqMessgeA) {
		
		logger.info("processA: {}", reqMessgeA);		
		
		ResMessageABProto.ResMessgeAB.Builder builder = ResMessageABProto.ResMessgeAB.newBuilder();
		builder.setPdId(1);
		builder.setPdContent("aa");
		builder.setPdDesc("111");
		
		return ResponseHelper.buildResponse(Constant.RES_STATE_SUCCESS, builder.build().toByteArray(), request);
	}
	
	@PbRequestMapping(reqCode=Constant.REQ_MESSAGE_B)
	public Response processB(Request request, ReqMessgeB reqMessgeB, @UserId Integer userId) {
		
		logger.info("processB: {}, userId:{}", reqMessgeB, userId);
		
		ResMessageABProto.ResMessgeAB.Builder builder = ResMessageABProto.ResMessgeAB.newBuilder();
		builder.setPdId(2);
		builder.setPdContent("bb");
		builder.setPdDesc("222");
		
		return ResponseHelper.buildResponse(Constant.RES_STATE_SUCCESS, builder.build().toByteArray(), request);
	}
}

（3）Dispatcher相关

DispatcherChannelHandler类

public class DispatcherChannelHandler extends ChannelInboundHandlerAdapter {
	
	private static final Logger logger = LoggerFactory.getLogger(DispatcherChannelHandler.class);
	
	private final HandlerMethodMapping handlerMethodMapping;
	
	public DispatcherChannelHandler(ApplicationContext context) {
		this.handlerMethodMapping = context.getBean(HandlerMethodMapping.class);
	}
	
	@Override
	public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
		RequestProto.Request req = (RequestProto.Request) msg;

		switch (req.getPdType()) {
		case Constant.REQ_TYPE_BIZ:
			this.doBizRequest(req, ctx);
			break;
		case Constant.REQ_TYPE_CHAT:	
			this.doChat(req, ctx);
			break;
		default:
			// 不能识别
			ResponseHelper.doErrorResponse(req, ctx, "pdType is not valid");
		}
	}

	@Override
	public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
		logger.warn("ctx close", cause);
		ctx.close();
	}
	
	private void doBizRequest(Request req, ChannelHandlerContext ctx) {
		
		try {
			// 参数
			byte[] pdData = req.getPdData().toByteArray();
			
			// 匹配HandlerMethod
			HandlerMethod handlerMethod = handlerMethodMapping.getHandler(req.getPdReqCode());
			
			// 处理
			HandlerMethodAdapter.process(req, ctx, handlerMethod, pdData);		
		} catch (Throwable t) {
			logger.error("doBizRequest error", t);
			ResponseHelper.doErrorResponse(req, ctx, t.getMessage());
		}
	}
	
	private void doChat(Request req, ChannelHandlerContext ctx) {
		// 略
		ResponseHelper.doErrorResponse(req, ctx, "暂不支持");
	}
}

HandlerMethod类

public class HandlerMethod {
	
	private final Object handler;

	private final Class<?> handlerType;

	private final Method method;
	
	private final Class<?>[] parameterTypes;
	
	public HandlerMethod(Object handler, Class<?> handlerType, Method method, Class<?>[] parameterTypes) {
		this.handler = handler;
		this.handlerType = handlerType;
		this.method = method;
		this.parameterTypes = parameterTypes;
	}

	public Object getHandler() {
		return handler;
	}

	public Class<?> getHandlerType() {
		return handlerType;
	}

	public Method getMethod() {
		return method;
	}

	public Class<?>[] getParameterTypes() {
		return parameterTypes;
	}
}

HandlerMethodMapping类

@Component
public class HandlerMethodMapping extends ApplicationObjectSupport implements InitializingBean {
	
	private final Map<Integer, HandlerMethod> mapHandlerMethod = new ConcurrentHashMap<>();
	
	public HandlerMethod getHandler(int reqCode) {
		return this.mapHandlerMethod.get(reqCode);
	}

	@Override
	public void afterPropertiesSet() throws Exception {
		initHandlerMethods();		
	}
	
	private void initHandlerMethods() {

		String[] beanNames = getApplicationContext().getBeanNamesForType(Object.class);

		for (String beanName : beanNames) {
			Class<?> beanType = null;
			try {
				beanType = getApplicationContext().getType(beanName);
			}
			catch (Throwable ex) {
				// An unresolvable bean type, probably from a lazy bean - let's ignore it.
				logger.debug("Could not resolve target class for bean with name '" + beanName + "'", ex);
			}
			if (beanType != null && isHandler(beanType)) {
				detectHandlerMethods(beanName);
			}	
		}
		
		logger.info("initHandlerMethods finish, size: " + mapHandlerMethod.size());
	}
	
	private boolean isHandler(Class<?> beanType) {
		return AnnotatedElementUtils.hasAnnotation(beanType, Controller.class);
	}
	
	private void detectHandlerMethods(final String beanName) {
		Class<?> handlerType = getApplicationContext().getType(beanName);
		Object handler = getApplicationContext().getBean(beanName);

		Method[] Methods = handlerType.getMethods();
		for(Method m : Methods) {
			PbRequestMapping reqMapping = m.getAnnotation(PbRequestMapping.class);
            if (reqMapping == null) {
                continue;
            }
            
            int reqCode = reqMapping.reqCode();
            Class<?>[] parameterTypes = m.getParameterTypes();            
            HandlerMethod HandlerMethod = new HandlerMethod(handler, handlerType, m, parameterTypes);
            
            if (mapHandlerMethod.containsKey(reqCode)) {
            	throw new RuntimeException("load HandlerMethod exception, reqCode match more method: " + reqCode);
            }
            mapHandlerMethod.put(reqCode, HandlerMethod);
		}
	}	
}

HandlerMethodAdapter类

public class HandlerMethodAdapter {
	
	private static final Logger logger = LoggerFactory.getLogger(HandlerMethodAdapter.class);
	private static final ExecutorService executor = Executors.newFixedThreadPool(100, new DemoThreadFactory("mmm"));
	
	public static void process(Request req, ChannelHandlerContext ctx, HandlerMethod handlerMethod, byte[] pdData) throws Exception {
		
		Object handler = handlerMethod.getHandler();
		Method method = handlerMethod.getMethod();
		Class<?>[] paramTypes = handlerMethod.getParameterTypes();
		
		// 参数
		Object[] params = getParams(req, ctx, method, paramTypes, pdData);
		
		executor.execute(new Runnable() {

			@Override
			public void run() {				
				try {
					// 处理
					Object result = method.invoke(handler, params);

					// 回复		
					ctx.writeAndFlush((ResponseProto.Response)result);					
				} catch (Throwable t) {
					logger.error("handler process excetpion", t);
					ResponseHelper.doErrorResponse(req, ctx, "inner exception: " + t.getMessage());
				}				
			}			
		});
	}
	
	private static Object[] getParams(Request req, ChannelHandlerContext ctx, Method method, Class<?>[] paramTypes, byte[] pdData) throws Exception {
				
		Annotation[][] parameterAnnotations = method.getParameterAnnotations();
		
		Object[] params = new Object[paramTypes.length];
        for (int i = 0; i < params.length; i++) {
        	Class<?> paramType = paramTypes[i];
        	Annotation[] annotations = parameterAnnotations[i];

        	if (ChannelHandlerContext.class.isAssignableFrom(paramType)) {
        		params[i] = ctx;
        	} else if (Request.class.isAssignableFrom(paramType)) {
        		params[i] = req;
        	} else if (GeneratedMessage.class.isAssignableFrom(paramType)) {
                if (pdData == null) {
                    params[i] = null;
                } else {
                    Method m = paramType.getMethod("parseFrom", ByteString.class);
                    Object o = m.invoke(paramType, ByteString.copyFrom(pdData));
                    params[i] = o;
                }
        	} else {
        		for (Annotation a : annotations) {
        			if (UserId.class.isAssignableFrom(a.getClass())) {
        				params[i] = getUserId();
        				break;
        			}
        		}
        	}
        }

		return params;
	}
	
	private static Integer getUserId() {
		// 略
		return 123;
	}
	
    private static final class DemoThreadFactory implements ThreadFactory {
        private static final AtomicInteger poolNumber = new AtomicInteger(1);
        private final ThreadGroup group;
        private final AtomicInteger threadNumber = new AtomicInteger(1);
        private final String namePrefix;

        public DemoThreadFactory(String alias) {
            group = new ThreadGroup(alias);            
            namePrefix = String.format("%s-pool-%d-thread-", alias, poolNumber.getAndIncrement());
        }

        @Override
        public Thread newThread(Runnable r) {
            Thread t = new Thread(group, r, namePrefix + threadNumber.getAndIncrement(), 0);
            
            if (t.isDaemon())
                t.setDaemon(false);
            if (t.getPriority() != Thread.NORM_PRIORITY)
                t.setPriority(Thread.NORM_PRIORITY);
            return t;
        }
    }	
}

（4）注解

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface PbRequestMapping {
	
    int reqCode();
}

@Target(ElementType.PARAMETER)
@Retention(RetentionPolicy.RUNTIME)
public @interface UserId { }

（5）辅助类

public interface Constant {
	
	int REQ_TYPE_BIZ = 1;
	int REQ_TYPE_CHAT = 2;
	
    int REQ_MESSAGE_A = 1;
    int REQ_MESSAGE_B = 2;
    
    int RES_STATE_SUCCESS = 0;
    int RES_STATE_FAIL = 1;
}

public class ResponseHelper {
	
	public static Response buildResponse(int state, byte[] data, Request request) {
		return buildResponse(state, "", data, request);
	}
	
	public static Response buildResponse(int state, String msg, byte[] data, Request request) {
		ResponseProto.Response.Builder builder = ResponseProto.Response.newBuilder();
		builder.setPdType(request.getPdType());
		builder.setPdResCode(request.getPdReqCode());
		builder.setPdState(state);
		builder.setPdMessage(msg);
		
		builder.setPdTimestamp(System.currentTimeMillis());
		builder.setPdData(ByteString.copyFrom(data));
		return builder.build();
	}	
	
	public static void doErrorResponse(Request req, ChannelHandlerContext ctx, String msg) {		
		Response res = buildResponse(Constant.RES_STATE_FAIL, msg, null, req);
		
		ctx.writeAndFlush(res);
	}	
}

6.3.4 客户端（测试）

public class DemoClient {

	public static void main(String[] args) throws Exception {
		new DemoClient().connect(8080, "127.0.0.1");
	}
	
	public void connect(int port, String host) throws Exception {
		EventLoopGroup group = new NioEventLoopGroup();
		try {
			Bootstrap b = new Bootstrap();
			b.group(group)
			.channel(NioSocketChannel.class)
			.option(ChannelOption.TCP_NODELAY, false)
			.handler(new ChildChannelHandler());

			// 同步等待成功
			ChannelFuture f = b.connect(host, port).sync();
			System.out.println(String.format("DemoClient start success, host: %s, port: %s", host, port));

			// 链路关闭
			f.channel().closeFuture().sync();
		} finally {
			// 释放线程池资源
			group.shutdownGracefully();
		}
	}

	private class ChildChannelHandler extends ChannelInitializer<SocketChannel> {
		@Override
		protected void initChannel(SocketChannel ch) throws Exception {
			ch.pipeline().addLast(new ProtobufVarint32FrameDecoder());
			ch.pipeline().addLast(new ProtobufDecoder(ResponseProto.Response.getDefaultInstance()));
			ch.pipeline().addLast(new ProtobufVarint32LengthFieldPrepender());
			ch.pipeline().addLast(new ProtobufEncoder());
			ch.pipeline().addLast(new DemoClientHandler());
			
			System.out.println("initChannel:pipeline");
		}
	}
}

public class DemoClientHandler extends ChannelInboundHandlerAdapter {
	
	private static final Logger logger = LoggerFactory.getLogger(DemoClientHandler.class);
	
	@Override
	public void channelActive(ChannelHandlerContext ctx) {		
//		ctx.writeAndFlush(this.getRequestForMessageA(100));
		ctx.writeAndFlush(this.getRequestForMessageB(200));
	}
	
	@Override
	public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
		ResponseProto.Response res = (ResponseProto.Response) msg;	
				
		if (res.getPdState() == 0) {
			ResMessageABProto.ResMessgeAB data = ResMessageABProto.ResMessgeAB.parseFrom(res.getPdData());
			logger.info("响应, state:{}, msg:{}, date:{}", res.getPdState(), res.getPdMessage(), data);
		} else {
			logger.info("响应, state:{}, msg:{}", res.getPdState(), res.getPdMessage());	
		}
	}

	@Override
	public void exceptionCaught(ChannelHandlerContext ctx, Throwable t) {		
		logger.error("netty-client exceptionCaught", t);
		ctx.close();
	}
	
	private RequestProto.Request getRequestForMessageA(int id) {
		RequestProto.Request.Builder reqBuilder = RequestProto.Request.newBuilder();
		reqBuilder.setPdType(Constant.REQ_TYPE_BIZ);
		reqBuilder.setPdReqCode(Constant.REQ_MESSAGE_A);
		reqBuilder.setPdTimestamp(System.currentTimeMillis());
		
		ReqMessageAProto.ReqMessgeA.Builder databuilder = ReqMessageAProto.ReqMessgeA.newBuilder();
		databuilder.setPdId(id);
		databuilder.setPdDesc("hello A");
		
		reqBuilder.setPdData(ByteString.copyFrom(databuilder.build().toByteArray()));		
		return reqBuilder.build();
	}	
	
	private RequestProto.Request getRequestForMessageB(int id) {
		RequestProto.Request.Builder reqBuilder = RequestProto.Request.newBuilder();
		reqBuilder.setPdType(Constant.REQ_TYPE_BIZ);
		reqBuilder.setPdReqCode(Constant.REQ_MESSAGE_B);
		reqBuilder.setPdTimestamp(System.currentTimeMillis());
		
		ReqMessageBProto.ReqMessgeB.Builder dataBuilder = ReqMessageBProto.ReqMessgeB.newBuilder();
		dataBuilder.setPdId(id);
		dataBuilder.setPdName("mm");
		dataBuilder.setPdDesc("hello B");
		
		reqBuilder.setPdData(ByteString.copyFrom(dataBuilder.build().toByteArray()));
		return reqBuilder.build();
	}	
}

6.4 消息推送（Protobuf）

这里只提供一下思路，就不列源码了，只要在6.3 案例基础上，增加维护一个TCP长链接的 “会话字典”就可以了，ChannelInboundHandlerAdapter已提供链接开启、关闭的两个事件handlerAdded（ChannelHandlerContext ctx），handlerRemoved（ChannelHandlerContext ctx），恰好满足要求，再结合业务本身的登录、退出逻辑，就可以维护一个userId 与ChannelHandlerContext的字典。

当消息推送时，通过userId 找到对应的ChannelHandlerContext，如果链接有效，就可以进行消息推送。