netty(一)Netty基础 NIO

最新推荐文章于 2021-09-03 17:52:23 发布

番茄柿

最新推荐文章于 2021-09-03 17:52:23 发布

阅读量150

点赞数

分类专栏： Netty

本文链接：https://blog.csdn.net/heyanfeng22/article/details/115744187

版权

Netty 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

Netty基础 NIO

IO演进

	IO存在5种模型，分别是：
	阻塞IO
	非阻塞IO
	多路复用IO
	信号驱动IO
	异步IO

	1、阻塞IO
	传统BIO存在两个阻塞的地方，accept()和read/write
	没有客户端链接时会阻塞在accept()
	连接上来了但是没有传输数据又会阻塞，导致后面的客户端链接不进来
	2、非阻塞 IO
	每个链接连上来后开一个线程去处理读写，这样数据读写就不阻塞了，但是线程里while循环消耗性能 
	3、多路复用IO
	存在一个单线程轮询所有的链接，判断链接是什么状态，有数据来了就处理数据，没有就继续轮询
	NIO就是多路复用IO
	4、信号驱动IO
	数据读取完之后，增加一个提示信号
	5、异步IO
	比信号驱动模型更高级，信号驱动模型中，进程在收到通知的是可以开始一个IO操作，异步IO模型中收到的通知是数据以及读取完毕。因为操作系统已经把数据读写完了

	BIO 最终客户端跟线程比是1:1；
	伪异步IO， 就是线程池

同步和异步

可以理解为数据的读取是谁操作的，应用读写的是同步，操作系统读写的是异步
异步的IO是只要通知系统要读写数据了就不管了，然后操作系统帮读写数据后通知应用读写好了

阻塞和非阻塞

数据没有准备好，是否会阻塞程序，例如传统BIO 中的accept()和read()、write() 方法

面向流和面向缓冲区

面向流
	每次只能读取一个字节或者多个字节，直至读完
	没有缓存在任何地方，不能前后移动
面向缓冲区
	数据读取到缓冲区，需要时可以再缓冲区中前后移动，增加了处理中的灵活性

选择器

选择器(Selector)允许一个单独的线程监控多个输入通道，这些通道负责读写数据

适用场景

BIO 连接数少，但是传输数据大的
NIO 连接数多，但是传输数据小的

Reactor，事件驱动模式

应用程序不主动调业务处理接口，而是把接口注册在Reactor上。
如果发生了某件事，Reactor主动调用业务处理的接口，
在NIO的基础上抽出来两个角色，Reactor和Handler
Reactor负责接收socket链接判断状态分发请求到handler，handler负责处理业务

主要组成
	1、句柄描述，表示监听的事件，比如文件描述符，是否到达可读可写
	2、同步事件分离器，谁来负责监听事件，NIO中就是selector，linux中为select，poll，epoll
	3、事件处理器，具体指回调函数
	4、分发器，将channel分发给具体的handler
	5、业务处理handler
实现步骤
	1、初始化一个Reactor管理器
	2、初始化事件处理器,设置事件源及回调函数
	3、将事件处理器注册到Reactor管理器上
	4、注册该事件
	5、进入循环等待事件发生并处理

NIO 三件套

通道 channel
缓冲区 Buffer 
选择器 Selector

Netty构成

ServerBootstrap:netty容器 把各部分都整合起来了
Channel : 代表socket链接
EventLoopGroup ：理解为线程池，一个group可以包含多个EventLoop
EventLoop ：处理具体的Channel，一个EventLoop可以处理多个Channel
ChannelPipeline：每个channel绑定一个pipeline，注册处理handler
Handler：具体的处理方法，主要是读和写
ChannelFuture：钩子方法

BIO/NIO/Netty实现数据传输

BIO-server

ServerSocket serverSocket = new ServerSocket(8080);

        InputStream inputStream = null;
        try
        {
            while (true)
            {
                Socket socket = serverSocket.accept();
                inputStream = socket.getInputStream();
                int content = inputStream.read(bytes);

                if(content>0)
                {
                    String str = new String(bytes,0,content,"utf-8");
                    System.out.println(str);
                }



            }

        }
        catch (Exception e)
        {

        }
        finally
        {
            inputStream.close();
        }

BIO-Client

Socket client = new Socket("127.0.0.1",8080);

        OutputStream os = null;
        try
        {
            os = client.getOutputStream();
            String str = "123123哈哈哈哈";
            os.write(str.getBytes());
        }
        catch (Exception e)
        {

        }
        finally
        {
            os.close();
            client.close();
        }

NIO-server

private static Selector selector;

    private static ByteBuffer buffer = ByteBuffer.allocate(1024);

    public static void main(String[] args) throws IOException
    {

        ServerSocketChannel server = ServerSocketChannel.open();
        server.bind(new InetSocketAddress(8080));
        server.configureBlocking(false);
        selector = Selector.open();
        server.register(selector,SelectionKey.OP_ACCEPT);

        while(true)
        {
            selector.select();
            Set<SelectionKey> keys= selector.selectedKeys();
            Iterator<SelectionKey> it = keys.iterator();
            while (it.hasNext())
            {
                SelectionKey key = it.next();
                it.remove();
                process(key);
            }


        }
    }

    public static void process(SelectionKey key) throws IOException
    {
        if(key.isAcceptable())
        {
            ServerSocketChannel server = (ServerSocketChannel) key.channel();
            SocketChannel channel = server.accept();
            channel.configureBlocking(false);
            key = channel.register(selector,SelectionKey.OP_READ);

        }

        else if(key.isReadable())
        {
            SocketChannel channel = (SocketChannel) key.channel();
            int len = channel.read(buffer);
            if(len > 0)
            {
                buffer.flip();
                String content = new String(buffer.array(),0,len);
                key = channel.register(selector,SelectionKey.OP_WRITE);
                key.attach(content);
                System.out.println("读取内容：" + content);
            }

        }
        else if(key.isWritable())
        {
            SocketChannel channel = (SocketChannel) key.channel();
            String content = (String)key.attachment();
            channel.write(ByteBuffer.wrap(("返回输出：" + content).getBytes()));

            channel.close();

        }
    }

NIO-client跟BIO可以一样
Netty服务端

public class MyNettyServer
{
    public void start()
    {

        ServerBootstrap strap = new ServerBootstrap();

        //boss线程
        EventLoopGroup boss = new NioEventLoopGroup();

        //工作线程
        EventLoopGroup work = new NioEventLoopGroup();

        try
        {
            strap.group(boss,work)
                    .channel(NioServerSocketChannel.class)
                    .option(ChannelOption.SO_BACKLOG,1024)
                    .childHandler(new ChannelInitializer<SocketChannel>()
                    {

                        @Override
                        protected void initChannel(SocketChannel socketChannel) throws Exception
                        {
                            socketChannel.pipeline().addLast(new MyNettyHandler());
                        }
                    })
                    .childOption(ChannelOption.SO_KEEPALIVE,true);

            //启动服务
            ChannelFuture f =strap.bind(8080).sync();

            System.out.println("==========启动成功==============");
            f.channel().closeFuture().sync();
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
        finally
        {
            boss.shutdownGracefully();
            work.shutdownGracefully();
        }



    }

    public class MyNettyHandler extends ChannelInboundHandlerAdapter
    {
        @Override
        public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception
        {
            // 第一种：接收字符串时的处理
            ByteBuf buf = (ByteBuf) msg;
            String rev = getMessage(buf);
            System.out.println("客户端收到服务器数据:" + rev);




        }

    }

    private String getMessage(ByteBuf buf) {

        byte[] con = new byte[buf.readableBytes()];
        buf.readBytes(con);
        try {
            return new String(con, "UTF-8");
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
            return null;
        }
    }

    public static void main(String[] args)
    {
        new MyNettyServer().start();
    }
}

客户端可以跟BIO一样，也可以用Netty，用 ServerBootstrap 、EventLoopGroup 也实现类似的功能

Netty手写tomcat

用propertis模拟web.xml

servlet.one.className=com.gupaoedu.vip.netty.tomcat.nio.servlet.FirstServlet
servlet.one.url=/firstServlet.do

servlet.two.className=com.gupaoedu.vip.netty.tomcat.nio.servlet.SecondServlet
servlet.two.url=/secondServlet.do

Request

public class GPRequest {

    private ChannelHandlerContext ctx;
    private HttpRequest req;

    public GPRequest(ChannelHandlerContext ctx, HttpRequest req) {
        this.ctx = ctx;
        this.req = req;
    }

    public String getUrl(){
        return this.req.uri();
    }
    public String getMethod(){
        return this.req.method().name();
    }

    public Map<String, List<String>> getParameters(){
        QueryStringDecoder decoder = new QueryStringDecoder(req.uri());
        return decoder.parameters();
    }

    public String getParameter(String name){
        Map<String,List<String >> params = getParameters();
        List<String> param = params.get(name);
        if(null == param){
            return null;
        }else {
            return param.get(0);
        }
    }

}

Response

public class GPResponse {

    private ChannelHandlerContext ctx;
    private HttpRequest req;

    public GPResponse(ChannelHandlerContext ctx, HttpRequest req) {
        this.ctx = ctx;
        this.req = req;
    }

    public void write(String s) throws Exception{

        if(null == s || 0 == s.length()){
            return;
        }

        try {
            FullHttpResponse response = new DefaultFullHttpResponse(
                    //设置HTTP版本号为1.1
                    HttpVersion.HTTP_1_1,
                    //设置返回HTTP状态码
                    HttpResponseStatus.OK,
                    //统一输出格式为UTF-8
                    Unpooled.wrappedBuffer(s.getBytes("UTF-8"))
            );

            response.headers().set("Content-Type", "text/html");

            ctx.write(response);
        }catch (Exception e){
            e.printStackTrace();
        }finally {
            ctx.flush();
            ctx.close();
        }


    }

}

Servlet

public abstract class GPServlet {

    public void service(GPRequest request, GPResponse response) throws Exception{
        if("GET".equalsIgnoreCase(request.getMethod())){
            doGet(request,response);
        }else {
            doPost(request,response);
        }
    }


    public abstract void doGet(GPRequest request, GPResponse response) throws Exception;

    public abstract void doPost(GPRequest request, GPResponse response) throws Exception;

}

服务实现

public class FirstServlet extends GPServlet {
    public void doGet(GPRequest request, GPResponse response) throws Exception {
        this.doPost(request,response);
    }

    public void doPost(GPRequest request, GPResponse response) throws Exception {
       response.write("This is first servlet from NIO.");
    }
}

主线程

public class GPTomcat {

    private int port = 8080;

    private ServerSocket server;

    private Properties webxml = new Properties();

    private Map<String, GPServlet> servletMapping = new HashMap<String, GPServlet>();

    public static void main(String[] args) {
        new GPTomcat().start();
    }

    //Tomcat的启动入口
    private void start() {
        //1、加载web.properties文件，解析配置
        init();

        //Boss线程
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        //Worker线程
        EventLoopGroup workerGroup = new NioEventLoopGroup();

        //2、创建Netty服务端对象
        ServerBootstrap server = new ServerBootstrap();

        try {
            //3、配置服务端参数
            server.group(bossGroup, workerGroup)
                    //配置主线程的处理逻辑
                    .channel(NioServerSocketChannel.class)
                    //子线程的回调处理，Handler
                    .childHandler(new ChannelInitializer() {
                        @Override
                        protected void initChannel(Channel client) throws Exception {
                            //处理回调的逻辑

                            //链式编程，责任链模式

                            //处理响应结果的封装
                            client.pipeline().addLast(new HttpResponseEncoder());
                            //用户请求过来，要解码
                            client.pipeline().addLast(new HttpRequestDecoder());
                            //用户自己的业务逻辑
                            client.pipeline().addLast(new GPTomcatHandler());

                        }
                    })
                    //配置主线程分配的最大线程数
                    .option(ChannelOption.SO_BACKLOG, 128)
                    //保持长连接
                    .childOption(ChannelOption.SO_KEEPALIVE, true);

            //启动服务
            ChannelFuture f = server.bind(this.port).sync();

            System.out.println("GP Tomcat 已启动，监听端口是: " + this.port);

            f.channel().closeFuture().sync();
        }catch (Exception e){
            e.printStackTrace();
        }finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }

    }


    private void init() {
        try {
            String WEB_INF = this.getClass().getResource("/").getPath();
            FileInputStream fis = new FileInputStream(WEB_INF + "web-nio.properties");

            webxml.load(fis);

            for (Object k : webxml.keySet()) {

                String key = k.toString();

                if(key.endsWith(".url")){

                    //将 servlet.xxx.url 的 .url 替换，只剩下 servlet.xxx当成  servletName
                    String servletName = key.replaceAll("\\.url$","");
                    String url = webxml.getProperty(key);

                    //拿到Serlvet的全类名
                    String className = webxml.getProperty(servletName + ".className");

                    //反射创建Servlet的实例
                    GPServlet obj = (GPServlet) Class.forName(className).newInstance();
                    //将URL和Servlet建立映射关系
                    servletMapping.put(url,obj);
                }

            }

        }catch (Exception e){
            e.printStackTrace();
        }
    }

    public class GPTomcatHandler extends ChannelInboundHandlerAdapter{
		
		/**
		** msg 是数据包
		**/
        @Override
        public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {

            if(msg instanceof HttpRequest){
                HttpRequest req = (HttpRequest) msg;

                GPRequest request = new GPRequest(ctx,req);
                GPResponse response = new GPResponse(ctx,req);

                String url = request.getUrl();

                if(servletMapping.containsKey(url)){
                    servletMapping.get(url).service(request,response);
                }else{
                    response.write("404 - Not Found!!");
                }

            }


        }
    }

}

Netty 重构RPC框架

provider-接口

public interface IRpcHelloService {
    String hello(String name);
}

provider-接口实现类

public class RpcHelloServiceImpl implements IRpcHelloService {
    public String hello(String name) {
        return "hello " + name + "!";
    }
}

服务端启动

public class RpcRegistry {
    private int port;
    public RpcRegistry(int port) {
        this.port = port;
    }


    private void start() {
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();

        try {
            ServerBootstrap server = new ServerBootstrap();
            server.group(bossGroup, workerGroup)
                    .channel(NioServerSocketChannel.class)
                    .childHandler(new ChannelInitializer() {
                        @Override
                        protected void initChannel(Channel ch) throws Exception {
                            //接收课客户端请求的处理流程
                            ChannelPipeline pipeline = ch.pipeline();

                            int fieldLength = 4;
                            //通用解码器设置
                            pipeline.addLast(new LengthFieldBasedFrameDecoder(Integer.MAX_VALUE,0,fieldLength,0,fieldLength));
                            //通用编码器
                            pipeline.addLast(new LengthFieldPrepender(fieldLength));
                            //对象编码器
                            pipeline.addLast("encoder",new ObjectEncoder());
                            //对象解码器
                            pipeline.addLast("decoder",new ObjectDecoder(Integer.MAX_VALUE, ClassResolvers.cacheDisabled(null)));

                            pipeline.addLast(new RegistryHandler());
                        }
                    })
                    .option(ChannelOption.SO_BACKLOG, 128)
                    .childOption(ChannelOption.SO_KEEPALIVE, true);

            ChannelFuture future = server.bind(this.port).sync();
            System.out.println("GP RPC registry is start,listen at " + this.port);
            future.channel().closeFuture().sync();
        }catch (Exception e){
            e.printStackTrace();
        }finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new RpcRegistry(8080).start();
    }

}

处理业务的handler

public class RegistryHandler extends ChannelInboundHandlerAdapter {

    //注册中心容器
    private static ConcurrentHashMap<String,Object> registryMap = new ConcurrentHashMap<String, Object>();

    private List<String> classNames = new ArrayList<String>();

    public RegistryHandler(){
        
        //扫描所有需要注册的类
        //com.gupaoedu.vip.netty.rpc.provider
        scannerClass("com.gupaoedu.vip.netty.rpc.provider");
        
        //将扫描到的类注册到一个容器中
        doRegister();
    }

    private void doRegister() {
        if(classNames.size() == 0){return;}

        for (String className : classNames) {
            try {
                Class<?> clazz = Class.forName(className);
                Class<?> i = clazz.getInterfaces()[0];
                registryMap.put(i.getName(),clazz.newInstance());
            }catch (Exception e){
                e.printStackTrace();
            }
        }
    }

    private void scannerClass(String packageName) {
        URL url = this.getClass().getClassLoader().getResource(packageName.replaceAll("\\.","/"));
        File dir = new File(url.getFile());

        for (File file : dir.listFiles()) {

            //如果是文件夹，继续递归
            if(file.isDirectory()){
                scannerClass(packageName + "." + file.getName());
            }else {
                classNames.add(packageName + "." + file.getName().replace(".class","").trim());
            }
        }
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        Object result = new Object();
        InvokerProtocol request = (InvokerProtocol) msg;

        if(registryMap.containsKey(request.getClassName())){
            //用反射直接调用Provider的方法
            Object provider = registryMap.get(request.getClassName());
            Method method = provider.getClass().getMethod(request.getMethodName(),request.getParams());
            result = method.invoke(provider,request.getValues());
        }
        ctx.write(result);
        ctx.flush();
        ctx.close();
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
       cause.printStackTrace();
       ctx.close();
    }
}

传输的数据实例

@Data
public class InvokerProtocol implements Serializable {

    private String className;  //类名
    private String methodName; //方法名称
    private Class<?>[] params; //形参列表
    private Object[] values;   //实参列表

}

客户端代理类

public class RpcProxy {
    public static <T> T create(Class<?> clazz){

        MethodProxy proxy = new MethodProxy(clazz);
        Class<?> [] interfaces = clazz.isInterface() ?
                                 new Class[]{clazz} :
                                 clazz.getInterfaces();
        T result = (T)Proxy.newProxyInstance(clazz.getClassLoader(),interfaces,proxy);
        return result;
    }

    public static class MethodProxy implements InvocationHandler {
        private Class<?> clazz;
        public MethodProxy(Class<?> clazz) {
            this.clazz = clazz;
        }

        public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
            if(Object.class.equals(method.getDeclaringClass())){
                return method.invoke(this,args);
            }else{
                return rpcInvoke(proxy,method,args);
            }
        }

        private Object rpcInvoke(Object proxy, Method method, Object[] args) {

            //封装请求的内容
            InvokerProtocol msg = new InvokerProtocol();
            msg.setClassName(this.clazz.getName());
            msg.setMethodName(method.getName());
            msg.setParams(method.getParameterTypes());
            msg.setValues(args);


            final RpcProxyHandler consumerHandler = new RpcProxyHandler();


            EventLoopGroup group = new NioEventLoopGroup();

            try {
                Bootstrap client = new Bootstrap();
                client.group(group)
                        .channel(NioSocketChannel.class)
                        .handler(new ChannelInitializer() {
                            @Override
                            protected void initChannel(Channel ch) throws Exception {
                                //接收课客户端请求的处理流程
                                ChannelPipeline pipeline = ch.pipeline();

                                int fieldLength = 4;
                                //通用解码器设置
                                pipeline.addLast(new LengthFieldBasedFrameDecoder(Integer.MAX_VALUE,0,fieldLength,0,fieldLength));
                                //通用编码器
                                pipeline.addLast(new LengthFieldPrepender(fieldLength));
                                //对象编码器
                                pipeline.addLast("encoder",new ObjectEncoder());
                                //对象解码器
                                pipeline.addLast("decoder",new ObjectDecoder(Integer.MAX_VALUE, ClassResolvers.cacheDisabled(null)));

                                pipeline.addLast("handler",consumerHandler);
                            }
                        })
                        .option(ChannelOption.TCP_NODELAY, true);

                ChannelFuture future = client.connect("localhost",8080).sync();
                future.channel().writeAndFlush(msg).sync();
                future.channel().closeFuture().sync();
            }catch (Exception e){
                e.printStackTrace();
            }finally {
                group.shutdownGracefully();
            }

            return consumerHandler.getResponse();
        }
    }
}

客户端业务处理类

public class RpcProxyHandler extends ChannelInboundHandlerAdapter {

    private Object response;

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        this.response = msg;
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        cause.printStackTrace();
    }

    public Object getResponse() {
        return this.response;
    }
}

客户端启动类

public class RpcConsumer {
    public static void main(String[] args) {

        IRpcHelloService rpcHello = RpcProxy.create(IRpcHelloService.class);
        System.out.println(rpcHello.hello("Tom"));

        IRpcService rpc = RpcProxy.create(IRpcService.class);

        System.out.println("8 + 2 = " + rpc.add(8,2));
        System.out.println("8 - 2 = " + rpc.sub(8,2));
        System.out.println("8 * 2 = " + rpc.mult(8,2));
        System.out.println("8 / 2 = " + rpc.div(8,2));

    }
}

总结

	1、了解了BIO的缺点，为什么阻塞哪里阻塞，针对BIO的缺点一步步改进，到非阻塞IO
	2、NIO 的原理，如何使用NIO实现网络数据传输
	3、Netty的设计模式，Netty替代BIO/NIO数据传输
	4、BIO/NIO/Netty 实现Tomcat和Rpc主要的变化在于，获取socket的处理，其他都差不多