SO_REUSEPORT学习笔记补遗

前言

因为能力有限，还是有很多东西（SO_REUSEADDR和SO_REUSEPORT的区别等）没有能够在一篇文字中表达清楚，作为补遗，也方便以后自己回过头来复习。

SO_REUSADDR VS SO_REUSEPORT

两者不是一码事，没有可比性。有时也会被其搞晕，自己总结的不好，推荐StackOverflow的Socket options SO_REUSEADDR and SO_REUSEPORT, how do they differ?资料，总结的很全面。

简单来说：

• 设置了SO_REUSADDR的应用可以避免TCP 的 TIME_WAIT 状态 时间过长无法复用端口，尤其表现在应用程序关闭-重启交替的瞬间
• SO_REUSEPORT更强大，隶属于同一个用户（防止端口劫持）的多个进程/线程共享一个端口，同时在内核层面替上层应用做数据包进程/线程的处理均衡

若有困惑，推荐两者都设置，不会有冲突。

Netty多线程使用SO_REUSEPORT

上一篇讲到SO_REUSEPORT，多个程绑定同一个端口，可以根据需要控制进程的数量。这里讲讲基于Netty 4.0.25+Epoll navtie transport在单个进程内多个线程绑定同一个端口的情况，也是比较实用的。

TCP服务器，同一个进程多线程绑定同一个端口

这是一个PING-PONG示范应用：

     public void run() throws Exception {
            final EventLoopGroup bossGroup = new EpollEventLoopGroup();
            final EventLoopGroup workerGroup = new EpollEventLoopGroup();
            ServerBootstrap b = new ServerBootstrap();

           b.group(bossGroup, workerGroup)
                     .channel(EpollServerSocketChannel. class)
                     .childHandler( new ChannelInitializer<SocketChannel>() {
                            @Override
                            public void initChannel(SocketChannel ch) throws Exception {
                                ch.pipeline().addLast(
                                            new StringDecoder(CharsetUtil.UTF_8 ),
                                            new StringEncoder(CharsetUtil.UTF_8 ),
                                            new PingPongServerHandler());
                           }
                     }).option(ChannelOption. SO_REUSEADDR, true)
                     .option(EpollChannelOption. SO_REUSEPORT, true)
                     .childOption(ChannelOption. SO_KEEPALIVE, true);

            int workerThreads = Runtime.getRuntime().availableProcessors();
           ChannelFuture future;
            for ( int i = 0; i < workerThreads; ++i) {
                future = b.bind( port).await();
                 if (!future.isSuccess())
                      throw new Exception(String. format("fail to bind on port = %d.",
                                 port), future.cause());
           }
           Runtime. getRuntime().addShutdownHook (new Thread(){
                 @Override
                 public void run(){
                     workerGroup.shutdownGracefully();
                     bossGroup.shutdownGracefully();
                }
           });
     }

打成jar包，在CentOS 7下面运行，检查同一个端口所打开的文件句柄。

# lsof -i:8000
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    3515 root   42u  IPv6  29040      0t0  TCP *:irdmi (LISTEN)
java    3515 root   43u  IPv6  29087      0t0  TCP *:irdmi (LISTEN)
java    3515 root   44u  IPv6  29088      0t0  TCP *:irdmi (LISTEN)
java    3515 root   45u  IPv6  29089      0t0  TCP *:irdmi (LISTEN)

同一进程，但打开的文件句柄是不一样的。

UDP服务器，多个线程绑同一个端口

/**
 * UDP谚语服务器，单进程多线程绑定同一端口示范
 */
public final class QuoteOfTheMomentServer {

       private static final int PORT = Integer.parseInt(System. getProperty("port" ,
                   "9000" ));

       public static void main(String[] args) throws Exception {
             final EventLoopGroup group = new EpollEventLoopGroup();

            Bootstrap b = new Bootstrap();
            b.group(group).channel(EpollDatagramChannel. class)
                        .option(EpollChannelOption. SO_REUSEPORT, true )
                        .handler( new QuoteOfTheMomentServerHandler());

             int workerThreads = Runtime.getRuntime().availableProcessors();
             for (int i = 0; i < workerThreads; ++i) {
                  ChannelFuture future = b.bind( PORT).await();
                   if (!future.isSuccess())
                         throw new Exception(String.format ("Fail to bind on port = %d.",
                                     PORT), future.cause());
            }

            Runtime. getRuntime().addShutdownHook(new Thread() {
                   @Override
                   public void run() {
                        group.shutdownGracefully();
                  }
            });
      }
}
}

@Sharable
class QuoteOfTheMomentServerHandler extends
            SimpleChannelInboundHandler<DatagramPacket> {

       private static final String[] quotes = {
                   "Where there is love there is life." ,
                   "First they ignore you, then they laugh at you, then they fight you, then you win.",
                   "Be the change you want to see in the world." ,
                   "The weak can never forgive. Forgiveness is the attribute of the strong.", };

       private static String nextQuote() {
             int quoteId = ThreadLocalRandom.current().nextInt( quotes .length );
             return quotes [quoteId];
      }

       @Override
       public void channelRead0(ChannelHandlerContext ctx, DatagramPacket packet)
                   throws Exception {
             if ("QOTM?" .equals(packet.content().toString(CharsetUtil. UTF_8))) {
                  ctx.write( new DatagramPacket(Unpooled.copiedBuffer( "QOTM: "
                              + nextQuote(), CharsetUtil. UTF_8), packet.sender()));
            }
      }

       @Override
       public void channelReadComplete(ChannelHandlerContext ctx) {
            ctx.flush();
      }

       @Override
       public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
            cause.printStackTrace();
      }
}

同样也要检测一下端口文件句柄打开情况：

# lsof -i:9000
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    3181 root   26u  IPv6  27188      0t0  UDP *:cslistener
java    3181 root   27u  IPv6  27217      0t0  UDP *:cslistener
java    3181 root   28u  IPv6  27218      0t0  UDP *:cslistener
java    3181 root   29u  IPv6  27219      0t0  UDP *:cslistener

小结

以上为Netty+SO_REUSEPORT多线程绑定同一端口的一些情况，是为记载。