TCP参数之backlog

运维同学的一个问题直接问的我蒙圈：请问你们dubbo的backlog参数设置的是多少？
我是谁，我在哪，我在干什么？

backlog参数含义

backlog参数为socket套接字监听端口时，内核为该套接字分配的一个队列大小，在服务端还没有来得及处理请求时暂时缓存请求所用的队列。如果队列已经被客户端socket占满了，还有新的连接过来，那么ServerSocket会拒绝新的连接。backlog提供了容量限制功能，避免太多的客户端socket占用太多服务器资源。
注意如果连接处于休眠状态即没有数据传输是不属于服务处理中的连接，所以不会计算在内

backlog参数使用

服务端仅接受连接请求不处理请求

/**
 * @author 会灰翔的灰机
 * @date 2020/9/19
 */
public class MyBacklog {

    public static void startServer() throws IOException {
        // 1. 创建服务端socket套接字，此处可以指定端口，backlog参数，但是我们放在后面再指定
        ServerSocket serverSocket = new ServerSocket();
        // 2. 创建服务地址并指定端口号
        SocketAddress socketAddress = new InetSocketAddress(InetAddress.getLocalHost(), 8888);
        // 3. 绑定服务地址及端口号，并设置backlog
        serverSocket.bind(socketAddress, 3);
    }

    public static void main(String[] args) throws Exception {
        startServer();
        int size = 5;
        for (int i=0; i < size; i++) {
            startClient();
            System.out.println("connect success size = " + (i + 1));
        }
    }

    public static void startClient() throws IOException {
        Socket socket = new Socket();
        SocketAddress socketAddress = new InetSocketAddress(InetAddress.getLocalHost(), 8888);
        socket.connect(socketAddress);
    }
}

请求连接到第四个时抛出异常，运行结果如下：

"D:\Program Files\Java\jdk1.8.0_131\bin\java.exe" -javaagent:D:\Applications\ideaIC-2019.3.1.win\lib\idea_rt.jar=62461:D:\Applications\ideaIC-2019.3.1.win\bin -Dfile.encoding=UTF-8 -classpath "D:\Program Files\Java\jdk1.8.0_131\jre\lib\charsets.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\deploy.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\access-bridge-64.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\cldrdata.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\dnsns.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\jaccess.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\jfxrt.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\localedata.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\nashorn.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunec.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunjce_provider.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunmscapi.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunpkcs11.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\zipfs.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\javaws.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jce.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jfr.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jfxswt.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jsse.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\management-agent.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\plugin.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\resources.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\rt.jar;D:\git\private\gallant\mytest\target\classes;D:\Applications\apache-maven-3.5.0\conf\repository\com\fasterxml\jackson\core\jackson-databind\2.10.1\jackson-databind-2.10.1.jar;D:\Applications\apache-maven-3.5.0\conf\repository\com\fasterxml\jackson\core\jackson-annotations\2.10.1\jackson-annotations-2.10.1.jar;D:\Applications\apache-maven-3.5.0\conf\repository\com\fasterxml\jackson\core\jackson-core\2.10.1\jackson-core-2.10.1.jar;D:\Applications\apache-maven-3.5.0\conf\repository\org\projectlombok\lombok\1.16.20\lombok-1.16.20.jar;D:\git\private\gallant\mytest\mytest-api\target\classes" com.mytest.MyBacklog
connect success size = 1
connect success size = 2
connect success size = 3
Exception in thread "main" java.net.ConnectException: Connection refused: connect
	at java.net.DualStackPlainSocketImpl.connect0(Native Method)
	at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at java.net.Socket.connect(Socket.java:538)
	at com.mytest.MyBacklog.startClient(MyBacklog.java:37)
	at com.mytest.MyBacklog.main(MyBacklog.java:29)

Process finished with exit code 1

服务端接受并处理请求

/**
 * @author 会灰翔的灰机
 * @date 2020/9/19
 */
public class MyBacklog2 {

    public static void startServer() throws IOException {
        // 1. 创建服务端socket套接字，此处可以指定端口，backlog参数，但是我们放在后面再指定
        final ServerSocket serverSocket = new ServerSocket();
        // 2. 创建服务地址并指定端口号
        SocketAddress socketAddress = new InetSocketAddress(InetAddress.getLocalHost(), 8888);
        // 3. 绑定服务地址及端口号，并设置backlog
        serverSocket.bind(socketAddress, 3);
        // 4. 处理请求
        new Thread(new Runnable() {
            @Override
            public void run() {
                int count = 1;
                while (true) {
                    try {
                        serverSocket.accept();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                    System.out.println("accept request success count = " + count++);
                }
            }
        }).start();
    }

    public static void main(String[] args) throws Exception {
        startServer();
        int size = 5;
        for (int i=0; i < size; i++) {
            startClient();
            System.out.println("connect success size = " + (i + 1));
        }
    }

    public static void startClient() throws IOException {
        Socket socket = new Socket();
        SocketAddress socketAddress = new InetSocketAddress(InetAddress.getLocalHost(), 8888);
        socket.connect(socketAddress);
    }
}

服务端正常处理请求完成后，休眠的连接是不占用backlog队列位置的，连接全部成功，输出如下：

"D:\Program Files\Java\jdk1.8.0_131\bin\java.exe" -javaagent:D:\Applications\ideaIC-2019.3.1.win\lib\idea_rt.jar=62947:D:\Applications\ideaIC-2019.3.1.win\bin -Dfile.encoding=UTF-8 -classpath "D:\Program Files\Java\jdk1.8.0_131\jre\lib\charsets.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\deploy.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\access-bridge-64.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\cldrdata.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\dnsns.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\jaccess.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\jfxrt.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\localedata.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\nashorn.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunec.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunjce_provider.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunmscapi.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunpkcs11.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\zipfs.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\javaws.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jce.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jfr.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jfxswt.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jsse.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\management-agent.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\plugin.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\resources.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\rt.jar;D:\git\private\gallant\mytest\target\classes;D:\Applications\apache-maven-3.5.0\conf\repository\com\fasterxml\jackson\core\jackson-databind\2.10.1\jackson-databind-2.10.1.jar;D:\Applications\apache-maven-3.5.0\conf\repository\com\fasterxml\jackson\core\jackson-annotations\2.10.1\jackson-annotations-2.10.1.jar;D:\Applications\apache-maven-3.5.0\conf\repository\com\fasterxml\jackson\core\jackson-core\2.10.1\jackson-core-2.10.1.jar;D:\Applications\apache-maven-3.5.0\conf\repository\org\projectlombok\lombok\1.16.20\lombok-1.16.20.jar;D:\git\private\gallant\mytest\mytest-api\target\classes" com.mytest.MyBacklog2
connect success size = 1
accept request success count = 1
accept request success count = 2
connect success size = 2
accept request success count = 3
connect success size = 3
connect success size = 4
accept request success count = 4
accept request success count = 5
connect success size = 5

服务端接受请求并处理请求，但处理很耗时场景

/**
 * @author 会灰翔的灰机
 * @date 2020/9/19
 */
public class MyBacklog3 {

    public static void startServer() throws IOException {
        // 1. 创建服务端socket套接字，此处可以指定端口，backlog参数，但是我们放在后面再指定
        final ServerSocket serverSocket = new ServerSocket();
        // 2. 创建服务地址并指定端口号
        SocketAddress socketAddress = new InetSocketAddress(InetAddress.getLocalHost(), 8888);
        // 3. 绑定服务地址及端口号，并设置backlog
        serverSocket.bind(socketAddress, 3);
        // 4. 处理请求
        new Thread(new Runnable() {
            @Override
            public void run() {
                int count = 1;
                while (true) {
                    try {
                        serverSocket.accept();
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                    System.out.println("accept request success count = " + count++);
                    try {
                        Thread.sleep(10000);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                }
            }
        }).start();
    }

    public static void main(String[] args) throws Exception {
        startServer();
        int size = 5;
        for (int i=0; i < size; i++) {
            startClient();
            System.out.println("connect success size = " + (i + 1));
        }
    }

    public static void startClient() throws IOException {
        Socket socket = new Socket();
        SocketAddress socketAddress = new InetSocketAddress(InetAddress.getLocalHost(), 8888);
        socket.connect(socketAddress);
    }
}

第五个连接进来时便抛出了异常，输出如下：

"D:\Program Files\Java\jdk1.8.0_131\bin\java.exe" -javaagent:D:\Applications\ideaIC-2019.3.1.win\lib\idea_rt.jar=63385:D:\Applications\ideaIC-2019.3.1.win\bin -Dfile.encoding=UTF-8 -classpath "D:\Program Files\Java\jdk1.8.0_131\jre\lib\charsets.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\deploy.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\access-bridge-64.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\cldrdata.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\dnsns.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\jaccess.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\jfxrt.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\localedata.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\nashorn.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunec.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunjce_provider.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunmscapi.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunpkcs11.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\zipfs.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\javaws.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jce.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jfr.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jfxswt.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\jsse.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\management-agent.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\plugin.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\resources.jar;D:\Program Files\Java\jdk1.8.0_131\jre\lib\rt.jar;D:\git\private\gallant\mytest\target\classes;D:\Applications\apache-maven-3.5.0\conf\repository\com\fasterxml\jackson\core\jackson-databind\2.10.1\jackson-databind-2.10.1.jar;D:\Applications\apache-maven-3.5.0\conf\repository\com\fasterxml\jackson\core\jackson-annotations\2.10.1\jackson-annotations-2.10.1.jar;D:\Applications\apache-maven-3.5.0\conf\repository\com\fasterxml\jackson\core\jackson-core\2.10.1\jackson-core-2.10.1.jar;D:\Applications\apache-maven-3.5.0\conf\repository\org\projectlombok\lombok\1.16.20\lombok-1.16.20.jar;D:\git\private\gallant\mytest\mytest-api\target\classes" com.mytest.MyBacklog3
connect success size = 1
accept request success count = 1
connect success size = 2
connect success size = 3
connect success size = 4
Exception in thread "main" java.net.ConnectException: Connection refused: connect
	at java.net.DualStackPlainSocketImpl.connect0(Native Method)
	at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at java.net.Socket.connect(Socket.java:538)
	at com.mytest.MyBacklog3.startClient(MyBacklog3.java:57)
	at com.mytest.MyBacklog3.main(MyBacklog3.java:49)
accept request success count = 2

由于一个请求正在处理中，不会占用队列，所以并发连接达到了4个。第一个案例中因为所有请求都没有在处理中，均占用了队列，所以第四个请求失败了

dubbo的backlog参数设置

dubbo默认使用的网络传输框架为netty，不多赘述，直接看NettyServer源码，父类com.alibaba.dubbo.remoting.transport.AbstractServer构造器中创建服务地址并指定端口号，回调子类doOpen绑定服务地址及端口

@Override
protected void doOpen() throws Throwable {
    NettyHelper.setNettyLoggerFactory();
    ExecutorService boss = Executors.newCachedThreadPool(new NamedThreadFactory("NettyServerBoss", true));
    ExecutorService worker = Executors.newCachedThreadPool(new NamedThreadFactory("NettyServerWorker", true));
    ChannelFactory channelFactory = new NioServerSocketChannelFactory(boss, worker, getUrl().getPositiveParameter(Constants.IO_THREADS_KEY, Constants.DEFAULT_IO_THREADS));
    bootstrap = new ServerBootstrap(channelFactory);
    ...
    bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
        @Override
        public ChannelPipeline getPipeline() {
            NettyCodecAdapter adapter = new NettyCodecAdapter(getCodec(), getUrl(), NettyServer.this);
            ChannelPipeline pipeline = Channels.pipeline();
            /*int idleTimeout = getIdleTimeout();
            if (idleTimeout > 10000) {
                pipeline.addLast("timer", new IdleStateHandler(timer, idleTimeout / 1000, 0, 0));
            }*/
            pipeline.addLast("decoder", adapter.getDecoder());
            pipeline.addLast("encoder", adapter.getEncoder());
            pipeline.addLast("handler", nettyHandler);
            return pipeline;
        }
	});
    // bind
    channel = bootstrap.bind(getBindAddress());
}

绑定地址，创建新通道channel，getFactory返回的通道工厂即上方构建的NioServerSocketChannelFactory实例

public Channel bind(final SocketAddress localAddress) {
    ...
    ChannelHandler binder = new Binder(localAddress, futureQueue);
    ChannelHandler parentHandler = getParentHandler();
    ChannelPipeline bossPipeline = pipeline();
    bossPipeline.addLast("binder", binder);
    if (parentHandler != null) {
        bossPipeline.addLast("userHandler", parentHandler);
    }
    Channel channel = getFactory().newChannel(bossPipeline);
    // Wait until the future is available.
    ChannelFuture future = null;
    boolean interrupted = false;
    do {
        try {
            future = futureQueue.poll(Integer.MAX_VALUE, TimeUnit.SECONDS);
        } catch (InterruptedException e) {
            interrupted = true;
        }
    } while (future == null);
    if (interrupted) {
        Thread.currentThread().interrupt();
    }
    // Wait for the future.
    future.awaitUninterruptibly();
    if (!future.isSuccess()) {
        future.getChannel().close().awaitUninterruptibly();
        throw new ChannelException("Failed to bind to: " + localAddress, future.getCause());
    }
    return channel;
}

创建新通道

public ServerSocketChannel newChannel(ChannelPipeline pipeline) {
    return new NioServerSocketChannel(this, pipeline, sink);
}

新通道构造器中打开服务端套接字

NioServerSocketChannel(
        ChannelFactory factory,
        ChannelPipeline pipeline,
        ChannelSink sink) {
    	// 当前channel通道、sink与pipeline绑定：pipeline.attach(this, sink);
    	super(factory, pipeline, sink);
    ...
        // 调用jdk nio包实现类打开服务端套接字通道
        socket = ServerSocketChannel.open();
    ...
        // 服务端套接字通道常见ServerSocket
    config = new DefaultServerSocketChannelConfig(socket.socket());
    // 发送通道打开事件
    fireChannelOpen(this);
}

发送通道打开事件，通道pipeline为绑定地址时创建并添加了Binder节点，打开事件通知回调Binder：org.jboss.netty.bootstrap.ServerBootstrap.Binder#channelOpen

public static void fireChannelOpen(Channel channel) {
    ...
    channel.getPipeline().sendUpstream(
            new UpstreamChannelStateEvent(
                    channel, ChannelState.OPEN, Boolean.TRUE));
}
public void handleUpstream(
        ChannelHandlerContext ctx, ChannelEvent e) throws Exception {
    ...
    } else if (e instanceof ChannelStateEvent) {
        ChannelStateEvent evt = (ChannelStateEvent) e;
        switch (evt.getState()) {
        case OPEN:
            if (Boolean.TRUE.equals(evt.getValue())) {
                channelOpen(ctx, evt);
            } else {
                channelClosed(ctx, evt);
            }
            break;
        ...
}

Binder打开事件处理

public void channelOpen(
        ChannelHandlerContext ctx,
        ChannelStateEvent evt) {
...
    // 当前channel中的管道工厂重置为NettyServer中管道工厂的匿名实现类
    evt.getChannel().getConfig().setPipelineFactory(getPipelineFactory());
...
    // 将ServerBootstrap参数设置到通道配置：org.jboss.netty.channel.socket.nio.NioServerSocketChannel#config
    evt.getChannel().getConfig().setOptions(parentOptions);
...
    事件通道NioServerSocketChannel绑定地址
    boolean finished = futureQueue.offer(evt.getChannel().bind(localAddress));
    assert finished;
}

通道配置设置：org.jboss.netty.channel.socket.nio.NioServerSocketChannel#config，包含backlog配置，所以backlog配置是通过ServerBootstrap透传进来的
调用通道父类绑定方法，获取管道发送绑定事件，此时的管道已被替换为NettyServer的匿名实现类

public ChannelFuture bind(SocketAddress localAddress) {
    return Channels.bind(this, localAddress);
}
public static ChannelFuture bind(Channel channel, SocketAddress localAddress) {
    ...
    channel.getPipeline().sendDownstream(new DownstreamChannelStateEvent(
            channel, future, ChannelState.BOUND, localAddress));
    return future;
}

默认通道管道发送绑定事件

public void sendDownstream(ChannelEvent e) {
    DefaultChannelHandlerContext tail = getActualDownstreamContext(this.tail);
    if (tail == null) {
        try {
            // 已到终点，sink向下行继续发送事件：NioServerSocketPipelineSink
            getSink().eventSunk(this, e);
            ...
}
        

向下行发送事件

public void eventSunk(
        ChannelPipeline pipeline, ChannelEvent e) throws Exception {
    Channel channel = e.getChannel();
    if (channel instanceof NioServerSocketChannel) {
        handleServerSocket(e);
    } else if (channel instanceof NioSocketChannel) {
        handleAcceptedSocket(e);
    }
}
private void handleServerSocket(ChannelEvent e) {
        if (!(e instanceof ChannelStateEvent)) {
            return;
        }
...
    // 处理绑定事件
    case BOUND:
        if (value != null) {
            bind(channel, future, (SocketAddress) value);
        } else {
            close(channel, future);
        }
        break;
    }
}

绑定事件处理

private void bind(
        NioServerSocketChannel channel, ChannelFuture future,
        SocketAddress localAddress) {
    ...
    try {
        // 服务端套接字绑定地址，并设置backlog参数
        channel.socket.socket().bind(localAddress, channel.getConfig().getBacklog());
        ...
}

ServerSocket绑定地址

public void bind(SocketAddress endpoint, int backlog) throws IOException {
    ...
    if (backlog < 1)
      backlog = 50;
    try {
        SecurityManager security = System.getSecurityManager();
        if (security != null)
            security.checkListen(epoint.getPort());
        getImpl().bind(epoint.getAddress(), epoint.getPort());
        getImpl().listen(backlog);
        bound = true;
    ...
}

没错，netty的backlog参数为int类型，如果dubbo服务端没有指定配置，默认值为0，jdk的ServerSocket判断如果backlog小于1则使用50配置

总结

请问你们dubbo的backlog参数设置的是多少？
dubbo（2.6.6）没有指定backlog配置，所以默认走jdk兜底配置：50