dubbo 线程模型

o_瓜田李下_o

已于 2022-02-08 22:39:20 修改

阅读量1.5k

点赞数

分类专栏： dubbo 文章标签： dubbo

于 2022-02-08 17:07:07 首次发布

本文链接：https://blog.csdn.net/weixin_43931625/article/details/122827657

版权

dubbo 专栏收录该内容

24 篇文章 1 订阅

订阅专栏

dubbo 线程模型

官网：https://dubbo.apache.org/zh/docs/advanced/thread-model/

线程模型

使用dubbo协议做服务调用时，服务提供端线程派发模式及线程池使用

***********

派发模式

线程分类

# I/O线程：nettyServer线程组（boss和worker线程组）
服务提供方nettyServer使用两级线程池，eventLoopGroup（boss线程组）接收客户端的连接请求，
并把完成tcp三次握手的连接请求发给eventLoopGroup（worker线程组）处理

# 业务线程池：executorService线程池
I/O线程如果能迅速处理消费端请求，并且不会发起新的i/o请求，则可直接在I/O线程处理，可以避免线程切换开销；
如果请求处理比较耗时，或者需要发起新的i/o请求（如查询数据库、磁盘读写），则需要将请求派发到业务线程池处理，否则可能会导致I/O线程阻塞

Dispatcher：线程派发接口

@SPI(value = AllDispatcher.NAME, scope = ExtensionScope.FRAMEWORK)  //默认为allDispatcher
public interface Dispatcher {

    /**
     * dispatch the message to threadpool.
     *
     * @param handler
     * @param url
     * @return channel handler
     */
    @Adaptive({Constants.DISPATCHER_KEY, "dispather", "channel.handler"})
    // The last two parameters are reserved for compatibility with the old configuration
    ChannelHandler dispatch(ChannelHandler handler, URL url);

}

派发模式说明

all（AllDispatcher）：所有消息（请求、响应、连接、断开、心跳）都由业务线程池处理
direct（DirectDispatcher）：所有消息都在I/O线程处理，不派发到业务线程池
message（MessageOnlyDispatcher）：请求响应消息在业务线程池处理，其他消息由I/O线程处理
execution（ExecutionDispatcher）：请求消息派发到业务线程池，其他消息由I/O线程处理
connection（ConnectionOrderedDispatcher）：连接、断开消息由I/O线程有序处理，其余消息由业务线程池处理

AllDispatcher

public class AllDispatcher implements Dispatcher {

    public static final String NAME = "all";

    @Override
    public ChannelHandler dispatch(ChannelHandler handler, URL url) {
        return new AllChannelHandler(handler, url);
    }

}

AllChannelHandler

public class AllChannelHandler extends WrappedChannelHandler {

    public AllChannelHandler(ChannelHandler handler, URL url) {
        super(handler, url);
    }

    @Override
    public void connected(Channel channel) throws RemotingException {  //处理请求连接事件
        ExecutorService executor = getSharedExecutorService();  //获取共享线程池，由线程池执行任务
        try {
            executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.CONNECTED));
        } catch (Throwable t) {
            throw new ExecutionException("connect event", channel, getClass() + " error when process connected event .", t);
        }
    }

    @Override
    public void disconnected(Channel channel) throws RemotingException {  //处理请求断开事件
        ExecutorService executor = getSharedExecutorService();
        try {
            executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.DISCONNECTED));
        } catch (Throwable t) {
            throw new ExecutionException("disconnect event", channel, getClass() + " error when process disconnected event .", t);
        }
    }

    @Override
    public void received(Channel channel, Object message) throws RemotingException {
                //请求处理、响应事件
        ExecutorService executor = getPreferredExecutorService(message);
        try {
            executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.RECEIVED, message));
        } catch (Throwable t) {
            if(message instanceof Request && t instanceof RejectedExecutionException){
                sendFeedback(channel, (Request) message, t);
                return;
            }
            throw new ExecutionException(message, channel, getClass() + " error when process received event .", t);
        }
    }

    @Override
    public void caught(Channel channel, Throwable exception) throws RemotingException {
                //异常处理事件
        ExecutorService executor = getSharedExecutorService();
        try {
            executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.CAUGHT, exception));
        } catch (Throwable t) {
            throw new ExecutionException("caught event", channel, getClass() + " error when process caught event .", t);
        }
    }
}

***********

业务线程池

ThreadPool：业务线程池

@SPI(value = "fixed", scope = ExtensionScope.FRAMEWORK)  //默认为fixed线程池
public interface ThreadPool {

    /**
     * Thread pool
     *
     * @param url URL contains thread parameter
     * @return thread pool
     */
    @Adaptive({THREADPOOL_KEY})
    Executor getExecutor(URL url);

}

线程池说明

# fixed：FixedThreadPool（默认）
线程池数量固定，默认为200
空闲线程存活时间设置为0
等待队列长度：int queues = url.getParameter(QUEUES_KEY, DEFAULT_QUEUES)
如果url中没有对应的key（queues），队列长度默认设置为0；
url中设置的queues小于0，队列长度为Integer.MAX；
url中设置的queues大于0，队列长度为queues
拒绝策略为丢弃新任务，并抛出异常
 
# limited：LimitedThreadPool
如果没有设置，核心线程数默认为0，最大线程数默认为200
空闲线程存活时间默认为Long.MAX_VALUE（空闲线程不回收）
等待队列设置同FixedThreadPool
拒绝策略为丢弃新任务，并抛出异常
 
# eager：EagerThreadPool
执行过程：新任务进来时，超过核心线程数，但是没有超过最大线程，
        会创建线程执行任务，如果超过最大线程数，则进入等待队列
如果没有设置，核心线程数默认为0，最大线程数默认为200
空闲线程存活时间默认为60s
等待队列长度如果没有设置默认为200，如果设置的值小于等于0，默认长度为1
拒绝策略为丢弃新任务，并抛出异常
 
# cached：CachedThreadPool
如果没有设置，核心线程数默认为0，最大线程数默认为Integer.MAX_VALUE
空闲线程存活时间默认为60s
等待队列设置同FixedThreadPool
拒绝策略为丢弃新任务，并抛出异常

FixedThreadPool：固定线程池

public class FixedThreadPool implements ThreadPool {

    @Override
    public Executor getExecutor(URL url) {
        String name = url.getParameter(THREAD_NAME_KEY, (String) url.getAttribute(THREAD_NAME_KEY, DEFAULT_THREAD_NAME));
        int threads = url.getParameter(THREADS_KEY, DEFAULT_THREADS);  //url中获取核心线程数，如果没有默认为200
        int queues = url.getParameter(QUEUES_KEY, DEFAULT_QUEUES);     //url中获取队列数，如果没有默认为0
        return new ThreadPoolExecutor(threads, threads, 0, TimeUnit.MILLISECONDS,
                queues == 0 ? new SynchronousQueue<Runnable>() :   //queues为0        ：及时提交队列，队列不存放任务
                        (queues < 0 ? new LinkedBlockingQueue<Runnable>()  //queues小于0，无界队列（队列长度为Integer.MAX）
                                : new LinkedBlockingQueue<Runnable>(queues)), //queues大于0，队列数设置为queues
                new NamedInternalThreadFactory(name, true), new AbortPolicyWithReport(name, url));
    }

}

LimitedThreadPool：可伸缩线程池，线程池数量只增长不回收

public class LimitedThreadPool implements ThreadPool {

    @Override
    public Executor getExecutor(URL url) {
        String name = url.getParameter(THREAD_NAME_KEY, (String) url.getAttribute(THREAD_NAME_KEY, DEFAULT_THREAD_NAME));
        int cores = url.getParameter(CORE_THREADS_KEY, DEFAULT_CORE_THREADS);
        int threads = url.getParameter(THREADS_KEY, DEFAULT_THREADS);
        int queues = url.getParameter(QUEUES_KEY, DEFAULT_QUEUES);
        return new ThreadPoolExecutor(cores, threads, Long.MAX_VALUE, TimeUnit.MILLISECONDS,
                queues == 0 ? new SynchronousQueue<Runnable>() :
                        (queues < 0 ? new LinkedBlockingQueue<Runnable>()
                                : new LinkedBlockingQueue<Runnable>(queues)),
                new NamedInternalThreadFactory(name, true), new AbortPolicyWithReport(name, url));
    }

}

EagerThreadPool：大于核心线程数时，优先创建线程执行任务，超过最大线程数时进入等待队列

public class EagerThreadPool implements ThreadPool {

    @Override
    public Executor getExecutor(URL url) {
        String name = url.getParameter(THREAD_NAME_KEY, (String) url.getAttribute(THREAD_NAME_KEY, DEFAULT_THREAD_NAME));
        int cores = url.getParameter(CORE_THREADS_KEY, DEFAULT_CORE_THREADS);
        int threads = url.getParameter(THREADS_KEY, Integer.MAX_VALUE);
        int queues = url.getParameter(QUEUES_KEY, DEFAULT_QUEUES);
        int alive = url.getParameter(ALIVE_KEY, DEFAULT_ALIVE);

        // init queue and executor
        TaskQueue<Runnable> taskQueue = new TaskQueue<Runnable>(queues <= 0 ? 1 : queues);
        EagerThreadPoolExecutor executor = new EagerThreadPoolExecutor(cores,
                threads,
                alive,
                TimeUnit.MILLISECONDS,
                taskQueue,
                new NamedInternalThreadFactory(name, true),
                new AbortPolicyWithReport(name, url));
        taskQueue.setExecutor(executor);
        return executor;
    }
}

CachedThreadPool：缓存线程池，空闲线程超过一分钟回收

public class CachedThreadPool implements ThreadPool {

    @Override
    public Executor getExecutor(URL url) {
        String name = url.getParameter(THREAD_NAME_KEY, (String) url.getAttribute(THREAD_NAME_KEY, DEFAULT_THREAD_NAME));
        int cores = url.getParameter(CORE_THREADS_KEY, DEFAULT_CORE_THREADS);
        int threads = url.getParameter(THREADS_KEY, Integer.MAX_VALUE);
        int queues = url.getParameter(QUEUES_KEY, DEFAULT_QUEUES);
        int alive = url.getParameter(ALIVE_KEY, DEFAULT_ALIVE);
        return new ThreadPoolExecutor(cores, threads, alive, TimeUnit.MILLISECONDS,
                queues == 0 ? new SynchronousQueue<Runnable>() :
                        (queues < 0 ? new LinkedBlockingQueue<Runnable>()
                                : new LinkedBlockingQueue<Runnable>(queues)),
                new NamedInternalThreadFactory(name, true), new AbortPolicyWithReport(name, url));
    }
}

配置示例

ProtocolConfig

public class ProtocolConfig extends AbstractConfig {

    private static final long serialVersionUID = 6913423882496634749L;

    private String name;         //协议名，线程模式对dubbo协议有效

    /**
     * Service ip address (when there are multiple network cards available)
     */
    private String host;         //服务ip地址
    private Integer port;        //服务端口

    /**
     * Context path
     */
    private String contextpath;

    private String threadpool;    //使用的线程池类型，可选值：fixed、cached、limited、eager
    private String threadname;    //线城池名称
    private Integer corethreads;  //线程池核心线程数
    private Integer threads;      //线程池最大线程数
    private Integer iothreads;    //io线程数
    private Integer alive;        //线程池空闲线程存活时间，单位为毫秒
    private Integer queues;       //线程池等待队列数

    private String dispatcher;    //线程派发模式，可选值：all、direct、message、execution、connection

    /**
     * Max acceptable connections
     */
    private Integer accepts;

    /**
     * Protocol codec
     */
    private String codec;

    /**
     * Serialization
     */
    private String serialization;

    /**
     * Charset
     */
    private String charset;

    /**
     * Payload max length
     */
    private Integer payload;

    /**
     * Buffer size
     */
    private Integer buffer;

    /**
     * Heartbeat interval
     */
    private Integer heartbeat;

    /**
     * Access log
     */
    private String accesslog;

    /**
     * Transporter
     */
    private String transporter;

    /**
     * How information is exchanged
     */
    private String exchanger;

    /**
     * Networker
     */
    private String networker;

    /**
     * Sever impl
     */
    private String server;

    /**
     * Client impl
     */
    private String client;

    /**
     * Supported telnet commands, separated with comma.
     */
    private String telnet;

    /**
     * Command line prompt
     */
    private String prompt;

    /**
     * Status check
     */
    private String status;

    /**
     * Whether to register
     */
    private Boolean register;

    /**
     * whether it is a persistent connection
     */
    //TODO add this to provider config
    private Boolean keepAlive;

    // TODO add this to provider config
    private String optimizer;

    /**
     * The extension
     */
    private String extension;

    /**
     * The customized parameters
     */
    private Map<String, String> parameters;

    private Boolean sslEnabled;

    public ProtocolConfig() {
    public ProtocolConfig(ApplicationModel applicationModel) {
    public ProtocolConfig(String name) {
    public ProtocolConfig(ApplicationModel applicationModel, String name) {
    public ProtocolConfig(String name, int port) {
    public ProtocolConfig(ApplicationModel applicationModel, String name, int port) {

application.yml配置示例

dubbo:
  application:
    name: dubbo-provider
    register-mode: instance
  registry:
    address: zookeeper://localhost:2181
    group: dubbo
    #register-mode: instance
    #register: false
  protocol:
    name: dubbo
    port: 20880
    threadpool: eager
    corethreads: 10
    threads: 100
    queues: 100
    alive: 60000