xxl-job调度执行流程及源码解析

项目地址github：

https://github.com/xuxueli/xxl-job

项目目录介绍：

xxl-job-admin：调度中心（server端启动项目包括页面）
xxl-job-core：公共依赖
xxl-job-executor-samples：执行器Sample示例（选择合适执行器）
：xxl-job-executor-sample-springboot：Springboot 版本管理执行器
：xxl-job-executor-sample-frameless：无框架版本

---

全文目录：

1.架构图
2.xxl-job-admin 调度中心源码解析及相关数据结构分析
3.快慢线程池JobTriggerPoolHelper源码解析
4.任务依赖解析

1.xxl-job-admin 调度中心

Springboot启动类：src/main/java/com/xxl/job/admin/XxlJobAdminApplication.java

启动后调用XxlJobAdminConfig类，初始化XxlJobScheduler类

@Component
public class XxlJobAdminConfig implements InitializingBean, DisposableBean {
    private static XxlJobAdminConfig adminConfig = null;
    public static XxlJobAdminConfig getAdminConfig() {
        return adminConfig;
    }
    // ---------------------- XxlJobScheduler ----------------------
    private XxlJobScheduler xxlJobScheduler;
    @Override
    public void afterPropertiesSet() throws Exception {
        adminConfig = this;
        xxlJobScheduler = new XxlJobScheduler();
        xxlJobScheduler.init();"
    }
    @Override
    public void destroy() throws Exception {
        xxlJobScheduler.destroy();
    }

初始化XxlJobScheduler

public class XxlJobScheduler  {
    private static final Logger logger = LoggerFactory.getLogger(XxlJobScheduler.class);
    public void init() throws Exception {
        // init i18n
        initI18n();
        // 初始化快、慢线程池触发器  调用XxlJobTrigger
        JobTriggerPoolHelper.toStart();
        // admin registry monitor run注册监控器线程启动
        JobRegistryHelper.getInstance().start();
        // admin fail-monitor run 失败监控线程启动
        JobFailMonitorHelper.getInstance().start();
        // admin lose-monitor run ( depend on JobTriggerPoolHelper )回调监控线程启动
        JobCompleteHelper.getInstance().start();
        // admin log report start
        JobLogReportHelper.getInstance().start();
        // start-schedule  ( depend on JobTriggerPoolHelper )作业调度初始化
        JobScheduleHelper.getInstance().start();
        logger.info(">>>>>>>>> init xxl-job admin success.。。");
    }
   ......
)

重点分析 JobScheduleHelper 作业调度类

1. 使用Map<Integer, List<Integer>> 的数据结构存储将要执行的任务清单，其中Integer的范围为0-60，list中存储将要执行的JobID
2. 使用scheduleThread线程扫描mysql中将要执行的调度清单
3. 使用ringThread线程执行第一步map中的执行清单

执行流程图如下：

scheduleThread线程详细代码如下：

1、 从数据库中查询出下次执行时间小于等于当前时间+5s的所有执行任务列表

//悲观锁
preparedStatement = conn.prepareStatement(  "select * from xxl_job_lock where lock_name = 'schedule_lock' for update" );
preparedStatement.execute();
// 1、pre read  查询出下次执行时间小于等于当前时间+5s的执行任务列表
long nowTime = System.currentTimeMillis();
List<XxlJobInfo> scheduleList = XxlJobAdminConfig.getAdminConfig().getXxlJobInfoDao().scheduleJobQuery(nowTime + PRE_READ_MS, preReadCount);

/* 查询语句如下
<select id="scheduleJobQuery" parameterType="java.util.HashMap" resultMap="XxlJobInfo">
SELECT <include refid="Base_Column_List" />
FROM xxl_job_info AS t
WHERE t.trigger_status = 1 
and t.trigger_next_time <![CDATA[ <= ]]> #{maxNextTime}
ORDER BY id ASC
LIMIT #{pagesize}
</select>
 */

2、判断下次执行时间是否小于（nowTime-5s）

if (nowTime > jobInfo.getTriggerNextTime() + PRE_READ_MS) {
  // 2.1、trigger-expire > 5s：pass && make next-trigger-time 
  // 1、misfire match
  MisfireStrategyEnum misfireStrategyEnum = MisfireStrategyEnum.match(jobInfo.getMisfireStrategy(), MisfireStrategyEnum.DO_NOTHING);
  if (MisfireStrategyEnum.FIRE_ONCE_NOW == misfireStrategyEnum) {
    // FIRE_ONCE_NOW 》 trigger
    JobTriggerPoolHelper.trigger(jobInfo.getId(), TriggerTypeEnum.MISFIRE, -1, null, null, null); 
  }

// 2、fresh next
//根据cron的设置计算下次执行时间，并刷新将新的下次执行时间和jobinfo
refreshNextValidTime(jobInfo, new Date());

3、判断 (nowTime-5s) <= 下次执行时间 小于 nowTime ,刷新完下次执行时间后在加入执行队列（此种情况刷新两次下次执行时间）

else if (nowTime > jobInfo.getTriggerNextTime()) {
// nowTime大于下次执行时间，并时间区间在5s以内
// 2.2、trigger-expire < 5s：direct-trigger && make next-trigger-time 
// 1、trigger
//加入到执行的线程池
JobTriggerPoolHelper.trigger(jobInfo.getId(), TriggerTypeEnum.CRON, -1, null, null, null); 
// 2、fresh next
//根据cron的设置计算下次执行时间，并刷新将新的下次执行时间和jobinfo
refreshNextValidTime(jobInfo, new Date()); 
// next-trigger-time in 5s, pre-read again //筛选5s内需要执行的作业
if (jobInfo.getTriggerStatus()==1 && nowTime + PRE_READ_MS > jobInfo.getTriggerNextTime()) { 
    // 1、make ring second 计算出下次执行的秒数
    int ringSecond = (int)((jobInfo.getTriggerNextTime()/1000)%60); 
    // 2、push time ring 将任务和将要执行的秒数添加至执行时间清单 写入 ringData
    pushTimeRing(ringSecond, jobInfo.getId()); 
    // 3、fresh next 刷新下次执行时间
    refreshNextValidTime(jobInfo, new Date(jobInfo.getTriggerNextTime())); 
}

4、nowTime < 下次执行时间 小于 （nowTime+5）,直接加入执行Map,之后修改下次执行时间

else {
 //下次执行时间大于 nowTime
 // 2.3、trigger-pre-read：time-ring trigger && make next-trigger-time 
 // 1、make ring second
 int ringSecond = (int)((jobInfo.getTriggerNextTime()/1000)%60); 
 // 2、push time ring
 pushTimeRing(ringSecond, jobInfo.getId()); 
 // 3、fresh next
 refreshNextValidTime(jobInfo, new Date(jobInfo.getTriggerNextTime()));
}

ringThread线程代码如下

try {
    // second data
    List<Integer> ringItemData = new ArrayList<>();
    int nowSecond = Calendar.getInstance().get(Calendar.SECOND);   // 避免处理耗时太长，跨过刻度，向前校验一个刻度；
    for (int i = 0; i < 2; i++) { //只取两个
        List<Integer> tmpData = ringData.remove( (nowSecond+60-i)%60 );
        if (tmpData != null) {
            ringItemData.addAll(tmpData);
        }
    }
    // ring trigger
    logger.debug(">>>>>>>>>>> xxl-job, time-ring beat : " + nowSecond + " = " + Arrays.asList(ringItemData) );
    if (ringItemData.size() > 0) {
        // do trigger
        for (int jobId: ringItemData) {
            // do trigger
            JobTriggerPoolHelper.trigger(jobId, TriggerTypeEnum.CRON, -1, null, null, null);
        }
        // clear
        ringItemData.clear();
    }
} 

原理如下图：

快慢线程池JobTriggerPoolHelper（fastTriggerPool，slowTriggerPool）

任务通过上述线程查询出来之后通过相应的算法触发任务的执行线程池，对应的类为JobTriggerPoolHelper，初始化 fastTriggerPool 和 slowTriggerPool 两类线程池，目的是为了高可用，调度线程池隔离，拆分为"Fast"和"Slow"两个线程池，1分钟窗口期内任务耗时达500ms超过10次，该窗口期内判定为慢任务，慢任务自动降级进入"Slow"线程池，避免耗尽调度线程，提高系统稳定性；

触发线程池
public static void trigger(int jobId, TriggerTypeEnum triggerType, int failRetryCount, String executorShardingParam, String executorParam, String addressList) {
    helper.addTrigger(jobId, triggerType, failRetryCount, executorShardingParam, executorParam, addressList);
    }

public void addTrigger(final int jobId,
                       final TriggerTypeEnum triggerType,
                       final int failRetryCount,
                       final String executorShardingParam,
                       final String executorParam,
                       final String addressList) {

    // choose thread pool
    ThreadPoolExecutor triggerPool_ = fastTriggerPool;
    AtomicInteger jobTimeoutCount = jobTimeoutCountMap.get(jobId);
    if (jobTimeoutCount!=null && jobTimeoutCount.get() > 10) {      // job-timeout 10 times in 1 min
        triggerPool_ = slowTriggerPool;
    }

    // trigger
    triggerPool_.execute(new Runnable() {
        @Override
        public void run() {

            long start = System.currentTimeMillis();

            try {
                // do trigger
                XxlJobTrigger.trigger(jobId, triggerType, failRetryCount, executorShardingParam, executorParam, addressList);
            } catch (Exception e) {
                logger.error(e.getMessage(), e);
            } finally {

                // check timeout-count-map
                long minTim_now = System.currentTimeMillis()/60000;
                if (minTim != minTim_now) {
                    minTim = minTim_now;
                    jobTimeoutCountMap.clear();
                }

                // incr timeout-count-map
                long cost = System.currentTimeMillis()-start;
                if (cost > 500) {       // ob-timeout threshold 500ms
                    AtomicInteger timeoutCount = jobTimeoutCountMap.putIfAbsent(jobId, new AtomicInteger(1));
                    if (timeoutCount != null) {
                        timeoutCount.incrementAndGet();
                    }
                } 
            } 
        }
    });
}

任务依赖

XXL-Job局限性比较大的一点是：只能通过表xxl_job_info重child_jobid字段进行子任务的配置，并且 只能配置多个任务依赖一个父级任务的情况，不能配置一个子任务依赖多个父级任务的情况

具体详细子任务触发源码如下：

private static void finishJob(XxlJobLog xxlJobLog){
// 1、handle success, to trigger child job
String triggerChildMsg = null;
if (XxlJobContext.HANDLE_COCE_SUCCESS == xxlJobLog.getHandleCode()) {
    XxlJobInfo xxlJobInfo = XxlJobAdminConfig.getAdminConfig().getXxlJobInfoDao().loadById(xxlJobLog.getJobId());
    if (xxlJobInfo!=null && xxlJobInfo.getChildJobId()!=null && xxlJobInfo.getChildJobId().trim().length()>0) {
        triggerChildMsg = "<br><br><span style=\"color:#00c0ef;\" > >>>>>>>>>>>"+ I18nUtil.getString("jobconf_trigger_child_run") +"<<<<<<<<<<< </span><br>";
        String[] childJobIds = xxlJobInfo.getChildJobId().split(",");
        for (int i = 0; i < childJobIds.length; i++) {
            int childJobId = (childJobIds[i]!=null && childJobIds[i].trim().length()>0 && isNumeric(childJobIds[i]))?Integer.valueOf(childJobIds[i]):-1;
            if (childJobId > 0) {
                JobTriggerPoolHelper.trigger(childJobId, TriggerTypeEnum.PARENT, -1, null, null, null);
                ReturnT<String> triggerChildResult = ReturnT.SUCCESS;
                // add msg
                triggerChildMsg += MessageFormat.format(I18nUtil.getString("jobconf_callback_child_msg1"),
                        (i+1),
                        childJobIds.length,
                        childJobIds[i],
                        (triggerChildResult.getCode()==ReturnT.SUCCESS_CODE?I18nUtil.getString("system_success"):I18nUtil.getString("system_fail")),
                        triggerChildResult.getMsg());
            } else {
                triggerChildMsg += MessageFormat.format(I18nUtil.getString("jobconf_callback_child_msg2"),
                        (i+1),
                        childJobIds.length,
                        childJobIds[i]);
            }
        }
    }
}

此 finishJob(XxlJobLog xxlJobLog) 方法是通过 callbackThreadPool 线程池触发 private ReturnTcallback(HandleCallbackParamhandleCallbackParam) 方法中的 XxlJobCompleter.updateHandleInfoAndFinish(log) 进行调用。