目录
4 xxl-job-admin代码（核心类XxlJobScheduler和核心配置类XxlJobAdminConfig）
4.2.1 initI18n() 配置语言版本以及加载任务堵塞策略
4.2.2 JobTriggerPoolHelper.toStart(); 初始化线程池
4.2.3 JobRegistryHelper.getInstance().start(); 注册表相关
4.2.4 JobFailMonitorHelper类 失败日志监控线程
4.2.5 JobCompleteHelper类 （将丢失主机信息调度日志更改状态）
4.2.7 JobScheduleHelper 进行任务调度的核心
4.2.8 XxlJobTrigger 触发器（任务进入快慢线程池之后的实际调用）
公司17年前的项目用quartz较多，17年后相对大型的项目逐步替换使用xxl-job
一、参考资料
官方代码地址：GitHub - xuxueli/xxl-job: A distributed task scheduling framework.（分布式任务调度平台XXL-JOB）
二、本地学习模拟
1 本地搭建项目
基本环境：mysql8.0+、jdk8、maven3.6.3、tomcat9.0
参考官网流程拉取代码并配置，不过由于数据库为mysql8.0+，使用的springboot版本为2.4因此
xxl-job-admin/src/main/resources/application.properties配置需要进行一定调整
management.server.servlet.context-path=/actuator
spring.resources.static-locations=classpath:/static/
spring.datasource.driver-class-name=com.mysql.jdbc.Driver要替换为
management.server.servlet.context-path=/actuator
spring.resources.static-locations=classpath:/static/
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver调整完配置，在8080和8081端口没有被占用的情况下，使用springBoot的的主启动类可以正常启动项目。（8080是默认admin调度中心端口，8081是默认执行器端口）本章内容中一般简称为admin/调度中心和executor/执行器。
admin调度中心路径http://localhost:8080/xxl-job-admin/
2 单个调度中心和单个执行器
官网git项目拉取后即满足要求，下面使用默认调度器和执行器尝试官网推荐的常用任务创建模式。
2.1 GLUE模式（略）
在控制台创建调度器后，可以直接在页面编辑要执行的任务代码，可以在项目运行时修改任务内容。（对编码能力要求高，容易出问题且调试不方便，很难配合业务使用，目前参与的项目中禁用该功能）
例如图中增加控制台数据语句并执行一次，可以在
a）executor控制台看到如下输出
11:21:19.744 logback [xxl-job, EmbedServer bizThreadPool-804149415] INFO c.x.job.core.executor.XxlJobExecutor - >>>>>>>>>>> xxl-job regist JobThread success, jobId:3, handler:com.xxl.job.core.handler.impl.GlueJobHandler@4b4a2128
XXL-JOB, lyhtestGLUE. systemb）admin调度调度中心看到执行日志
2022-08-27 11:21:19 [com.xxl.job.core.thread.JobThread#run]-[133]-[xxl-job, JobThread-3-1661570479744]
----------- xxl-job job execute start -----------
----------- Param:
2022-08-27 11:21:19 [com.xxl.job.core.handler.impl.GlueJobHandler#execute]-[25]-[xxl-job, JobThread-3-1661570479744] ----------- glue.version:1661570474000 -----------
2022-08-27 11:21:19 [sun.reflect.NativeMethodAccessorImpl#invoke0]-[-2]-[xxl-job, JobThread-3-1661570479744] XXL-JOB, lyhtestGLUE.
2022-08-27 11:21:19 [com.xxl.job.core.thread.JobThread#run]-[179]-[xxl-job, JobThread-3-1661570479744]
----------- xxl-job job execute end(finish) -----------
----------- Result: handleCode=200, handleMsg = null
2022-08-27 11:21:19 [com.xxl.job.core.thread.TriggerCallbackThread#callbackLog]-[197]-[xxl-job, executor TriggerCallbackThread]
----------- xxl-job job callback finish.
[Load Log Finish]2.2 BEAN模式（常用）
在代码里编写Job任务，再由调度中心控制（任务修改后需要重启项目）。这种在目前项目中用的多，容易调试和排错，运行稳定，不容易被侵入。
2.2.1 编写一个测试案例
如下图，在executor里编写定时任务，类需要加@Component注解供spring识别，方法需要增加@XxlJob注解供admin调度器识别。这里线程会睡7s，而admin配置的执行间隔为5s一次。
备注@XxlJob除了设定任务名value，还有init和destroy可以调用（默认不调用）
@Component
public class LyhXxlJob {
private static Logger logger = LoggerFactory.getLogger(LyhXxlJob.class);
// 调度任务名
@XxlJob("lyhFirstXxlJob")
public void lyhFirstXxlJob() throws InterruptedException {
System.out.println("lyhFirstXxlJob start:" + new Date());
// 睡7s
Thread.sleep(7000);
System.out.println("lyhFirstXxlJob end:" + new Date());
}
}
2.2.2 测试不同阻塞处理策略
框架自带三种类型策略
【单机串行】和【丢弃后续调度】都是以保证任务完整运行为前提的，【覆盖之前调度】则是以保证调度复核设置。
a）单机串行
上图阻塞处理策略配置的是【单机串行】看调度日志和输出日志，程序运行时间7s超过调度任务间隔时间5s时，调度方式会变成等待前一次程序运行完后才执行，而不是并行执行或是跳过某次任务。（调度任务仍是5s秒调用一次）
lyhFirstXxlJob start:Sat Aug 27 18:13:36 CST 2022
lyhFirstXxlJob end:Sat Aug 27 18:13:43 CST 2022
lyhFirstXxlJob start:Sat Aug 27 18:13:43 CST 2022
lyhFirstXxlJob end:Sat Aug 27 18:13:50 CST 2022
lyhFirstXxlJob start:Sat Aug 27 18:13:50 CST 2022
lyhFirstXxlJob end:Sat Aug 27 18:13:57 CST 2022
lyhFirstXxlJob start:Sat Aug 27 18:13:57 CST 2022
lyhFirstXxlJob end:Sat Aug 27 18:14:04 CST 2022
lyhFirstXxlJob start:Sat Aug 27 18:14:04 CST 2022
lyhFirstXxlJob end:Sat Aug 27 18:14:11 CST 2022
lyhFirstXxlJob start:Sat Aug 27 18:14:11 CST 2022b）丢弃后续调度
下图的失败只是调度中心调度任务时失败，而调度成功的任务本身运行是正常的。
lyhFirstXxlJob start:Sun Aug 28 16:47:35 CST 2022
lyhFirstXxlJob end:Sun Aug 28 16:47:42 CST 2022
lyhFirstXxlJob start:Sun Aug 28 16:47:45 CST 2022
lyhFirstXxlJob end:Sun Aug 28 16:47:52 CST 2022
lyhFirstXxlJob start:Sun Aug 28 16:47:55 CST 2022
lyhFirstXxlJob end:Sun Aug 28 16:48:02 CST 2022
lyhFirstXxlJob start:Sun Aug 28 16:48:05 CST 2022
lyhFirstXxlJob end:Sun Aug 28 16:48:12 CST 2022code：500
msg：block strategy effect：Discard Later把【失败重试次数】设置为1，再次测试。
看到失败重试不会影响本身调度频率，并且不符合调用条件时也会失败。
c）覆盖之前调度
保证调度器每次调度都是成功的，任务未执行完时直接杀死任务。并且也会触发失败重试。
17:00:30.012 logback [xxl-job, EmbedServer bizThreadPool-92347499] INFO c.x.job.core.executor.XxlJobExecutor - >>>>>>>>>>> xxl-job regist JobThread success, jobId:4, handler:com.xxl.job.core.handler.impl.MethodJobHandler@6707a4bf[class com.xxl.job.executor.service.jobhandler.LyhXxlJob#lyhFirstXxlJob]
lyhFirstXxlJob start:Sun Aug 28 17:00:30 CST 2022
17:00:35.017 logback [xxl-job, EmbedServer bizThreadPool-92347499] INFO c.x.job.core.executor.XxlJobExecutor - >>>>>>>>>>> xxl-job regist JobThread success, jobId:4, handler:com.xxl.job.core.handler.impl.MethodJobHandler@6707a4bf[class com.xxl.job.executor.service.jobhandler.LyhXxlJob#lyhFirstXxlJob]
lyhFirstXxlJob start:Sun Aug 28 17:00:35 CST 2022
17:00:35.084 logback [xxl-job, JobThread-4-1661677230012] INFO com.xxl.job.core.thread.JobThread - >>>>>>>>>>> xxl-job JobThread stoped, hashCode:Thread[xxl-job, JobThread-4-1661677230012,10,main]
17:00:36.379 logback [xxl-job, EmbedServer bizThreadPool-92347499] INFO c.x.job.core.executor.XxlJobExecutor - >>>>>>>>>>> xxl-job regist JobThread success, jobId:4, handler:com.xxl.job.core.handler.impl.MethodJobHandler@6707a4bf[class com.xxl.job.executor.service.jobhandler.LyhXxlJob#lyhFirstXxlJob]
lyhFirstXxlJob start:Sun Aug 28 17:00:36 CST 2022
17:00:36.414 logback [xxl-job, JobThread-4-1661677235017] INFO com.xxl.job.core.thread.JobThread - >>>>>>>>>>> xxl-job JobThread stoped, hashCode:Thread[xxl-job, JobThread-4-1661677235017,10,main]
17:00:40.013 logback [xxl-job, EmbedServer bizThreadPool-92347499] INFO c.x.job.core.executor.XxlJobExecutor - >>>>>>>>>>> xxl-job regist JobThread success, jobId:4, handler:com.xxl.job.core.handler.impl.MethodJobHandler@6707a4bf[class com.xxl.job.executor.service.jobhandler.LyhXxlJob#lyhFirstXxlJob]
lyhFirstXxlJob start:Sun Aug 28 17:00:40 CST 2022
17:00:40.047 logback [xxl-job, JobThread-4-1661677236379] INFO com.xxl.job.core.thread.JobThread - >>>>>>>>>>> xxl-job JobThread stoped, hashCode:Thread[xxl-job, JobThread-4-1661677236379,10,main]
17:00:45.022 logback [xxl-job, EmbedServer bizThreadPool-92347499] INFO c.x.job.core.executor.XxlJobExecutor - >>>>>>>>>>> xxl-job regist JobThread success, jobId:4, handler:com.xxl.job.core.handler.impl.MethodJobHandler@6707a4bf[class com.xxl.job.executor.service.jobhandler.LyhXxlJob#lyhFirstXxlJob]
lyhFirstXxlJob start:Sun Aug 28 17:00:45 CST 2022
17:00:45.025 logback [xxl-job, JobThread-4-1661677240013] INFO com.xxl.job.core.thread.JobThread - >>>>>>>>>>> xxl-job JobThread stoped, hashCode:Thread[xxl-job, JobThread-4-1661677240013,10,main]
17:00:46.414 logback [xxl-job, EmbedServer bizThreadPool-92347499] INFO c.x.job.core.executor.XxlJobExecutor - >>>>>>>>>>> xxl-job regist JobThread success, jobId:4, handler:com.xxl.job.core.handler.impl.MethodJobHandler@6707a4bf[class com.xxl.job.executor.service.jobhandler.LyhXxlJob#lyhFirstXxlJob]
lyhFirstXxlJob start:Sun Aug 28 17:00:46 CST 2022
17:00:46.422 logback [xxl-job, JobThread-4-1661677245022] INFO com.xxl.job.core.thread.JobThread - >>>>>>>>>>> xxl-job JobThread stoped, hashCode:Thread[xxl-job, JobThread-4-1661677245022,10,main]
17:00:46.468 logback [xxl-job, EmbedServer bizThreadPool-92347499] INFO c.x.job.core.executor.XxlJobExecutor - >>>>>>>>>>> xxl-job regist JobThread success, jobId:4, handler:com.xxl.job.core.handler.impl.MethodJobHandler@6707a4bf[class com.xxl.job.executor.service.jobhandler.LyhXxlJob#lyhFirstXxlJob]
lyhFirstXxlJob start:Sun Aug 28 17:00:46 CST 2022
17:00:46.475 logback [xxl-job, JobThread-4-1661677246414] INFO com.xxl.job.core.thread.JobThread - >>>>>>>>>>> xxl-job JobThread stoped, hashCode:Thread[xxl-job, JobThread-4-1661677246414,10,main]
17:00:50.015 logback [xxl-job, EmbedServer bizThreadPool-92347499] INFO c.x.job.core.executor.XxlJobExecutor - >>>>>>>>>>> xxl-job regist JobThread success, jobId:4, handler:com.xxl.job.core.handler.impl.MethodJobHandler@6707a4bf[class com.xxl.job.executor.service.jobhandler.LyhXxlJob#lyhFirstXxlJob]
lyhFirstXxlJob start:Sun Aug 28 17:00:50 CST 2022
17:00:50.018 logback [xxl-job, JobThread-4-1661677246467] INFO com.xxl.job.core.thread.JobThread - >>>>>>>>>>> xxl-job JobThread stoped, hashCode:Thread[xxl-job, JobThread-4-1661677246467,10,main]
17:00:56.490 logback [xxl-job, EmbedServer bizThreadPool-92347499] INFO c.x.job.core.executor.XxlJobExecutor - >>>>>>>>>>> xxl-job regist JobThread success, jobId:4, handler:com.xxl.job.core.handler.impl.MethodJobHandler@6707a4bf[class com.xxl.job.executor.service.jobhandler.LyhXxlJob#lyhFirstXxlJob]
lyhFirstXxlJob start:Sun Aug 28 17:00:56 CST 2022
17:00:56.494 logback [xxl-job, JobThread-4-1661677250015] INFO com.xxl.job.core.thread.JobThread - >>>>>>>>>>> xxl-job JobThread stoped, hashCode:Thread[xxl-job, JobThread-4-1661677250015,10,main]
lyhFirstXxlJob end:Sun Aug 28 17:01:03 CST 2022
lyhFirstXxlJob start:Sun Aug 28 17:01:06 CST 2022
lyhFirstXxlJob end:Sun Aug 28 17:01:13 CST 2022block strategy effect：Cover Early [job running, killed]3 单调度中心+多执行器
参考案例新增执行器，然后可以在任务管理里切换执行器，不同执行器的任务不会自动共享。不过这里新增任务时，也使用了lyhFirstXxlJob这一个JobHandler。测试下不同调度器调同一个任务是否会有影响。
3.1 创建多执行器环境
3.1.1搭建三台的XxlJobExecutor
application-02.properties和application-03.properties需要进行一定配置修改。
// application-02
server.port=8082
xxl.job.executor.ip=127.0.0.1
xxl.job.executor.port=9998
// application-03
server.port=8083
xxl.job.executor.ip=127.0.0.1
xxl.job.executor.port=9997官网备注：
三台执行器在同一个集群内：xxl.job.executor.appname=xxl-job-executor-sample不变，供调度中心识别。
执行器回调地址保持一致：xxl.job.executor.address=不变，执行器可以根据该配置进行自动注册等操作。
新建对应的springboot启动类，然后就可以同时启动多个Executor项目。
3.1.2启动一台Admin调度中心
在调度中心执行器管理的OnLine机器地址处，看到配置好并已经启动的Executor说明配置完成。
然后就可以在任务管理处配置任务信息，例如路由策略
3.2 测试不同阻塞处理策略
3.2.1 任务配置
单调度中心+多执行器情况下
任务调度频率为5s一次，任务执行时间为22s。
a）路由策略为随机
仍是5s调用一次，不过是admin按时间节点调用随机执行器，不会看执行器是否在执行任务。而各执行器在接收到调度任务后，本身运行情况与单机模式相同。
b）路由策略为轮询
5s调用一次，三个调度器依次调用。而各执行器本身运行情况与单机模式相同。
还进行了其他测试，总体来说由于admin调度器和executor是解耦合的，常规配置时功能相互独立。admin只是调用，将任务分配给各个executor，具体执行逻辑由每个executor自身控制。
4 xxl-job-admin代码（核心类XxlJobScheduler和核心配置类XxlJobAdminConfig）
XxlJobScheduler：是调度器初始化核心，负责扫描需要调用的组件。
XxlJobAdminConfig：用于加载配置文件，读取配置文件后调用XxlJobScheduler.init()方法启动配置。
4.1 XxlJobAdminConfig
读取application.properties配置文件数据
a）由于实现了InitializingBean接口，因此可以重写afterPropertiesSet()方法初始化调度器，读取配置后调用XxlJobScheduler.init()方法初始化XxlJobScheduler：
@Override
public void afterPropertiesSet() throws Exception {
// 静态声明——>初始化一个单例对象
adminConfig = this;
// 初始化xxljob调度器
xxlJobScheduler = new XxlJobScheduler();
// 调用初始化方法
xxlJobScheduler.init();
}b）同理实现DisposableBean接口里的destroy()方法销毁调度器
@Override
public void destroy() throws Exception {
xxlJobScheduler.destroy();
}4.2 XxlJobScheduler
public void init() throws Exception {
// init i18n 调度中心国际化配置
initI18n();
// admin trigger pool start 初始化触发器线程池
JobTriggerPoolHelper.toStart();
// admin registry monitor run 初始化注册表有关方法
JobRegistryHelper.getInstance().start();
// admin fail-monitor run 初始化监视器,主要失败发送邮箱,重试触发器
JobFailMonitorHelper.getInstance().start();
// admin lose-monitor run ( depend on JobTriggerPoolHelper ) 将丢失主机信息调度日志更改状态
JobCompleteHelper.getInstance().start();
// admin log report start 初始化统计表
JobLogReportHelper.getInstance().start();
// start-schedule ( depend on JobTriggerPoolHelper ) 启动调度器
JobScheduleHelper.getInstance().start();
logger.info(">>>>>>>>> init xxl-job admin success.");
} public void destroy() throws Exception {
// 总体来说停止顺序 是init()加载顺序的反向
// stop-schedule 停止调度器
JobScheduleHelper.getInstance().toStop();
// admin log report stop 停止初始化统计表
JobLogReportHelper.getInstance().toStop();
// admin lose-monitor stop 停止丢失处理相关方法
JobCompleteHelper.getInstance().toStop();
// admin fail-monitor stop 停止监视器
JobFailMonitorHelper.getInstance().toStop();
// admin registry stop 停止注册表
JobRegistryHelper.getInstance().toStop();
// admin trigger pool stop 停止触发器线程池
JobTriggerPoolHelper.toStop();
}4.2.1 initI18n() 配置语言版本以及加载任务堵塞策略
启动时通过打断点方式查看具体执行了哪些操作
private void initI18n(){
for (ExecutorBlockStrategyEnum item:ExecutorBlockStrategyEnum.values()) {
item.setTitle(I18nUtil.getString("jobconf_block_".concat(item.name())));
}
}a) 国际化语言配置：通过I18nUtil.getString方法加载src/main/resources/i18n目录下的语言配置，具体读取哪一个是 application.properties文件得xxl.job.i18n参数确定的（新加语言转化properties文件的话需要修改XxlJobAdminConfig.getI18n()方法，默认是message_zh_CN.properties这一个）。
b) ExecutorBlockStrategyEnum：任务阻塞策略枚举类，for循环给每个策略都配置一份国际化语言配置。策略具体包括SERIAL_EXECUTION、DISCARD_LATER、COVER_EARLY对应下图的高级配置-阻塞处理策略。
4.2.2 JobTriggerPoolHelper.toStart(); 初始化线程池
a> 主要工作是初始化快慢两种线程池，并没有开始使用。参数对应application.properties的配置，且XxlJobAdminConfig代码中限制了两个参数的最小值
public void start(){
// 快线程池
fastTriggerPool = new ThreadPoolExecutor(
10,
XxlJobAdminConfig.getAdminConfig().getTriggerPoolFastMax(),
60L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<Runnable>(1000),
new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "xxl-job, admin JobTriggerPoolHelper-fastTriggerPool-" + r.hashCode());
}
});
// 慢线程池
slowTriggerPool = new ThreadPoolExecutor(
10,
XxlJobAdminConfig.getAdminConfig().getTriggerPoolSlowMax(),
60L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<Runnable>(2000),
new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "xxl-job, admin JobTriggerPoolHelper-slowTriggerPool-" + r.hashCode());
}
});
}b>构建线程池使用了ThreadPoolExecutor，且任务队列使用了LinkedBlockingQueue
/**
* 线程池
* @param corePoolSize 核心线程数
* @param maximumPoolSize 最大线程数
* @param keepAliveTime 线程空闲时间
* @param unit 时间单位
* @param workQueue 任务队列
* @param threadFactory 线程工厂
*/
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory) {
this(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue,
threadFactory, defaultHandler);
}c> 具体使用一般是调用JobTriggerPoolHelper.trigger或JobTriggerPoolHelper.addTrigger方法实现。
public void addTrigger(final int jobId,
final TriggerTypeEnum triggerType,
final int failRetryCount,
final String executorShardingParam,
final String executorParam,
final String addressList) {
// 默认用的快线程
ThreadPoolExecutor triggerPool_ = fastTriggerPool;
// AtomicInteger 多线程 原子性，这里根据上下文代码是任务执行时间超过500ms的次数
AtomicInteger jobTimeoutCount = jobTimeoutCountMap.get(jobId);
// 一分钟内执行次数超过十次
if (jobTimeoutCount!=null && jobTimeoutCount.get() > 10) { // job-timeout 10 times in 1 min
triggerPool_ = slowTriggerPool;
}
// trigger
triggerPool_.execute(new Runnable() {
@Override
public void run() {
long start = System.currentTimeMillis();
try {
// do trigger 任务触发相关
XxlJobTrigger.trigger(jobId, triggerType, failRetryCount, executorShardingParam, executorParam, addressList);
} catch (Exception e) {
logger.error(e.getMessage(), e);
} finally {
// 下面主要是统计任务执行时间的，以便下次执行时选用快线程池还是慢线程池
// check timeout-count-map
long minTim_now = System.currentTimeMillis()/60000;
if (minTim != minTim_now) {
minTim = minTim_now;
jobTimeoutCountMap.clear();
}
// incr timeout-count-map
long cost = System.currentTimeMillis()-start;
// 超过500ms时任务超时执行次数加一
if (cost > 500) {
AtomicInteger timeoutCount = jobTimeoutCountMap.putIfAbsent(jobId, new AtomicInteger(1));
if (timeoutCount != null) {
timeoutCount.incrementAndGet();
}
}
}
}
});
}根据JobTriggerPoolHelper的addTrigger方法可以知道，优先使用快线程，但是如果一分钟之内某个任务的执行时间有10次以上超过500ms，那么就把该任务放入慢线程池执行。
addTrigger会调用XxlJobTrigger.trigger(jobId, triggerType, failRetryCount, executorShardingParam, executorParam, addressList);方法，这是任务触发相关部分。
4.2.3 JobRegistryHelper.getInstance().start(); 注册表相关
JobRegistryHelper.getInstance().start(); 单例启动
a）创建一个registryOrRemoveThreadPool线程池，用于服务注册或者服务删除
主要用在JobRegistryHelper类的 registry、toStop、registryRemove方法
b）并启动一个线程registryMonitorThread监听服务状态
public void start(){
// for registry or remove 服务注册或删除线程池
registryOrRemoveThreadPool = new ThreadPoolExecutor(
2,
10,
30L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<Runnable>(2000),
new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "xxl-job, admin JobRegistryMonitorHelper-registryOrRemoveThreadPool-" + r.hashCode());
}
},
new RejectedExecutionHandler() {
@Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
r.run();
logger.warn(">>>>>>>>>>> xxl-job, registry or remove too fast, match threadpool rejected handler(run now).");
}
});
// for monitor 注册服务监听器
registryMonitorThread = new Thread(new Runnable() {
@Override
public void run() {
while (!toStop) {
try {
// auto registry group 获取配置为自动注册的执行器 xxl_job_group表 address_type 执行器地址类型：0=自动注册、1=手动录入
List<XxlJobGroup> groupList = XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().findByAddressType(0);
if (groupList!=null && !groupList.isEmpty()) {
// remove dead address (admin/executor) 找到90s内没有数据更新（没有心跳）的机器 xxl_job_registry表
List<Integer> ids = XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().findDead(RegistryConfig.DEAD_TIMEOUT, new Date());
// 从注册表中移除机器数据
if (ids!=null && ids.size()>0) {
XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().removeDead(ids);
}
// fresh online address (admin/executor) 归集执行器下的注册机ip地址
HashMap<String, List<String>> appAddressMap = new HashMap<String, List<String>>();
// 找到90s内有数据更新（有心跳）的机器 xxl_job_registry表
List<XxlJobRegistry> list = XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().findAll(RegistryConfig.DEAD_TIMEOUT, new Date());
if (list != null) {
for (XxlJobRegistry item: list) {
if (RegistryConfig.RegistType.EXECUTOR.name().equals(item.getRegistryGroup())) {
String appname = item.getRegistryKey();
List<String> registryList = appAddressMap.get(appname);
if (registryList == null) {
registryList = new ArrayList<String>();
}
if (!registryList.contains(item.getRegistryValue())) {
registryList.add(item.getRegistryValue());
}
appAddressMap.put(appname, registryList);
}
}
}
// fresh group address 将注册表中的数据归档存入到执行器中并入库
for (XxlJobGroup group: groupList) {
List<String> registryList = appAddressMap.get(group.getAppname());
String addressListStr = null;
if (registryList!=null && !registryList.isEmpty()) {
Collections.sort(registryList);
StringBuilder addressListSB = new StringBuilder();
for (String item:registryList) {
addressListSB.append(item).append(",");
}
addressListStr = addressListSB.toString();
addressListStr = addressListStr.substring(0, addressListStr.length()-1);
}
group.setAddressList(addressListStr);
group.setUpdateTime(new Date());
XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().update(group);
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(">>>>>>>>>>> xxl-job, job registry monitor thread error:{}", e);
}
}
try {
TimeUnit.SECONDS.sleep(RegistryConfig.BEAT_TIMEOUT);
} catch (InterruptedException e) {
if (!toStop) {
logger.error(">>>>>>>>>>> xxl-job, job registry monitor thread error:{}", e);
}
}
}
logger.info(">>>>>>>>>>> xxl-job, job registry monitor thread stop");
}
});
registryMonitorThread.setDaemon(true);
registryMonitorThread.setName("xxl-job, admin JobRegistryMonitorHelper-registryMonitorThread");
registryMonitorThread.start();
}4.2.4 JobFailMonitorHelper类 失败日志监控线程
JobFailMonitorHelper.getInstance().start();单例启动，启动另一个线程进行邮件预警
public void start(){
monitorThread = new Thread(new Runnable() {
@Override
public void run() {
// monitor
while (!toStop) {
try {
// 从xxl_job_log 表中获取执行失败的记录
List<Long> failLogIds = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().findFailJobLogIds(1000);
if (failLogIds!=null && !failLogIds.isEmpty()) {
for (long failLogId: failLogIds) {
// lock log 锁定日志记录
int lockRet = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().updateAlarmStatus(failLogId, 0, -1);
if (lockRet < 1) {
continue;
}
// xxl_job_log 表日志记录
XxlJobLog log = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().load(failLogId);
// xxl_job_info 表任务详情
XxlJobInfo info = XxlJobAdminConfig.getAdminConfig().getXxlJobInfoDao().loadById(log.getJobId());
// 1、fail retry monitor
// executorFailRetryCount; 失败重试次数
if (log.getExecutorFailRetryCount() > 0) {
// 发起重试 JobTriggerPoolHelper.trigger()
JobTriggerPoolHelper.trigger(log.getJobId(), TriggerTypeEnum.RETRY, (log.getExecutorFailRetryCount()-1), log.getExecutorShardingParam(), log.getExecutorParam(), null);
String retryMsg = "<br><br><span style=\"color:#F39C12;\" > >>>>>>>>>>>"+ I18nUtil.getString("jobconf_trigger_type_retry") +"<<<<<<<<<<< </span><br>";
log.setTriggerMsg(log.getTriggerMsg() + retryMsg);
XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().updateTriggerInfo(log);
}
// 2、fail alarm monitor 失败任务报警
// 告警状态：0-默认、-1=锁定状态、1-无需告警、2-告警成功、3-告警失败
int newAlarmStatus = 0;
if (info != null) {
// 这里是调用email预警
boolean alarmResult = XxlJobAdminConfig.getAdminConfig().getJobAlarmer().alarm(info, log);
newAlarmStatus = alarmResult?2:3;
} else {
newAlarmStatus = 1;
}
// 更新预警状态
XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().updateAlarmStatus(failLogId, -1, newAlarmStatus);
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(">>>>>>>>>>> xxl-job, job fail monitor thread error:{}", e);
}
}
try {
TimeUnit.SECONDS.sleep(10);
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
logger.info(">>>>>>>>>>> xxl-job, job fail monitor thread stop");
}
});
monitorThread.setDaemon(true);
monitorThread.setName("xxl-job, admin JobFailMonitorHelper");
monitorThread.start();
}4.2.5 JobCompleteHelper类 （将丢失主机信息调度日志更改状态）
JobCompleteHelper.getInstance().start();单例启动
a）创建一个callbackThreadPool线程池用来更新xxl_job_log表记录
b）启动一个monitorThread线程，处理执行超时和丢失的任务
public void start(){
// for callback
callbackThreadPool = new ThreadPoolExecutor(
2,
20,
30L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<Runnable>(3000),
new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "xxl-job, admin JobLosedMonitorHelper-callbackThreadPool-" + r.hashCode());
}
},
new RejectedExecutionHandler() {
@Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
r.run();
logger.warn(">>>>>>>>>>> xxl-job, callback too fast, match threadpool rejected handler(run now).");
}
});
// for monitor
monitorThread = new Thread(new Runnable() {
@Override
public void run() {
// wait for JobTriggerPoolHelper-init
try {
TimeUnit.MILLISECONDS.sleep(50);
} catch (InterruptedException e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
// monitor
while (!toStop) {
try {
// 任务结果丢失处理：调度记录停留在 "运行中" 状态超过10min，且对应执行器心跳注册失败不在线，则将本地调度主动标记失败；
Date losedTime = DateUtil.addMinutes(new Date(), -10);
List<Long> losedJobIds = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().findLostJobIds(losedTime);
if (losedJobIds!=null && losedJobIds.size()>0) {
for (Long logId: losedJobIds) {
XxlJobLog jobLog = new XxlJobLog();
jobLog.setId(logId);
jobLog.setHandleTime(new Date());
jobLog.setHandleCode(ReturnT.FAIL_CODE);
jobLog.setHandleMsg( I18nUtil.getString("joblog_lost_fail") );
XxlJobCompleter.updateHandleInfoAndFinish(jobLog);
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(">>>>>>>>>>> xxl-job, job fail monitor thread error:{}", e);
}
}
try {
TimeUnit.SECONDS.sleep(60);
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
logger.info(">>>>>>>>>>> xxl-job, JobLosedMonitorHelper stop");
}
});
monitorThread.setDaemon(true);
monitorThread.setName("xxl-job, admin JobLosedMonitorHelper");
monitorThread.start();
}4.2.6 JobLogReportHelper类
主要是统计xxl_job_log表数据形成报表等，以及根据配置时间清理xxl_job_log里的过期日志
public void start(){
logrThread = new Thread(new Runnable() {
@Override
public void run() {
// last clean log time
long lastCleanLogTime = 0;
while (!toStop) {
// 1、log-report refresh: refresh log report in 3 days
// 刷新最近三天的日志
try {
for (int i = 0; i < 3; i++) {
// today
Calendar itemDay = Calendar.getInstance();
itemDay.add(Calendar.DAY_OF_MONTH, -i);
itemDay.set(Calendar.HOUR_OF_DAY, 0);
itemDay.set(Calendar.MINUTE, 0);
itemDay.set(Calendar.SECOND, 0);
itemDay.set(Calendar.MILLISECOND, 0);
Date todayFrom = itemDay.getTime();
itemDay.set(Calendar.HOUR_OF_DAY, 23);
itemDay.set(Calendar.MINUTE, 59);
itemDay.set(Calendar.SECOND, 59);
itemDay.set(Calendar.MILLISECOND, 999);
Date todayTo = itemDay.getTime();
// refresh log-report every minute
XxlJobLogReport xxlJobLogReport = new XxlJobLogReport();
xxlJobLogReport.setTriggerDay(todayFrom);
xxlJobLogReport.setRunningCount(0);
xxlJobLogReport.setSucCount(0);
xxlJobLogReport.setFailCount(0);
// 查询xxl_job_log表记录来生成报表
Map<String, Object> triggerCountMap = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().findLogReport(todayFrom, todayTo);
if (triggerCountMap!=null && triggerCountMap.size()>0) {
int triggerDayCount = triggerCountMap.containsKey("triggerDayCount")?Integer.valueOf(String.valueOf(triggerCountMap.get("triggerDayCount"))):0;
int triggerDayCountRunning = triggerCountMap.containsKey("triggerDayCountRunning")?Integer.valueOf(String.valueOf(triggerCountMap.get("triggerDayCountRunning"))):0;
int triggerDayCountSuc = triggerCountMap.containsKey("triggerDayCountSuc")?Integer.valueOf(String.valueOf(triggerCountMap.get("triggerDayCountSuc"))):0;
int triggerDayCountFail = triggerDayCount - triggerDayCountRunning - triggerDayCountSuc;
xxlJobLogReport.setRunningCount(triggerDayCountRunning);
xxlJobLogReport.setSucCount(triggerDayCountSuc);
xxlJobLogReport.setFailCount(triggerDayCountFail);
}
// do refresh
// 更新或者新增报表记录到xxl_job_log_report
int ret = XxlJobAdminConfig.getAdminConfig().getXxlJobLogReportDao().update(xxlJobLogReport);
if (ret < 1) {
XxlJobAdminConfig.getAdminConfig().getXxlJobLogReportDao().save(xxlJobLogReport);
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(">>>>>>>>>>> xxl-job, job log report thread error:{}", e);
}
}
// 2、log-clean: switch open & once each day
// 日志清理，判断上次清理时间是不是一天前
if (XxlJobAdminConfig.getAdminConfig().getLogretentiondays()>0
&& System.currentTimeMillis() - lastCleanLogTime > 24*60*60*1000) {
// expire-time
Calendar expiredDay = Calendar.getInstance();
expiredDay.add(Calendar.DAY_OF_MONTH, -1 * XxlJobAdminConfig.getAdminConfig().getLogretentiondays());
expiredDay.set(Calendar.HOUR_OF_DAY, 0);
expiredDay.set(Calendar.MINUTE, 0);
expiredDay.set(Calendar.SECOND, 0);
expiredDay.set(Calendar.MILLISECOND, 0);
Date clearBeforeTime = expiredDay.getTime();
// clean expired log
// 清理过期日志，
List<Long> logIds = null;
do {
logIds = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().findClearLogIds(0, 0, clearBeforeTime, 0, 1000);
if (logIds!=null && logIds.size()>0) {
XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().clearLog(logIds);
}
} while (logIds!=null && logIds.size()>0);
// update clean time
lastCleanLogTime = System.currentTimeMillis();
}
try {
TimeUnit.MINUTES.sleep(1);
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
logger.info(">>>>>>>>>>> xxl-job, job log report thread stop");
}
});
logrThread.setDaemon(true);
logrThread.setName("xxl-job, admin JobLogReportHelper");
logrThread.start();
}4.2.7 JobScheduleHelper 进行任务调度的核心
JobScheduleHelper控制调度任务的启动，会初始化两个线程来判断定时任务是否执行
1）scheduleThread基础调度线程（根据一定间隔轮循查看需要执行哪些定时任务）
2）ringThread时间轮调度线程（针对下次触发时间不到5S的任务）
public void start(){
// schedule thread
scheduleThread = new Thread(new Runnable() {
@Override
public void run() {
try {
// 这里睡眠的目的是为了让线程时间对齐整秒，时间范围是5S内。
TimeUnit.MILLISECONDS.sleep(5000 - System.currentTimeMillis()%1000 );
} catch (InterruptedException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
logger.info(">>>>>>>>> init xxl-job admin scheduler success.");
// pre-read count: treadpool-size * trigger-qps (each trigger cost 50ms, qps = 1000/50 = 20)
// （调度快线程池最大线程配置 + 调度慢线程池最大线程配置） * 20
int preReadCount = (XxlJobAdminConfig.getAdminConfig().getTriggerPoolFastMax() + XxlJobAdminConfig.getAdminConfig().getTriggerPoolSlowMax()) * 20;
while (!scheduleThreadToStop) {
// Scan Job
long start = System.currentTimeMillis();
Connection conn = null;
Boolean connAutoCommit = null;
PreparedStatement preparedStatement = null;
boolean preReadSuc = true;
try {
// 手动获取链接并且设置为不自动提交
conn = XxlJobAdminConfig.getAdminConfig().getDataSource().getConnection();
connAutoCommit = conn.getAutoCommit();
conn.setAutoCommit(false);
// 获取锁（悲观锁） 这里主要是为了避免 集群式调度中心重复调用任务的情况
// 因为查询语句默认是没有锁的，select * for update 会让select语句产生一个排它锁(X), 这个锁和update的效果一样，会使两个事务无法同时更新一条记录
preparedStatement = conn.prepareStatement( "select * from xxl_job_lock where lock_name = 'schedule_lock' for update" );
preparedStatement.execute();
// tx start
// 1、pre read 由PRE_READ_MS控制，默认是5S
long nowTime = System.currentTimeMillis();
// 获取xxl_job_info表中读取触发时间小于 当前时间+5s的所有任务，以这个范围做任务触发判断
List<XxlJobInfo> scheduleList = XxlJobAdminConfig.getAdminConfig().getXxlJobInfoDao().scheduleJobQuery(nowTime + PRE_READ_MS, preReadCount);
/*
* 汇总下下面的业务判断
* 任务触发判断
* 1.当前时间 > 下次调度时间+5s 即超过有效误差，算做任务过期
* 需要查看当前任务的过期策略
* 2.下次调度时间+5s > 当前时间 > 下次调度时间，任务没有过期
* 可以进行调度
* 3.当前时间 <= 下次调度时间 任务将要调度
* 进行预调度，使用ringData、ringThread（时间轮线程）进行排序预备调度
*/
if (scheduleList!=null && scheduleList.size()>0) {
// 2、push time-ring
for (XxlJobInfo jobInfo: scheduleList) {
// time-ring jump
// 当前时间 > 下次调度时间+5s 即超过有效误差，算做任务过期
if (nowTime > jobInfo.getTriggerNextTime() + PRE_READ_MS) {
// 2.1、trigger-expire > 5s：pass && make next-trigger-time
logger.warn(">>>>>>>>>>> xxl-job, schedule misfire, jobId = " + jobInfo.getId());
// 1、misfire match
// 查看当前任务的过期策略
MisfireStrategyEnum misfireStrategyEnum = MisfireStrategyEnum.match(jobInfo.getMisfireStrategy(), MisfireStrategyEnum.DO_NOTHING);
// 如果是 【立即执行一次】 那么就执行一次
if (MisfireStrategyEnum.FIRE_ONCE_NOW == misfireStrategyEnum) {
// FIRE_ONCE_NOW 》 trigger
JobTriggerPoolHelper.trigger(jobInfo.getId(), TriggerTypeEnum.MISFIRE, -1, null, null, null);
logger.debug(">>>>>>>>>>> xxl-job, schedule push trigger : jobId = " + jobInfo.getId() );
}
// 2、fresh next
// 更新下次执行时间
refreshNextValidTime(jobInfo, new Date());
// 下次调度时间+5s > nowTime > 下次调度时间，任务没有过期
} else if (nowTime > jobInfo.getTriggerNextTime()) {
// 2.2、trigger-expire < 5s：direct-trigger && make next-trigger-time
// 1、trigger
// 任务调度
JobTriggerPoolHelper.trigger(jobInfo.getId(), TriggerTypeEnum.CRON, -1, null, null, null);
logger.debug(">>>>>>>>>>> xxl-job, schedule push trigger : jobId = " + jobInfo.getId() );
// 2、fresh next
// 更新下次执行时间
refreshNextValidTime(jobInfo, new Date());
// next-trigger-time in 5s, pre-read again
// 1运行，0停止； 调度运行 && 当前时间+5s > 下次调度时间
if (jobInfo.getTriggerStatus()==1 && nowTime + PRE_READ_MS > jobInfo.getTriggerNextTime()) {
// 1、make ring second
int ringSecond = (int)((jobInfo.getTriggerNextTime()/1000)%60);
// 2、push time ring
pushTimeRing(ringSecond, jobInfo.getId());
// 3、fresh next
refreshNextValidTime(jobInfo, new Date(jobInfo.getTriggerNextTime()));
}
} else {
// 下次执行时间不足5S的任务方入时间轮
// 2.3、trigger-pre-read：time-ring trigger && make next-trigger-time
// 1、make ring second
int ringSecond = (int)((jobInfo.getTriggerNextTime()/1000)%60);
// 2、push time ring
pushTimeRing(ringSecond, jobInfo.getId());
// 3、fresh next
refreshNextValidTime(jobInfo, new Date(jobInfo.getTriggerNextTime()));
}
}
// 3、update trigger info
for (XxlJobInfo jobInfo: scheduleList) {
XxlJobAdminConfig.getAdminConfig().getXxlJobInfoDao().scheduleUpdate(jobInfo);
}
} else {
preReadSuc = false;
}
// tx stop
} catch (Exception e) {
if (!scheduleThreadToStop) {
logger.error(">>>>>>>>>>> xxl-job, JobScheduleHelper#scheduleThread error:{}", e);
}
} finally {
// commit
if (conn != null) {
try {
conn.commit();
} catch (SQLException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
try {
conn.setAutoCommit(connAutoCommit);
} catch (SQLException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
try {
conn.close();
} catch (SQLException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
}
// close PreparedStatement
if (null != preparedStatement) {
try {
preparedStatement.close();
} catch (SQLException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
}
}
long cost = System.currentTimeMillis()-start;
// Wait seconds, align second
if (cost < 1000) { // scan-overtime, not wait
try {
// pre-read period: success > scan each second; fail > skip this period;
TimeUnit.MILLISECONDS.sleep((preReadSuc?1000:PRE_READ_MS) - System.currentTimeMillis()%1000);
} catch (InterruptedException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
}
}
logger.info(">>>>>>>>>>> xxl-job, JobScheduleHelper#scheduleThread stop");
}
});
scheduleThread.setDaemon(true);
scheduleThread.setName("xxl-job, admin JobScheduleHelper#scheduleThread");
scheduleThread.start();
// ring thread
ringThread = new Thread(new Runnable() {
@Override
public void run() {
while (!ringThreadToStop) {
// align second
try {
// 也是为了对齐时间整秒，范围是1S内
TimeUnit.MILLISECONDS.sleep(1000 - System.currentTimeMillis() % 1000);
} catch (InterruptedException e) {
if (!ringThreadToStop) {
logger.error(e.getMessage(), e);
}
}
try {
// second data
// 根据当前时间获取ringData的数据
List<Integer> ringItemData = new ArrayList<>();
int nowSecond = Calendar.getInstance().get(Calendar.SECOND); // 避免处理耗时太长，跨过刻度，向前校验一个刻度；
// 时间轮是秒为节点的，这里相当于是取当前秒和下一秒
for (int i = 0; i < 2; i++) {
List<Integer> tmpData = ringData.remove( (nowSecond+60-i)%60 );
if (tmpData != null) {
ringItemData.addAll(tmpData);
}
}
// ring trigger
logger.debug(">>>>>>>>>>> xxl-job, time-ring beat : " + nowSecond + " = " + Arrays.asList(ringItemData) );
// 触发任务
if (ringItemData.size() > 0) {
// do trigger
for (int jobId: ringItemData) {
// do trigger
JobTriggerPoolHelper.trigger(jobId, TriggerTypeEnum.CRON, -1, null, null, null);
}
// clear
ringItemData.clear();
}
} catch (Exception e) {
if (!ringThreadToStop) {
logger.error(">>>>>>>>>>> xxl-job, JobScheduleHelper#ringThread error:{}", e);
}
}
}
logger.info(">>>>>>>>>>> xxl-job, JobScheduleHelper#ringThread stop");
}
});
ringThread.setDaemon(true);
ringThread.setName("xxl-job, admin JobScheduleHelper#ringThread");
ringThread.start();
}4.2.8 XxlJobTrigger 触发器（任务进入快慢线程池之后的实际调用）
/**
* trigger job
*
* @param jobId
* @param triggerType
* 根据TriggerTypeEnum触发类型枚举类可以看到有下面六种可以选择
* 1）jobconf_trigger_type_cron：Cron触发
* 2）jobconf_trigger_type_manual：手动触发
* 3）jobconf_trigger_type_parent：父任务触发
* 4）jobconf_trigger_type_api：API触发
* 5）jobconf_trigger_type_retry：失败重试触发
* 6）jobconf_trigger_type_misfire：调度过期补偿
* @param failRetryCount 失败重试次数
* >=0: use this param
* <0: use param from job info config
* @param executorShardingParam 分片参数 分片序号/分片总数
* @param executorParam 执行器，任务参数
* null: use job param
* not null: cover job param
* @param addressList
* addressType 执行器地址类型：0=自动注册、1=手动录入
* addressList 执行器地址列表，多地址逗号分隔(手动录入)
* null: use executor addressList
* not null: cover
*/
public static void trigger(int jobId,
TriggerTypeEnum triggerType,
int failRetryCount,
String executorShardingParam,
String executorParam,
String addressList) {
// load data 数据来自xxl_job_info
XxlJobInfo jobInfo = XxlJobAdminConfig.getAdminConfig().getXxlJobInfoDao().loadById(jobId);
if (jobInfo == null) {
logger.warn(">>>>>>>>>>>> trigger fail, jobId invalid，jobId={}", jobId);
return;
}
if (executorParam != null) {
jobInfo.setExecutorParam(executorParam);
}
int finalFailRetryCount = failRetryCount>=0?failRetryCount:jobInfo.getExecutorFailRetryCount();
XxlJobGroup group = XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().load(jobInfo.getJobGroup());
// cover addressList 执行器地址列表，多地址逗号分隔(手动录入)
if (addressList!=null && addressList.trim().length()>0) {
group.setAddressType(1);
group.setAddressList(addressList.trim());
}
// sharding param 分片信息
int[] shardingParam = null;
if (executorShardingParam!=null){
String[] shardingArr = executorShardingParam.split("/");
if (shardingArr.length==2 && isNumeric(shardingArr[0]) && isNumeric(shardingArr[1])) {
shardingParam = new int[2];
shardingParam[0] = Integer.valueOf(shardingArr[0]);
shardingParam[1] = Integer.valueOf(shardingArr[1]);
}
}
// 路由策略判断
// 这里主要判断是否是分片广播，因为分片广播与其他类型差异大。
if (ExecutorRouteStrategyEnum.SHARDING_BROADCAST==ExecutorRouteStrategyEnum.match(jobInfo.getExecutorRouteStrategy(), null)
&& group.getRegistryList()!=null && !group.getRegistryList().isEmpty()
&& shardingParam==null) {
// 这里是均分数据给多个执行器，相互任务不重叠，增加效率
for (int i = 0; i < group.getRegistryList().size(); i++) {
// 调度部分
processTrigger(group, jobInfo, finalFailRetryCount, triggerType, i, group.getRegistryList().size());
}
} else {
if (shardingParam == null) {
shardingParam = new int[]{0, 1};
}
// 调度部分
processTrigger(group, jobInfo, finalFailRetryCount, triggerType, shardingParam[0], shardingParam[1]);
}
} /** 调度核心部分
* @param group job group, registry list may be empty
* @param jobInfo
* @param finalFailRetryCount
* @param triggerType
* @param index sharding index 分片广播的序号
* @param total sharding index 分片广播的总分片数
*/
private static void processTrigger(XxlJobGroup group, XxlJobInfo jobInfo, int finalFailRetryCount, TriggerTypeEnum triggerType, int index, int total){
// param 阻塞处理策略
ExecutorBlockStrategyEnum blockStrategy = ExecutorBlockStrategyEnum.match(jobInfo.getExecutorBlockStrategy(), ExecutorBlockStrategyEnum.SERIAL_EXECUTION); // block strategy
ExecutorRouteStrategyEnum executorRouteStrategyEnum = ExecutorRouteStrategyEnum.match(jobInfo.getExecutorRouteStrategy(), null); // route strategy
// 分片参数
String shardingParam = (ExecutorRouteStrategyEnum.SHARDING_BROADCAST==executorRouteStrategyEnum)?String.valueOf(index).concat("/").concat(String.valueOf(total)):null;
// 1、save log-id 日志
XxlJobLog jobLog = new XxlJobLog();
jobLog.setJobGroup(jobInfo.getJobGroup());
jobLog.setJobId(jobInfo.getId());
jobLog.setTriggerTime(new Date());
XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().save(jobLog);
logger.debug(">>>>>>>>>>> xxl-job trigger start, jobId:{}", jobLog.getId());
// 2、init trigger-param 触发器参数
TriggerParam triggerParam = new TriggerParam();
triggerParam.setJobId(jobInfo.getId());
triggerParam.setExecutorHandler(jobInfo.getExecutorHandler());
triggerParam.setExecutorParams(jobInfo.getExecutorParam());
triggerParam.setExecutorBlockStrategy(jobInfo.getExecutorBlockStrategy());
triggerParam.setExecutorTimeout(jobInfo.getExecutorTimeout());
triggerParam.setLogId(jobLog.getId());
triggerParam.setLogDateTime(jobLog.getTriggerTime().getTime());
triggerParam.setGlueType(jobInfo.getGlueType());
triggerParam.setGlueSource(jobInfo.getGlueSource());
triggerParam.setGlueUpdatetime(jobInfo.getGlueUpdatetime().getTime());
triggerParam.setBroadcastIndex(index);
triggerParam.setBroadcastTotal(total);
// 3、init address 执行器地址信息
String address = null;
ReturnT<String> routeAddressResult = null;
// 执行器地址列表
if (group.getRegistryList()!=null && !group.getRegistryList().isEmpty()) {
if (ExecutorRouteStrategyEnum.SHARDING_BROADCAST == executorRouteStrategyEnum) {
if (index < group.getRegistryList().size()) {
address = group.getRegistryList().get(index);
} else {
address = group.getRegistryList().get(0);
}
} else {
routeAddressResult = executorRouteStrategyEnum.getRouter().route(triggerParam, group.getRegistryList());
if (routeAddressResult.getCode() == ReturnT.SUCCESS_CODE) {
address = routeAddressResult.getContent();
}
}
} else {
routeAddressResult = new ReturnT<String>(ReturnT.FAIL_CODE, I18nUtil.getString("jobconf_trigger_address_empty"));
}
// 4、trigger remote executor 远程触发调度执行器
ReturnT<String> triggerResult = null;
if (address != null) {
// 实际上是发了一个POST请求给执行器
triggerResult = runExecutor(triggerParam, address);
} else {
triggerResult = new ReturnT<String>(ReturnT.FAIL_CODE, null);
}
// 5、collection trigger info 执行信息汇总
StringBuffer triggerMsgSb = new StringBuffer();
triggerMsgSb.append(I18nUtil.getString("jobconf_trigger_type")).append("：").append(triggerType.getTitle());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_admin_adress")).append("：").append(IpUtil.getIp());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_exe_regtype")).append("：")
.append( (group.getAddressType() == 0)?I18nUtil.getString("jobgroup_field_addressType_0"):I18nUtil.getString("jobgroup_field_addressType_1") );
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_exe_regaddress")).append("：").append(group.getRegistryList());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorRouteStrategy")).append("：").append(executorRouteStrategyEnum.getTitle());
if (shardingParam != null) {
triggerMsgSb.append("("+shardingParam+")");
}
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorBlockStrategy")).append("：").append(blockStrategy.getTitle());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_timeout")).append("：").append(jobInfo.getExecutorTimeout());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorFailRetryCount")).append("：").append(finalFailRetryCount);
triggerMsgSb.append("<br><br><span style=\"color:#00c0ef;\" > >>>>>>>>>>>"+ I18nUtil.getString("jobconf_trigger_run") +"<<<<<<<<<<< </span><br>")
.append((routeAddressResult!=null&&routeAddressResult.getMsg()!=null)?routeAddressResult.getMsg()+"<br><br>":"").append(triggerResult.getMsg()!=null?triggerResult.getMsg():"");
// 6、save log trigger-info 日志更新
jobLog.setExecutorAddress(address);
jobLog.setExecutorHandler(jobInfo.getExecutorHandler());
jobLog.setExecutorParam(jobInfo.getExecutorParam());
jobLog.setExecutorShardingParam(shardingParam);
jobLog.setExecutorFailRetryCount(finalFailRetryCount);
//jobLog.setTriggerTime();
jobLog.setTriggerCode(triggerResult.getCode());
jobLog.setTriggerMsg(triggerMsgSb.toString());
XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().updateTriggerInfo(jobLog);
logger.debug(">>>>>>>>>>> xxl-job trigger end, jobId:{}", jobLog.getId());
}补充：公司所用项目虽然是集群模式，但是避免定时任务重复调用效果是通过数据库完成的。有表专门存储需要执行的定时任务和是否执行状态，调度器调度前会先查询待执行任务的是否状态，若为未执行，则进行修改操作修改成功时update xx set state = 1 and prod = '调度器id'，修改成功才会执行定时任务。修改失败则再次查询数据库是否是已被其他调度器调度，然后再判断其他报错逻辑之类的。
4764
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
