一、主题
XXL-JOB是一个轻量级分布式任务调度平台，笔者带入新手的视角，从项目概览和一个示例程序开始，浅析项目的启动流程，然后以业务角度解析几个关键请求的执行过程，最后站在纯技术角度扩展一部分技术内容，带大伙浅浅分析一波浅析XXL-JOB框架。
二、项目概览
（一）源码地址
https://gitee.com/xuxueli0323/xxl-job.git
（二）项目概览
xxl-job-admin：调度中心
xxl-job-core：公共依赖
xxl-job-executor-samples：执行器Sample示例（选择合适的版本执行器，可直接使用，也可以参考其并将现有项目改造成执行器）
：xxl-job-executor-sample-springboot：Springboot版本，通过Springboot管理执行器，推荐这种方式；
：xxl-job-executor-sample-frameless：无框架版本；
（三）具体依赖
1.xxl-job-core的pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.xuxueli</groupId>
<artifactId>xxl-job</artifactId>
<version>2.4.1-SNAPSHOT</version>
</parent>
<artifactId>xxl-job-core</artifactId>
<packaging>jar</packaging>
<name>${project.artifactId}</name>
<description>A distributed task scheduling framework.</description>
<url>https://www.xuxueli.com/</url>
<dependencies>
<!-- embed server: netty + gson -->
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-codec-http</artifactId>
<version>${netty.version}</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>${gson.version}</version>
</dependency>
<!-- plugin -->
<!-- groovy-all -->
<dependency>
<groupId>org.apache.groovy</groupId>
<artifactId>groovy</artifactId>
<version>${groovy.version}</version>
</dependency>
<!-- spring-context -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>${spring.version}</version>
<scope>provided</scope>
</dependency>
<!-- base -->
<!-- slf4j -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j-api.version}</version>
</dependency>
<!-- javax.annotation-api -->
<dependency>
<groupId>javax.annotation</groupId>
<artifactId>javax.annotation-api</artifactId>
<version>${javax.annotation-api.version}</version>
<scope>provided</scope>
</dependency>
</dependencies>
</project>
2.xxl-job-admin的pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.xuxueli</groupId>
<artifactId>xxl-job</artifactId>
<version>2.4.1-SNAPSHOT</version>
</parent>
<artifactId>xxl-job-admin</artifactId>
<packaging>jar</packaging>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>${spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<!-- starter-web：spring-webmvc + autoconfigure + logback + yaml + tomcat -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- starter-test：junit + spring-test + mockito -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!-- freemarker-starter -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-freemarker</artifactId>
</dependency>
<!-- mail-starter -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-mail</artifactId>
</dependency>
<!-- starter-actuator -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- mybatis-starter：mybatis + mybatis-spring + hikari（default） -->
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>${mybatis-spring-boot-starter.version}</version>
</dependency>
<!-- mysql -->
<dependency>
<groupId>com.mysql</groupId>
<artifactId>mysql-connector-j</artifactId>
<version>${mysql-connector-j.version}</version>
</dependency>
<!-- xxl-job-core -->
<dependency>
<groupId>com.xuxueli</groupId>
<artifactId>xxl-job-core</artifactId>
<version>${project.parent.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>${spring-boot.version}</version>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- docker -->
<plugin>
<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.4.13</version>
<configuration>
<!-- made of '[a-z0-9-_.]' -->
<imageName>${project.artifactId}:${project.version}</imageName>
<dockerDirectory>${project.basedir}</dockerDirectory>
<resources>
<resource>
<targetPath>/</targetPath>
<directory>${project.build.directory}</directory>
<include>${project.build.finalName}.jar</include>
</resource>
</resources>
</configuration>
</plugin>
</plugins>
</build>
</project>
3.xxl-job-executor-sample-frameless的pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.xuxueli</groupId>
<artifactId>xxl-job-executor-samples</artifactId>
<version>2.4.1-SNAPSHOT</version>
</parent>
<artifactId>xxl-job-executor-sample-frameless</artifactId>
<packaging>jar</packaging>
<name>${project.artifactId}</name>
<description>Example executor project for spring boot.</description>
<url>https://www.xuxueli.com/</url>
<dependencies>
<!-- slf4j -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4j-api.version}</version>
</dependency>
<!-- junit -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>${junit-jupiter.version}</version>
<scope>test</scope>
</dependency>
<!-- xxl-job-core -->
<dependency>
<groupId>com.xuxueli</groupId>
<artifactId>xxl-job-core</artifactId>
<version>${project.parent.version}</version>
</dependency>
</dependencies>
</project>
4.xxl-job-executor-sample-springboot的pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.xuxueli</groupId>
<artifactId>xxl-job-executor-samples</artifactId>
<version>2.4.1-SNAPSHOT</version>
</parent>
<artifactId>xxl-job-executor-sample-springboot</artifactId>
<packaging>jar</packaging>
<name>${project.artifactId}</name>
<description>Example executor project for spring boot.</description>
<url>https://www.xuxueli.com/</url>
<properties>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<!-- Import dependency management from Spring Boot (依赖管理：继承一些默认的依赖，工程需要依赖的jar包的管理，申明其他dependency的时候就不需要version) -->
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>${spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<!-- spring-boot-starter-web (spring-webmvc + tomcat) -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!-- xxl-job-core -->
<dependency>
<groupId>com.xuxueli</groupId>
<artifactId>xxl-job-core</artifactId>
<version>${project.parent.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<!-- spring-boot-maven-plugin (提供了直接运行项目的插件：如果是通过parent方式继承spring-boot-starter-parent则不用此插件) -->
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>${spring-boot.version}</version>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
（四）依赖概览
1、依赖xxl-job-core依赖netty
2、xxl-job-admin依赖xxl-job-core、mysql-connector-j
3、xxl-job-executor-sample-frameless依赖xxl-job-core
4、xxl-job-executor-sample-springboot依赖xxl-job-core
5、xxl-job-executor-sample-frameless、xxl-job-executor-sample-springboot与xxl-job-admin无依赖关系
三、示例程序
（一）环境准备
1、jdk-17.0.7
2、mysql-5.7.41-winx64
3、IntelliJ IDEA 2023.1 (Ultimate Edition)
4、apache-maven-3.6.2
（二）示例大纲
1、示例程序包含一个调度中心和两个执行器，其中执行器由xxl-job-executor-sample-frameless和xxl-job-executor-sample-springboot组成，项目做了少许改造，本意是借助Quick Start强化调度中心、执行器并非强耦合关系，为服务端与客户端关系，这也是上面所列pom依赖关系的一种衍生应用。
2、Quick Start官方参考地址：https://www.xuxueli.com/xxl-job/#1.6 环境
（三）操作步骤
1.启动mysql服务
2.导入数据库脚本
3.调度中心项目改造
（1）卸载xxl-job-executor-samples模块
（2）移除xxl-job-executor-samples项目依赖
4.修改调度中心配置
（1）修改数据库配置
5.启动调度中心服务
6.访问调度中心页面
访问地址：http://localhost:8080/xxl-job-admin
账户/密码：admin/123456
7.执行器（frameless）项目改造
（1）卸载xxl-job-admin、xxl-job-sample-springboot模块
（2）移除xxl-job-admin、xxl-job-sample-springboot依赖
（3）新增demoFramelessJobHandler执行器
8.修改执行器（frameless）配置
（1）修改调度中心地址配置、修改服务端口、修改执行器AppName
9.启动执行器（frameless）服务
10.执行器（springboot）项目改造
（1）卸载xxl-job-admin、xxl-job-sample-frameless模块
（2）移除xxl-job-admin、xxl-job-sample-frameless依赖
（3）新增demoSpringbootJobHandler执行器
11.修改执行器（springboot）配置
（1）修改调度中心地址配置、修改服务端口、修改执行器AppName
12.启动执行器（springboot）服务
13.调度中心页面新增执行器
1、访问调度中心页面 http://localhost:8080/xxl-job-admin -> 执行器管理 -> 新增 -> 保存
14.调度中心页面新增任务
15.调度中心页面启动任务
1、访问调度中心页面 http://localhost:8080/xxl-job-admin -> 任务管理 -> 选择任务 -> 操作 -> 启动 -> 确定
（四）调度中心页面（数据库角度）
1.xxl_job_group表
1、作用：存储执行器信息
2、相关接口：http://localhost:8080/xxl-job-admin/jobgroup/pageList
2.xxl_job_info表
1、作用：存储任务配置信息
2、相关接口：http://localhost:8080/xxl-job-admin/jobinfo/pageList
3.xxl_job_log表
1、作用：存储任务执行日志信息
2、相关接口：http://localhost:8080/xxl-job-admin/joblog/pageList
4.xxl_job_logglue表
1、作用：存储glue脚本语言信息（除bean外的6种运行模式使用）
2、相关接口：http://127.0.0.1:8080/xxl-job-admin/jobcode/save
5.xxl_job_log_report表
1、作用：存储每日任务执行情况统计信息
2、相关接口：http://localhost:8080/xxl-job-admin/chartInfo
6.xxl_job_registry表
1、作用：存储执行器对应执行节点注册信息
2、相关接口：http://localhost:8080/api/registry
7.xxl_job_user表
1、作用：存储账户信息，对应界面管理信息如下截图
2、相关接口：http://localhost:8080/xxl-job-admin/user/pageList
（五）数据库ER图
1、ER图
待补充
2、文字说明
① xxl_job_group：执行器信息表
② xxl_job_info：任务配置信息表，关联关系
（n）xxl_job_info.job_group（1）xxl_job_group.id
③ xxl_job_log：任务执行信息表，关联关系
（n）xxl_job_log.job_id （1）xxl_job_info.id
④ xxl_job_logglue：存储glue脚本语言信息（除bean外的6种运行模式使用）
⑤ xxl_job_log_report：每日任务执行情况（xxl_job_log）统计信息表
⑥ xxl_job_registry：执行节点信息表，关联关系
（n）xxl_job_registry.registry_key （1）xxl_job_group.app_name
⑦ xxl_job_user：用户表
⑧ xxl_job_lock：表锁，集群模式下xxl-job进行任务调度时，为了避免任务被重复调度，会先通过SQL对表xxl_job_lock的记录schedule_lock加锁
四、代码分析
（一）调度中心启动流程
1.源码详解
1）step 1
1、调度中心为springboot项目，启动时会注入com.xxl.job.admin.core.conf.XxlJobAdminConfig类，并执行回调方法com.xxl.job.admin.core.conf.XxlJobAdminConfig#afterPropertiesSet；
2、com.xxl.job.admin.core.conf.XxlJobAdminConfig类注入了系统运行所需的各种类、变量，并对外暴露相应方法方法
3、com.xxl.job.admin.core.conf.XxlJobAdminConfig#afterPropertiesSet则为咱们调度中心启动代码的入口
package com.xxl.job.admin.core.conf;
import *;
@Component
public class XxlJobAdminConfig implements InitializingBean, DisposableBean {
private static XxlJobAdminConfig adminConfig = null;
public static XxlJobAdminConfig getAdminConfig() {
return adminConfig;
}
private XxlJobScheduler xxlJobScheduler;
@Override
public void afterPropertiesSet() throws Exception {
// 系统启动入口
adminConfig = this;
xxlJobScheduler = new XxlJobScheduler();
xxlJobScheduler.init();
}
@Override
public void destroy() throws Exception {
xxlJobScheduler.destroy();
}
@Value("${xxl.job.i18n}")
private String i18n;
@Value("${xxl.job.accessToken}")
private String accessToken;
@Value("${spring.mail.from}")
private String emailFrom;
@Value("${xxl.job.triggerpool.fast.max}")
private int triggerPoolFastMax;
@Value("${xxl.job.triggerpool.slow.max}")
private int triggerPoolSlowMax;
@Value("${xxl.job.logretentiondays}")
private int logretentiondays;
@Resource
private XxlJobLogDao xxlJobLogDao;
@Resource
private XxlJobInfoDao xxlJobInfoDao;
@Resource
private XxlJobRegistryDao xxlJobRegistryDao;
@Resource
private XxlJobGroupDao xxlJobGroupDao;
@Resource
private XxlJobLogReportDao xxlJobLogReportDao;
@Resource
private JavaMailSender mailSender;
@Resource
private DataSource dataSource;
@Resource
private JobAlarmer jobAlarmer;
public String getI18n() {
if (!Arrays.asList("zh_CN", "zh_TC", "en").contains(i18n)) {
return "zh_CN";
}
return i18n;
}
public String getAccessToken() {
return accessToken;
}
public String getEmailFrom() {
return emailFrom;
}
/**
* 快任务-调度线程池最大线程配置【必填】
*/
public int getTriggerPoolFastMax() {
if (triggerPoolFastMax < 200) {
return 200;
}
return triggerPoolFastMax;
}
/**
* 慢任务-调度线程池最大线程配置【必填】
*/
public int getTriggerPoolSlowMax() {
if (triggerPoolSlowMax < 100) {
return 100;
}
return triggerPoolSlowMax;
}
/**
* 日志最大保留时间（默认xxl.job.logretentiondays=30），必须大于7天，否则关闭（-1）
*/
public int getLogretentiondays() {
if (logretentiondays < 7) {
return -1;
}
return logretentiondays;
}
public XxlJobLogDao getXxlJobLogDao() {
return xxlJobLogDao;
}
public XxlJobInfoDao getXxlJobInfoDao() {
return xxlJobInfoDao;
}
public XxlJobRegistryDao getXxlJobRegistryDao() {
return xxlJobRegistryDao;
}
public XxlJobGroupDao getXxlJobGroupDao() {
return xxlJobGroupDao;
}
public XxlJobLogReportDao getXxlJobLogReportDao() {
return xxlJobLogReportDao;
}
public JavaMailSender getMailSender() {
return mailSender;
}
public DataSource getDataSource() {
return dataSource;
}
public JobAlarmer getJobAlarmer() {
return jobAlarmer;
}
}
2）step 2
1、com.xxl.job.admin.core.scheduler.XxlJobScheduler#init具体实现如下（接下来对init每行代码进行逐个分析）
package com.xxl.job.admin.core.scheduler;
import *;
public class XxlJobScheduler {
private static final Logger logger = LoggerFactory.getLogger(XxlJobScheduler.class);
public void init() throws Exception {
// 1.1、加载国际化配置, 默认加载/resources/i18n/message_zh_CN.properties，赋值给变量com.xxl.job.admin.core.util.I18nUtil.prop，
// 1.2、并初始化\替换系统枚举、常量
initI18n();
// 2.1、初始化快、慢任务 两个 任务调度线程池，并赋值给com.xxl.job.admin.core.thread.JobTriggerPoolHelper.fastTriggerPool，
// com.xxl.job.admin.core.thread.JobTriggerPoolHelper.slowTriggerPool俩变量
// 注：任务执行时，调用com.xxl.job.admin.core.thread.JobTriggerPoolHelper.addTrigger方法，内部会根据条件自动判断到底使用那个线程池
JobTriggerPoolHelper.toStart();
// 3.1、初始化 任务执行节点注册 线程池，并赋值给com.xxl.job.admin.core.thread.JobRegistryHelper.registryOrRemoveThreadPool，
// 注：手动注册执行器时，调用com.xxl.job.admin.core.thread.JobRegistryHelper.registry方法，用于设置执行器对应的任务执行节信息
// 3.2、启动 任务执行节点注册 后台监控线程，赋值给com.xxl.job.admin.core.thread.JobRegistryHelper.registryMonitorThread变量，
// 每30s执行一次，主要负责 刷新所有 执行器地址类型 为 自动注册类型 xxl_job_group表 执行器信息（同时移除失效/超时任务执行节点信息）
JobRegistryHelper.getInstance().start();
// 4.1、启动 失败任务 后台监控线程，并赋值给com.xxl.job.admin.core.thread.JobFailMonitorHelper.monitorThread变量，
// 每10s执行一次，主要负责失败任务的重试、已经失败任务的报警工作
JobFailMonitorHelper.getInstance().start();
// 5.1、始化 任务回调方法 调度 线程池，并赋值给com.xxl.job.admin.core.thread.JobCompleteHelper.callbackThreadPool，
// 注：任务执行结束时，调用com.xxl.job.admin.core.thread.JobCompleteHelper.callback方法，执行默认回调方法（即主动结束当前方法执行，如果有子任务则触发子任务执行，并更新xxl_job_log表记录）
// 5.2、启动 执行结果丢失任务 后台监控线程，赋值给com.xxl.job.admin.core.thread.JobCompleteHelper.monitorThread
// 每60s执行一次，主要负责 将 执行结果丢失任务 主动标记为失败 （调度记录停留在 "运行中" 状态超过10min，且对应执行器心跳注册失败不在线，则将本地调度主动标记失败）
JobCompleteHelper.getInstance().start();// admin lose-monitor run ( depend on JobTriggerPoolHelper )
// 6.1、启动 定时日志清理 后台监控线程，赋值给com.xxl.job.admin.core.thread.JobLogReportHelper.logrThread
// 每1天执行一次，主要负责 清理 已过期的xxl_job_log表记录（xxl.job.logretentiondays）
JobLogReportHelper.getInstance().start();
// https://www.dandelioncloud.cn/article/details/1611003603865272322
// 启动 俩任务调度 后台监控线程，赋值给com.xxl.job.admin.core.thread.JobScheduleHelper.scheduleThread，com.xxl.job.admin.core.thread.JobScheduleHelper.ringThread
// 7.1、线程 scheduleThread 运行中不断的从任务表中查询 查询近 5秒 中要执行的任务，
// 分情况将判断任务是否立即执行，或者将任务执行时间除以 1000 变为秒之后再与 60 求余添加到时间轮中（时间轮实现方式比较简单，就是一个 Map 结构数据，key值0-60，value是任务ID列表 Map ringData）
// 7.2、线程 ringThread 运行中不断根据当前时间求余从 时间轮 ringData 中获取任务列表，取出任务之后执行任务
JobScheduleHelper.getInstance().start();// start-schedule ( depend on JobTriggerPoolHelper )
}
public void destroy() throws Exception {
JobScheduleHelper.getInstance().toStop();
JobLogReportHelper.getInstance().toStop();
JobCompleteHelper.getInstance().toStop();
JobFailMonitorHelper.getInstance().toStop();
JobRegistryHelper.getInstance().toStop();
JobTriggerPoolHelper.toStop();
}
private void initI18n(){
// 替换 阻塞处理策略 对应的title值
for (ExecutorBlockStrategyEnum item:ExecutorBlockStrategyEnum.values()) {
// I18nUtil.getString方法，默认加载/resources/i18n/message_zh_CN.properties，赋值给变量com.xxl.job.admin.core.util.I18nUtil.prop
// 然后按照给定的key返回对应的值
item.setTitle(I18nUtil.getString("jobconf_block_".concat(item.name())));
}
}
private static ConcurrentMap<String, ExecutorBiz> executorBizRepository = new ConcurrentHashMap<String, ExecutorBiz>();
/**
* 获取address对应ExecutorBiz实例（ExecutorBizClient）
*/
public static ExecutorBiz getExecutorBiz(String address) throws Exception {
// 验证address非空
if (address==null || address.trim().length()==0) {
return null;
}
// 从executorBizRepository中加载缓存信息
address = address.trim();
ExecutorBiz executorBiz = executorBizRepository.get(address);
if (executorBiz != null) {
return executorBiz;
}
// 设置ExecutorBiz实例，并存入executorBizRepository缓存
executorBiz = new ExecutorBizClient(address, XxlJobAdminConfig.getAdminConfig().getAccessToken());
executorBizRepository.put(address, executorBiz);
return executorBiz;
}
}
3）step 3
1、com.xxl.job.admin.core.scheduler.XxlJobScheduler#init方法内部调用com.xxl.job.admin.core.scheduler.XxlJobScheduler#initI18n方法，代码实现见step 2
2、com.xxl.job.admin.core.scheduler.XxlJobScheduler#initI18n方法，内部调用了com.xxl.job.core.enums.ExecutorBlockStrategyEnum#values方法，具体实现如下
package com.xxl.job.core.enums;
/**
* 阻塞处理策略 枚举
* Created by xuxueli on 17/5/9.
*/
public enum ExecutorBlockStrategyEnum {
//单机串行
SERIAL_EXECUTION("Serial execution"),
//丢弃后续调度
DISCARD_LATER("Discard Later"),
//覆盖之前调度
COVER_EARLY("Cover Early");
private String title;
private ExecutorBlockStrategyEnum (String title) {
this.title = title;
}
public void setTitle(String title) {
this.title = title;
}
public String getTitle() {
return title;
}
public static ExecutorBlockStrategyEnum match(String name, ExecutorBlockStrategyEnum defaultItem) {
if (name != null) {
for (ExecutorBlockStrategyEnum item:ExecutorBlockStrategyEnum.values()) {
if (item.name().equals(name)) {
return item;
}
}
}
return defaultItem;
}
}
3、com.xxl.job.admin.core.scheduler.XxlJobScheduler#initI18n方法，内部调用了com.xxl.job.admin.core.util.I18nUtil#getString方法，具体实现如下
package com.xxl.job.admin.core.util;
import *;
public class I18nUtil {
private static Logger logger = LoggerFactory.getLogger(I18nUtil.class);
private static Properties prop = null;
public static Properties loadI18nProp(){
if (prop != null) {
return prop;
}
try {
// 构建i18n配置文件
String i18n = XxlJobAdminConfig.getAdminConfig().getI18n();
String i18nFile = MessageFormat.format("i18n/message_{0}.properties", i18n);
// 加载i18n对应prop
Resource resource = new ClassPathResource(i18nFile);
EncodedResource encodedResource = new EncodedResource(resource,"UTF-8");
prop = PropertiesLoaderUtils.loadProperties(encodedResource);
} catch (IOException e) {
logger.error(e.getMessage(), e);
}
return prop;
}
public static String getString(String key) {
// 默认加载/resources/i18n/message_zh_CN.properties，赋值给变量com.xxl.job.admin.core.util.I18nUtil.prop
return loadI18nProp().getProperty(key);
}
public static String getMultString(String... keys) {
Map<String, String> map = new HashMap<String, String>();
Properties prop = loadI18nProp();
if (keys!=null && keys.length>0) {
for (String key: keys) {
map.put(key, prop.getProperty(key));
}
} else {
for (String key: prop.stringPropertyNames()) {
map.put(key, prop.getProperty(key));
}
}
String json = JacksonUtil.writeValueAsString(map);
return json;
}
}
4）step 4
1、com.xxl.job.admin.core.scheduler.XxlJobScheduler#init方法内部调用方法com.xxl.job.admin.core.thread.JobTriggerPoolHelper#toStart，具体实现如下
package com.xxl.job.admin.core.thread;
import *;
public class JobTriggerPoolHelper {
private static Logger logger = LoggerFactory.getLogger(JobTriggerPoolHelper.class);
private ThreadPoolExecutor fastTriggerPool = null;
private ThreadPoolExecutor slowTriggerPool = null;
public void start(){
fastTriggerPool = new ThreadPoolExecutor(
10,
XxlJobAdminConfig.getAdminConfig().getTriggerPoolFastMax(),
60L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<Runnable>(1000),
new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "xxl-job, admin JobTriggerPoolHelper-fastTriggerPool-" + r.hashCode());
}
});
slowTriggerPool = new ThreadPoolExecutor(
10,
XxlJobAdminConfig.getAdminConfig().getTriggerPoolSlowMax(),
60L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<Runnable>(2000),
new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "xxl-job, admin JobTriggerPoolHelper-slowTriggerPool-" + r.hashCode());
}
});
}
public void stop() {
//triggerPool.shutdown();
fastTriggerPool.shutdownNow();
slowTriggerPool.shutdownNow();
}
private volatile long minTim = System.currentTimeMillis()/60000; // ms > min
private volatile ConcurrentMap<Integer, AtomicInteger> jobTimeoutCountMap = new ConcurrentHashMap<>();
/**
* 添加 执行任务
*/
public void addTrigger(final int jobId,
final TriggerTypeEnum triggerType,
final int failRetryCount,
final String executorShardingParam,
final String executorParam,
final String addressList) {
// 内部会根据条件自动判断到底使用那个线程池 (一分钟超时10次，防止频繁请求)
ThreadPoolExecutor triggerPool_ = fastTriggerPool;
AtomicInteger jobTimeoutCount = jobTimeoutCountMap.get(jobId);
if (jobTimeoutCount!=null && jobTimeoutCount.get() > 10) { // job-timeout 10 times in 1 min
triggerPool_ = slowTriggerPool;
}
// 触发任务执行
triggerPool_.execute(new Runnable() {
@Override
public void run() {
long start = System.currentTimeMillis();
try {
XxlJobTrigger.trigger(jobId, triggerType, failRetryCount, executorShardingParam, executorParam, addressList);
} catch (Exception e) {
logger.error(e.getMessage(), e);
} finally {
// 刷新任务执行时间（单位：分钟）
long minTim_now = System.currentTimeMillis()/60000;
if (minTim != minTim_now) { // 执行结束时间 跨 分钟
minTim = minTim_now;// 刷新上次执行时间（单位：分钟）
jobTimeoutCountMap.clear();
}
// 消耗时间 大于 500ms, 任务执行超时次数+1
long cost = System.currentTimeMillis()-start;
if (cost > 500) {
AtomicInteger timeoutCount = jobTimeoutCountMap.putIfAbsent(jobId, new AtomicInteger(1));//不存在则赋后面的值
if (timeoutCount != null) {
timeoutCount.incrementAndGet();
}
}
}
}
});
}
private static JobTriggerPoolHelper helper = new JobTriggerPoolHelper();
public static void toStart() {
helper.start();
}
public static void toStop() {
helper.stop();
}
public static void trigger(int jobId, TriggerTypeEnum triggerType, int failRetryCount, String executorShardingParam, String executorParam, String addressList) {
helper.addTrigger(jobId, triggerType, failRetryCount, executorShardingParam, executorParam, addressList);
}
}
2、com.xxl.job.admin.core.thread.JobTriggerPoolHelper#toStart方法内部调用方法com.xxl.job.admin.core.trigger.XxlJobTrigger#trigger，具体实现如下
package com.xxl.job.admin.core.trigger;
import *;
public class XxlJobTrigger {
private static Logger logger = LoggerFactory.getLogger(XxlJobTrigger.class);
/**
* 触发任务执行
*/
public static void trigger(int jobId,
TriggerTypeEnum triggerType,
int failRetryCount,
String executorShardingParam,
String executorParam,
String addressList) {
// 组装任务执行参数，触发任务执行;通过jobId，加载数据
XxlJobInfo jobInfo = XxlJobAdminConfig.getAdminConfig().getXxlJobInfoDao().loadById(jobId);
if (jobInfo == null) {
return;
}
if (executorParam != null) {
jobInfo.setExecutorParam(executorParam);
}
int finalFailRetryCount = failRetryCount>=0?failRetryCount:jobInfo.getExecutorFailRetryCount();
XxlJobGroup group = XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().load(jobInfo.getJobGroup());
// 覆盖待执行地址（优先外部传入执行器地址）
if (addressList!=null && addressList.trim().length()>0) {
group.setAddressType(1);//执行器地址类型：0=自动注册、1=手动录入
group.setAddressList(addressList.trim());
}
// 组装分片参数 并赋值给变量shardingParam
int[] shardingParam = null;
if (executorShardingParam!=null){
String[] shardingArr = executorShardingParam.split("/");
if (shardingArr.length==2 && isNumeric(shardingArr[0]) && isNumeric(shardingArr[1])) {
shardingParam = new int[2];
shardingParam[0] = Integer.valueOf(shardingArr[0]);
shardingParam[1] = Integer.valueOf(shardingArr[1]);
}
}
// 路由策略 是否为 分片广播
if (ExecutorRouteStrategyEnum.SHARDING_BROADCAST==ExecutorRouteStrategyEnum.match(jobInfo.getExecutorRouteStrategy(), null)
&& group.getRegistryList()!=null && !group.getRegistryList().isEmpty()
&& shardingParam==null) {
for (int i = 0; i < group.getRegistryList().size(); i++) {
processTrigger(group, jobInfo, finalFailRetryCount, triggerType, i, group.getRegistryList().size());
}
} else {
// 路由策略 为 其他9种
if (shardingParam == null) {
shardingParam = new int[]{0, 1};
}
// 执行任务
processTrigger(group, jobInfo, finalFailRetryCount, triggerType, shardingParam[0], shardingParam[1]);
}
}
private static boolean isNumeric(String str){
try {
int result = Integer.valueOf(str);
return true;
} catch (NumberFormatException e) {
return false;
}
}
/**
* 处理任务
*/
private static void processTrigger(XxlJobGroup group, XxlJobInfo jobInfo, int finalFailRetryCount, TriggerTypeEnum triggerType, int index, int total){
// 阻塞处理策略 枚举
ExecutorBlockStrategyEnum blockStrategy = ExecutorBlockStrategyEnum.match(jobInfo.getExecutorBlockStrategy(), ExecutorBlockStrategyEnum.SERIAL_EXECUTION); // block strategy
// 路由策略 枚举
ExecutorRouteStrategyEnum executorRouteStrategyEnum = ExecutorRouteStrategyEnum.match(jobInfo.getExecutorRouteStrategy(), null); // route strategy
// 任务分片参数
String shardingParam = (ExecutorRouteStrategyEnum.SHARDING_BROADCAST==executorRouteStrategyEnum)?String.valueOf(index).concat("/").concat(String.valueOf(total)):null;
// 1、保存一条xxl_job_log记录（**）
XxlJobLog jobLog = new XxlJobLog();
jobLog.setJobGroup(jobInfo.getJobGroup());
jobLog.setJobId(jobInfo.getId());
jobLog.setTriggerTime(new Date());
XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().save(jobLog);
// 2、初始化任务执行参数triggerParam， 用于后续远程调度入参 以及 具体路由地址入参
TriggerParam triggerParam = new TriggerParam();
triggerParam.setJobId(jobInfo.getId());
triggerParam.setExecutorHandler(jobInfo.getExecutorHandler());
triggerParam.setExecutorParams(jobInfo.getExecutorParam());
triggerParam.setExecutorBlockStrategy(jobInfo.getExecutorBlockStrategy());
triggerParam.setExecutorTimeout(jobInfo.getExecutorTimeout());
triggerParam.setLogId(jobLog.getId());
triggerParam.setLogDateTime(jobLog.getTriggerTime().getTime());
triggerParam.setGlueType(jobInfo.getGlueType());
triggerParam.setGlueSource(jobInfo.getGlueSource());
triggerParam.setGlueUpdatetime(jobInfo.getGlueUpdatetime().getTime());
triggerParam.setBroadcastIndex(index);
triggerParam.setBroadcastTotal(total);
// 3、获取 具体任务执行地址 address
String address = null;
ReturnT<String> routeAddressResult = null;
if (group.getRegistryList()!=null && !group.getRegistryList().isEmpty()) {
if (ExecutorRouteStrategyEnum.SHARDING_BROADCAST == executorRouteStrategyEnum) {
// 分片广播 路由策略
if (index < group.getRegistryList().size()) {
address = group.getRegistryList().get(index);
} else {
address = group.getRegistryList().get(0);
}
} else {
// 另外9种分片广播策略
routeAddressResult = executorRouteStrategyEnum.getRouter().route(triggerParam, group.getRegistryList());
if (routeAddressResult.getCode() == ReturnT.SUCCESS_CODE) {
address = routeAddressResult.getContent();
}
}
} else {
routeAddressResult = new ReturnT<String>(ReturnT.FAIL_CODE, I18nUtil.getString("jobconf_trigger_address_empty"));
}
// 4、触发远程执行器
ReturnT<String> triggerResult = null;
if (address != null) {
triggerResult = runExecutor(triggerParam, address);//
} else {
// 获取具体执行地址为空，直接返回执行失败
triggerResult = new ReturnT<String>(ReturnT.FAIL_CODE, null);
}
// 5、收集 执行节点 信息
StringBuffer triggerMsgSb = new StringBuffer();
triggerMsgSb.append(I18nUtil.getString("jobconf_trigger_type")).append("：").append(triggerType.getTitle());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_admin_adress")).append("：").append(IpUtil.getIp());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_exe_regtype")).append("：")
.append( (group.getAddressType() == 0)?I18nUtil.getString("jobgroup_field_addressType_0"):I18nUtil.getString("jobgroup_field_addressType_1") );
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_exe_regaddress")).append("：").append(group.getRegistryList());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorRouteStrategy")).append("：").append(executorRouteStrategyEnum.getTitle());
if (shardingParam != null) {
triggerMsgSb.append("("+shardingParam+")");
}
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorBlockStrategy")).append("：").append(blockStrategy.getTitle());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_timeout")).append("：").append(jobInfo.getExecutorTimeout());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorFailRetryCount")).append("：").append(finalFailRetryCount);
triggerMsgSb.append("<br><br><span style=\"color:#00c0ef;\" > >>>"+ I18nUtil.getString("jobconf_trigger_run") +"<<< </span><br>")
.append((routeAddressResult!=null&&routeAddressResult.getMsg()!=null)?routeAddressResult.getMsg()+"<br><br>":"").append(triggerResult.getMsg()!=null?triggerResult.getMsg():"");
// 6、更新 执行节点 信息
jobLog.setExecutorAddress(address);
jobLog.setExecutorHandler(jobInfo.getExecutorHandler());
jobLog.setExecutorParam(jobInfo.getExecutorParam());
jobLog.setExecutorShardingParam(shardingParam);
jobLog.setExecutorFailRetryCount(finalFailRetryCount);
//jobLog.setTriggerTime();
jobLog.setTriggerCode(triggerResult.getCode());
jobLog.setTriggerMsg(triggerMsgSb.toString());
XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().updateTriggerInfo(jobLog);
}
/**
* 执行程序运行
*/
public static ReturnT<String> runExecutor(TriggerParam triggerParam, String address){
ReturnT<String> runResult = null;
try {
ExecutorBiz executorBiz = XxlJobScheduler.getExecutorBiz(address);
runResult = executorBiz.run(triggerParam);//
} catch (Exception e) {
runResult = new ReturnT<String>(ReturnT.FAIL_CODE, ThrowableUtil.toString(e));
}
StringBuffer runResultSB = new StringBuffer(I18nUtil.getString("jobconf_trigger_run") + "：");
runResultSB.append("<br>address：").append(address);
runResultSB.append("<br>code：").append(runResult.getCode());
runResultSB.append("<br>msg：").append(runResult.getMsg());
runResult.setMsg(runResultSB.toString());
return runResult;
}
}
5）step 5
1、com.xxl.job.admin.core.scheduler.XxlJobScheduler#init方法内部调用com.xxl.job.admin.core.thread.JobRegistryHelper#start方法，具体实现如下
package com.xxl.job.admin.core.thread;
import *;
public class JobRegistryHelper {
private static Logger logger = LoggerFactory.getLogger(JobRegistryHelper.class);
private static JobRegistryHelper instance = new JobRegistryHelper();
public static JobRegistryHelper getInstance(){
return instance;
}
private ThreadPoolExecutor registryOrRemoveThreadPool = null;
private Thread registryMonitorThread;
private volatile boolean toStop = false;
public void start(){
registryOrRemoveThreadPool = new ThreadPoolExecutor(
2,
10,
30L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<Runnable>(2000),
new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "xxl-job, admin JobRegistryMonitorHelper-registryOrRemoveThreadPool-" + r.hashCode());
}
},
new RejectedExecutionHandler() {
@Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
r.run();
}
});
registryMonitorThread = new Thread(new Runnable() {
@Override
public void run() {
while (!toStop) {
try {
//扫描xxl_job_group表所有自动注册的记录(address_type 执行器地址类型：0=自动注册、1=手动录入)
List<XxlJobGroup> groupList = XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().findByAddressType(0);
if (groupList!=null && !groupList.isEmpty()) {
//循环执行器，扫描xxl_job_registry表所有超时的执行器-任务执行节点记录（更新时间update_time超过90s）
List<Integer> ids = XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().findDead(RegistryConfig.DEAD_TIMEOUT, new Date());
if (ids!=null && ids.size()>0) {
//移除超时的 任务执行节点记录
XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().removeDead(ids);
}
// > - < //
//刷新 执行器 所有在线 任务执行节点 信息，并存储到变量appAddressMap
HashMap<String, List<String>> appAddressMap = new HashMap<String, List<String>>();
//扫描xxl_job_registry表所有未超时的执行器-任务执行节点记录（更新时间update_time在90s之内）
List<XxlJobRegistry> list = XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().findAll(RegistryConfig.DEAD_TIMEOUT, new Date());
if (list != null) {
for (XxlJobRegistry item: list) {
//过滤RegistryConfig.RegistType.ADMIN类型数据，找到RegistryConfig.RegistType.EXECUTOR数据
if (RegistryConfig.RegistType.EXECUTOR.name().equals(item.getRegistryGroup())) {
String appname = item.getRegistryKey();
List<String> registryList = appAddressMap.get(appname);
if (registryList == null) {
registryList = new ArrayList<String>();
}
if (!registryList.contains(item.getRegistryValue())) {
registryList.add(item.getRegistryValue());
}
appAddressMap.put(appname, registryList);//xxl_job_registry registry_key
}
}
}
//刷新 所有 执行器地址类型 为 自动注册类型 xxl_job_group表 执行器信息
for (XxlJobGroup group: groupList) {
List<String> registryList = appAddressMap.get(group.getAppname());//xxl_job_group app_name
String addressListStr = null;
if (registryList!=null && !registryList.isEmpty()) {
Collections.sort(registryList);
StringBuilder addressListSB = new StringBuilder();
for (String item:registryList) {
addressListSB.append(item).append(",");
}
addressListStr = addressListSB.toString();
addressListStr = addressListStr.substring(0, addressListStr.length()-1);
}
group.setAddressList(addressListStr);
group.setUpdateTime(new Date());
XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().update(group);
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
// 睡眠30s
try {
TimeUnit.SECONDS.sleep(RegistryConfig.BEAT_TIMEOUT);
} catch (InterruptedException e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
}
});
registryMonitorThread.setDaemon(true);
registryMonitorThread.setName("xxl-job, admin JobRegistryMonitorHelper-registryMonitorThread");
registryMonitorThread.start();
}
public void toStop(){
toStop = true;
registryOrRemoveThreadPool.shutdownNow();
registryMonitorThread.interrupt();
try {
registryMonitorThread.join();
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
/**
* 执行节点启动时，会每隔30s，主动向调度中心发起注册请求，即调用此方法
*/
public ReturnT<String> registry(RegistryParam registryParam) {
// 参数校验
if (!StringUtils.hasText(registryParam.getRegistryGroup())
|| !StringUtils.hasText(registryParam.getRegistryKey())
|| !StringUtils.hasText(registryParam.getRegistryValue())) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "Illegal Argument.");
}
registryOrRemoveThreadPool.execute(new Runnable() {
@Override
public void run() {
// 更新 xxl_job_registry表 任务执行节点 信息
int ret = XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().registryUpdate(registryParam.getRegistryGroup(), registryParam.getRegistryKey(), registryParam.getRegistryValue(), new Date());
if (ret < 1) {
// 没找到对应记录，就新增一条 xxl_job_registry表 任务执行节点 记录
XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().registrySave(registryParam.getRegistryGroup(), registryParam.getRegistryKey(), registryParam.getRegistryValue(), new Date());
// 刷新执行器 对应 任务执行节点信息（默认空实现）
freshGroupRegistryInfo(registryParam);
}
}
});
return ReturnT.SUCCESS;
}
/**
* 执行节点停止时，会主动向调度中心发起移除注册信息请求，即调用此方法
*/
public ReturnT<String> registryRemove(RegistryParam registryParam) {
// 参数校验
if (!StringUtils.hasText(registryParam.getRegistryGroup())
|| !StringUtils.hasText(registryParam.getRegistryKey())
|| !StringUtils.hasText(registryParam.getRegistryValue())) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "Illegal Argument.");
}
// async execute
registryOrRemoveThreadPool.execute(new Runnable() {
@Override
public void run() {
// 移除 xxl_job_registry表 任务执行节点 信息
int ret = XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().registryDelete(registryParam.getRegistryGroup(), registryParam.getRegistryKey(), registryParam.getRegistryValue());
if (ret > 0) {//执行成功
// 刷新执行器 对应 任务执行节点信息（默认空实现）
freshGroupRegistryInfo(registryParam);
}
}
});
return ReturnT.SUCCESS;
}
private void freshGroupRegistryInfo(RegistryParam registryParam){
}
}
6）step 6
1、com.xxl.job.admin.core.scheduler.XxlJobScheduler#init方法内部调用com.xxl.job.admin.core.thread.JobFailMonitorHelper#start方法，具体实现如下
package com.xxl.job.admin.core.thread;
import *;
public class JobFailMonitorHelper {
private static Logger logger = LoggerFactory.getLogger(JobFailMonitorHelper.class);
private static JobFailMonitorHelper instance = new JobFailMonitorHelper();
public static JobFailMonitorHelper getInstance(){
return instance;
}
private Thread monitorThread;
private volatile boolean toStop = false;
public void start(){
monitorThread = new Thread(new Runnable() {
@Override
public void run() {
while (!toStop) {
try {
// 循环扫描xxl_job_log表，每次取1000条记录，取出执行失败的记录集合
List<Long> failLogIds = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().findFailJobLogIds(1000);
if (failLogIds!=null && !failLogIds.isEmpty()) {
for (long failLogId: failLogIds) {
// 更新记录状态，标记记录正在处理中；锁记录，如果更新失败表示此记录锁已被其他人持有 ---- 这也是xxl中db锁的具体应用之一
int lockRet = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().updateAlarmStatus(failLogId, 0, -1);
if (lockRet < 1) {
continue;
}
XxlJobLog log = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().load(failLogId);
XxlJobInfo info = XxlJobAdminConfig.getAdminConfig().getXxlJobInfoDao().loadById(log.getJobId());
// 1、失败任务重试
if (log.getExecutorFailRetryCount() > 0) {
// 任务重试!!!
JobTriggerPoolHelper.trigger(log.getJobId(), TriggerTypeEnum.RETRY, (log.getExecutorFailRetryCount()-1), log.getExecutorShardingParam(), log.getExecutorParam(), null);
String retryMsg = "<br><br><span style=\"color:#F39C12;\" > >>>"+ I18nUtil.getString("jobconf_trigger_type_retry") +"<<< </span><br>";
log.setTriggerMsg(log.getTriggerMsg() + retryMsg);
XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().updateTriggerInfo(log);
}
// 2、执行 失败任务的报警 程序
int newAlarmStatus = 0; // 告警状态：0-默认、-1=锁定状态、1-无需告警、2-告警成功、3-告警失败
if (info != null) {
boolean alarmResult = XxlJobAdminConfig.getAdminConfig().getJobAlarmer().alarm(info, log);
newAlarmStatus = alarmResult?2:3;
} else {
newAlarmStatus = 1;
}
XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().updateAlarmStatus(failLogId, -1, newAlarmStatus);
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
// 睡眠10s
try {
TimeUnit.SECONDS.sleep(10);
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
}
});
monitorThread.setDaemon(true);
monitorThread.setName("xxl-job, admin JobFailMonitorHelper");
monitorThread.start();
}
public void toStop(){
toStop = true;
monitorThread.interrupt();
try {
monitorThread.join();
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
}
7）step 7
1、com.xxl.job.admin.core.scheduler.XxlJobScheduler#init方法内部调用com.xxl.job.admin.core.thread.JobCompleteHelper#start方法，具体实现如下
package com.xxl.job.admin.core.thread;
import *;
public class JobCompleteHelper {
private static Logger logger = LoggerFactory.getLogger(JobCompleteHelper.class);
private static JobCompleteHelper instance = new JobCompleteHelper();
public static JobCompleteHelper getInstance(){
return instance;
}
private ThreadPoolExecutor callbackThreadPool = null;
private Thread monitorThread;
private volatile boolean toStop = false;
public void start(){
callbackThreadPool = new ThreadPoolExecutor(
2,
20,
30L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<Runnable>(3000),
new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "xxl-job, admin JobLosedMonitorHelper-callbackThreadPool-" + r.hashCode());
}
},
new RejectedExecutionHandler() {
@Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
r.run();
}
});
monitorThread = new Thread(new Runnable() {
@Override
public void run() {
// wait for JobTriggerPoolHelper-init
try {
TimeUnit.MILLISECONDS.sleep(50);
} catch (InterruptedException e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
while (!toStop) {
try {
// 任务结果丢失处理：调度记录停留在 "运行中" 状态超过10min，且对应执行器心跳注册失败不在线，则将本地调度主动标记失败；
Date losedTime = DateUtil.addMinutes(new Date(), -10);
// 扫描xxl_job_log表，找到所有执行结果丢失任务记录id集合
List<Long> losedJobIds = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().findLostJobIds(losedTime);
if (losedJobIds!=null && losedJobIds.size()>0) {
for (Long logId: losedJobIds) {
XxlJobLog jobLog = new XxlJobLog();
jobLog.setId(logId);
jobLog.setHandleTime(new Date());// 设置处理时间
jobLog.setHandleCode(ReturnT.FAIL_CODE);// 设置处理结果状态码为500： 执行失败
jobLog.setHandleMsg( I18nUtil.getString("joblog_lost_fail") );// 任务结果丢失，标记失败
// 标记当前xxl_job_log表记录为执行失败
XxlJobCompleter.updateHandleInfoAndFinish(jobLog);
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
// 睡眠60s
try {
TimeUnit.SECONDS.sleep(60);
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
}
});
monitorThread.setDaemon(true);
monitorThread.setName("xxl-job, admin JobLosedMonitorHelper");
monitorThread.start();
}
public void toStop(){
toStop = true;
callbackThreadPool.shutdownNow();
monitorThread.interrupt();
try {
monitorThread.join();
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
/**
* 执行节点启动后，
* 任务回调 监控线程 & 失败回调重试 监控线程 会主动消费 com.xxl.job.core.thread.TriggerCallbackThread.callBackQueue队列，
* 主动向调度中心发起回调请求，即调用此方法
*/
public ReturnT<String> callback(List<HandleCallbackParam> callbackParamList) {
callbackThreadPool.execute(new Runnable() {
@Override
public void run() {
for (HandleCallbackParam handleCallbackParam: callbackParamList) {
//执行回调方法
ReturnT<String> callbackResult = callback(handleCallbackParam);
}
}
});
return ReturnT.SUCCESS;
}
private ReturnT<String> callback(HandleCallbackParam handleCallbackParam) {
// 加载xxl_job_log表记录
XxlJobLog log = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().load(handleCallbackParam.getLogId());
if (log == null) {//callback方法执行的前提是xxl_job_log表记录存在
return new ReturnT<String>(ReturnT.FAIL_CODE, "log item not found.");
}
// 判断handleCode避免重复执行
if (log.getHandleCode() > 0) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "log repeate callback."); // avoid repeat callback, trigger child job etc
}
// 重设handleMsg字段记录
StringBuffer handleMsg = new StringBuffer();
if (log.getHandleMsg()!=null) {
handleMsg.append(log.getHandleMsg()).append("<br>");
}
if (handleCallbackParam.getHandleMsg() != null) {
handleMsg.append(handleCallbackParam.getHandleMsg());
}
// 结束当前任务执行，如果有子任务则触发子任务执行，并且更新xxl_job_log表记录（更新handle_time、handle_code、handle_msg这三个字段）
log.setHandleTime(new Date());
log.setHandleCode(handleCallbackParam.getHandleCode());
log.setHandleMsg(handleMsg.toString());
XxlJobCompleter.updateHandleInfoAndFinish(log);
return ReturnT.SUCCESS;
}
}
8）step 8
1、com.xxl.job.admin.core.scheduler.XxlJobScheduler#init方法内部调用com.xxl.job.admin.core.thread.JobLogReportHelper#start方法，具体实现如下
package com.xxl.job.admin.core.thread;
import *;
public class JobLogReportHelper {
private static Logger logger = LoggerFactory.getLogger(JobLogReportHelper.class);
private static JobLogReportHelper instance = new JobLogReportHelper();
public static JobLogReportHelper getInstance(){
return instance;
}
private Thread logrThread;
private volatile boolean toStop = false;
public void start(){
logrThread = new Thread(new Runnable() {
@Override
public void run() {
long lastCleanLogTime = 0;// lastCleanLogTime上次清理日志时间
while (!toStop) {
// 1、刷新3天内xxl_job_log_report表，每日任务执行情况统计记录
try {
for (int i = 0; i < 3; i++) {
Calendar itemDay = Calendar.getInstance();
itemDay.add(Calendar.DAY_OF_MONTH, -i);
itemDay.set(Calendar.HOUR_OF_DAY, 0);
itemDay.set(Calendar.MINUTE, 0);
itemDay.set(Calendar.SECOND, 0);
itemDay.set(Calendar.MILLISECOND, 0);
Date todayFrom = itemDay.getTime();
itemDay.set(Calendar.HOUR_OF_DAY, 23);
itemDay.set(Calendar.MINUTE, 59);
itemDay.set(Calendar.SECOND, 59);
itemDay.set(Calendar.MILLISECOND, 999);
Date todayTo = itemDay.getTime();
XxlJobLogReport xxlJobLogReport = new XxlJobLogReport();
xxlJobLogReport.setTriggerDay(todayFrom);
xxlJobLogReport.setRunningCount(0);
xxlJobLogReport.setSucCount(0);
xxlJobLogReport.setFailCount(0);
// 查询时间段内xxl_job_log表执行情况统计
Map<String, Object> triggerCountMap = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().findLogReport(todayFrom, todayTo);
if (triggerCountMap!=null && triggerCountMap.size()>0) {
int triggerDayCount = triggerCountMap.containsKey("triggerDayCount")?Integer.valueOf(String.valueOf(triggerCountMap.get("triggerDayCount"))):0;
int triggerDayCountRunning = triggerCountMap.containsKey("triggerDayCountRunning")?Integer.valueOf(String.valueOf(triggerCountMap.get("triggerDayCountRunning"))):0;
int triggerDayCountSuc = triggerCountMap.containsKey("triggerDayCountSuc")?Integer.valueOf(String.valueOf(triggerCountMap.get("triggerDayCountSuc"))):0;
int triggerDayCountFail = triggerDayCount - triggerDayCountRunning - triggerDayCountSuc;
xxlJobLogReport.setRunningCount(triggerDayCountRunning);// 正在运行任务总数
xxlJobLogReport.setSucCount(triggerDayCountSuc);// 执行成功任务总数
xxlJobLogReport.setFailCount(triggerDayCountFail);// 执行失败任务总数
}
// 更新xxl_job_log_report表记录
int ret = XxlJobAdminConfig.getAdminConfig().getXxlJobLogReportDao().update(xxlJobLogReport);
if (ret < 1) {
// 不存在，则执行保存
XxlJobAdminConfig.getAdminConfig().getXxlJobLogReportDao().save(xxlJobLogReport);
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
// 2、判断日志最大保留时间是否开启 & 距离上次日志清理时间 超过1天
if (XxlJobAdminConfig.getAdminConfig().getLogretentiondays()>0
&& System.currentTimeMillis() - lastCleanLogTime > 24*60*60*1000) {
Calendar expiredDay = Calendar.getInstance();// 过期时间
expiredDay.add(Calendar.DAY_OF_MONTH, -1 * XxlJobAdminConfig.getAdminConfig().getLogretentiondays());
expiredDay.set(Calendar.HOUR_OF_DAY, 0);
expiredDay.set(Calendar.MINUTE, 0);
expiredDay.set(Calendar.SECOND, 0);
expiredDay.set(Calendar.MILLISECOND, 0);
Date clearBeforeTime = expiredDay.getTime();
// 清理过期日志
List<Long> logIds = null;
do {
logIds = XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().findClearLogIds(0, 0, clearBeforeTime, 0, 1000);
if (logIds!=null && logIds.size()>0) {
// 清理xxl_job_log表记录
XxlJobAdminConfig.getAdminConfig().getXxlJobLogDao().clearLog(logIds);
}
} while (logIds!=null && logIds.size()>0);
// 更新上次清理时间
lastCleanLogTime = System.currentTimeMillis();
}
// 睡眠1分钟
try {
TimeUnit.MINUTES.sleep(1);
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
}
});
logrThread.setDaemon(true);
logrThread.setName("xxl-job, admin JobLogReportHelper");
logrThread.start();
}
public void toStop(){
toStop = true;
// interrupt and wait
logrThread.interrupt();
try {
logrThread.join();
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
}
9）step 9
1、com.xxl.job.admin.core.scheduler.XxlJobScheduler#init方法内部调用com.xxl.job.admin.core.thread.JobScheduleHelper#start方法，具体实现如下
package com.xxl.job.admin.core.thread;
import *;
public class JobScheduleHelper {
private static Logger logger = LoggerFactory.getLogger(JobScheduleHelper.class);
private static JobScheduleHelper instance = new JobScheduleHelper();
public static JobScheduleHelper getInstance(){
return instance;
}
public static final long PRE_READ_MS = 5000;// 任务间隔大小
private Thread scheduleThread;
private Thread ringThread;
private volatile boolean scheduleThreadToStop = false; // 任务调度线程 是否关闭
private volatile boolean ringThreadToStop = false;
private volatile static Map<Integer, List<Integer>> ringData = new ConcurrentHashMap<>();
public void start(){
// schedule thread
scheduleThread = new Thread(new Runnable() {
@Override
public void run() {
// 睡眠
try {
TimeUnit.MILLISECONDS.sleep(5000 - System.currentTimeMillis()%1000 );
} catch (InterruptedException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
// pre-read count: treadpool-size * trigger-qps (each trigger cost 50ms, qps = 1000/50 = 20)
// 每次读取到任务个数 = （快任务 + 慢任务-调度线程池最大线程配置）* 20 （每个trigger花费50ms, QPS = 1000/50 = 20）
int preReadCount = (XxlJobAdminConfig.getAdminConfig().getTriggerPoolFastMax() + XxlJobAdminConfig.getAdminConfig().getTriggerPoolSlowMax()) * 20;
while (!scheduleThreadToStop) {
// 扫描任务
long start = System.currentTimeMillis();
Connection conn = null;
Boolean connAutoCommit = null;
PreparedStatement preparedStatement = null;
boolean preReadSuc = true;
try {
conn = XxlJobAdminConfig.getAdminConfig().getDataSource().getConnection();
connAutoCommit = conn.getAutoCommit();
conn.setAutoCommit(false);
// https://github.com/xuxueli/xxl-job/pull/2766
// 集群模式下xxl-job进行任务调度时，为了避免任务被重复调度，因此会先通过以下SQL对表xxl_job_lock的记录schedule_lock加锁
preparedStatement = conn.prepareStatement( "select * from xxl_job_lock where lock_name = 'schedule_lock' for update" );
preparedStatement.execute();
// tx start
// 1、预读开始
long nowTime = System.currentTimeMillis();
// 查询 xxl_job_info表 trigger_status =1 && trigger_next_time 小于等于 当前时间 + 5000ms，就是接下来 5 秒要执行到任务
List<XxlJobInfo> scheduleList = XxlJobAdminConfig.getAdminConfig().getXxlJobInfoDao().scheduleJobQuery(nowTime + PRE_READ_MS, preReadCount);
if (scheduleList!=null && scheduleList.size()>0) {
for (XxlJobInfo jobInfo: scheduleList) {
/**
* - - - - - - - - -17-18-19-20-21-22-23-24-25-26-27-28-29-30-31-32-34-35-36-37- - - - - - - - - - - - - - -
* ^ ^ ^
* | | |
* | | <- query
* 调度过期 <-|-> 立即执行 now 待执行 <-|
*/
if (nowTime > jobInfo.getTriggerNextTime() + PRE_READ_MS) {// 调度过期
// 1、调度过期策略 为 立即执行（则任务触发类型-调度过期补偿 : 立即执行一次）
MisfireStrategyEnum misfireStrategyEnum = MisfireStrategyEnum.match(jobInfo.getMisfireStrategy(), MisfireStrategyEnum.DO_NOTHING);
if (MisfireStrategyEnum.FIRE_ONCE_NOW == misfireStrategyEnum) {
JobTriggerPoolHelper.trigger(jobInfo.getId(), TriggerTypeEnum.MISFIRE, -1, null, null, null);
}
// 2、刷新XxlJobInfo 对象 调度状态、上次调度时间、下次调度时间
refreshNextValidTime(jobInfo, new Date());
} else if (nowTime > jobInfo.getTriggerNextTime()) {// 当前时间往前推5s之内
// 1、任务触发类型-Cron触发 : 立即执行一次
JobTriggerPoolHelper.trigger(jobInfo.getId(), TriggerTypeEnum.CRON, -1, null, null, null);
// 2、刷新XxlJobInfo 对象 调度状态、上次调度时间、下次调度时间
refreshNextValidTime(jobInfo, new Date());
// 3、如果接下来 5 秒内还执行则直接放到时间轮中
if (jobInfo.getTriggerStatus()==1 && nowTime + PRE_READ_MS > jobInfo.getTriggerNextTime()) {
int ringSecond = (int)((jobInfo.getTriggerNextTime()/1000)%60);
pushTimeRing(ringSecond, jobInfo.getId());
refreshNextValidTime(jobInfo, new Date(jobInfo.getTriggerNextTime()));
}
} else {// 当前时间往后推5s之内
// 1、将XxlJobInfo id直接放到时间轮中
int ringSecond = (int)((jobInfo.getTriggerNextTime()/1000)%60);
pushTimeRing(ringSecond, jobInfo.getId());
refreshNextValidTime(jobInfo, new Date(jobInfo.getTriggerNextTime()));
}
}
// 3、更新XxlJobInfo 对象（调度状态、上次调度时间、下次调度时间）
for (XxlJobInfo jobInfo: scheduleList) {
XxlJobAdminConfig.getAdminConfig().getXxlJobInfoDao().scheduleUpdate(jobInfo);
}
} else {
preReadSuc = false;// 预读失败
}
// tx stop
} catch (Exception e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
} finally {
// 事务提交，释放链接
if (conn != null) {
try {
conn.commit();
} catch (SQLException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
try {
conn.setAutoCommit(connAutoCommit);
} catch (SQLException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
try {
conn.close();
} catch (SQLException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
}
if (null != preparedStatement) {
try {
preparedStatement.close();
} catch (SQLException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
}
}
long cost = System.currentTimeMillis()-start;
// 等待数秒，时间对其（方式频繁请求数据库）
if (cost < 1000) { // 扫描时间小于1秒，等待
try {
// 预读周期 : 执行成功，睡眠1s内; 执行失败，睡眠5s内;
TimeUnit.MILLISECONDS.sleep((preReadSuc?1000:PRE_READ_MS) - System.currentTimeMillis()%1000);
} catch (InterruptedException e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
}
}
}
});
scheduleThread.setDaemon(true);
scheduleThread.setName("xxl-job, admin JobScheduleHelper#scheduleThread");
scheduleThread.start();
ringThread = new Thread(new Runnable() {
@Override
public void run() {
while (!ringThreadToStop) {
// 时间对其, 睡眠后刚好是整数秒（程序读秒）
try {
TimeUnit.MILLISECONDS.sleep(1000 - System.currentTimeMillis() % 1000);
} catch (InterruptedException e) {
if (!ringThreadToStop) {
logger.error(e.getMessage(), e);
}
}
try {
// 取2个刻度 时间环 待执行任务
List<Integer> ringItemData = new ArrayList<>();
int nowSecond = Calendar.getInstance().get(Calendar.SECOND); // 避免处理耗时太长，跨过刻度，向前校验一个刻度；
for (int i = 0; i < 2; i++) {
List<Integer> tmpData = ringData.remove( (nowSecond+60-i)%60 );
if (tmpData != null) {
ringItemData.addAll(tmpData);
}
}
// 循环执行任务
if (ringItemData.size() > 0) {
for (int jobId: ringItemData) {
JobTriggerPoolHelper.trigger(jobId, TriggerTypeEnum.CRON, -1, null, null, null);
}
ringItemData.clear();
}
} catch (Exception e) {
if (!ringThreadToStop) {
logger.error(e.getMessage(), e);
}
}
}
}
});
ringThread.setDaemon(true);
ringThread.setName("xxl-job, admin JobScheduleHelper#ringThread");
ringThread.start();
}
/**
* 刷新 调度状态、上次调度时间、下次调度时间
*/
private void refreshNextValidTime(XxlJobInfo jobInfo, Date fromTime) throws Exception {
Date nextValidTime = generateNextValidTime(jobInfo, fromTime);
if (nextValidTime != null) {
jobInfo.setTriggerLastTime(jobInfo.getTriggerNextTime());
jobInfo.setTriggerNextTime(nextValidTime.getTime());
} else {
jobInfo.setTriggerStatus(0);// 调度状态：0-停止，1-运行
jobInfo.setTriggerLastTime(0);
jobInfo.setTriggerNextTime(0);
}
}
private void pushTimeRing(int ringSecond, int jobId){
// push async ring
List<Integer> ringItemData = ringData.get(ringSecond);
if (ringItemData == null) {
ringItemData = new ArrayList<Integer>();
ringData.put(ringSecond, ringItemData);
}
ringItemData.add(jobId);
}
public void toStop(){
scheduleThreadToStop = true;
try {
TimeUnit.SECONDS.sleep(1);
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
if (scheduleThread.getState() != Thread.State.TERMINATED){
scheduleThread.interrupt();
try {
scheduleThread.join();
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
boolean hasRingData = false;
if (!ringData.isEmpty()) {
for (int second : ringData.keySet()) {
List<Integer> tmpData = ringData.get(second);
if (tmpData!=null && tmpData.size()>0) {
hasRingData = true;
break;
}
}
}
if (hasRingData) {
try {
TimeUnit.SECONDS.sleep(8);
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
ringThreadToStop = true;
try {
TimeUnit.SECONDS.sleep(1);
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
if (ringThread.getState() != Thread.State.TERMINATED){
ringThread.interrupt();
try {
ringThread.join();
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
}
/**
* 生成下次执行时间
*/
public static Date generateNextValidTime(XxlJobInfo jobInfo, Date fromTime) throws Exception {
ScheduleTypeEnum scheduleTypeEnum = ScheduleTypeEnum.match(jobInfo.getScheduleType(), null);
if (ScheduleTypeEnum.CRON == scheduleTypeEnum) {// 调度类型 : CORN
Date nextValidTime = new CronExpression(jobInfo.getScheduleConf()).getNextValidTimeAfter(fromTime);
return nextValidTime;
} else if (ScheduleTypeEnum.FIX_RATE == scheduleTypeEnum) {// 调度类型 : 固定速度
return new Date(fromTime.getTime() + Integer.valueOf(jobInfo.getScheduleConf())*1000 );
}
return null;
}
}
（二）执行器（frameless）启动流程
1.源码详解
1）step 1
1、执行器为普通java项目，启动方法为com.xxl.job.executor.sample.frameless.FramelessApplication#main，具体代码实现如下
package com.xxl.job.executor.sample.frameless;
import *;
public class FramelessApplication {
private static Logger logger = LoggerFactory.getLogger(FramelessApplication.class);
public static void main(String[] args) {
try {
FrameLessXxlJobConfig.getInstance().initXxlJobExecutor();
// 阻塞直到线程被打断
while (true) {
try {
TimeUnit.HOURS.sleep(1);
} catch (InterruptedException e) {
break;
}
}
} catch (Exception e) {
logger.error(e.getMessage(), e);
} finally {
FrameLessXxlJobConfig.getInstance().destroyXxlJobExecutor();
}
}
}
2）step 2
1、com.xxl.job.executor.sample.frameless.FramelessApplication#main方法，内部调用com.xxl.job.executor.sample.frameless.config.FrameLessXxlJobConfig#initXxlJobExecutor方法，具体实现如下
package com.xxl.job.executor.sample.frameless.config;
import *;
public class FrameLessXxlJobConfig {
private static Logger logger = LoggerFactory.getLogger(FrameLessXxlJobConfig.class);
private static FrameLessXxlJobConfig instance = new FrameLessXxlJobConfig();
public static FrameLessXxlJobConfig getInstance() {
return instance;
}
private XxlJobSimpleExecutor xxlJobExecutor = null;
public void initXxlJobExecutor() {
// 1、加载resources/xxl-job-executor.properties配置文件
Properties xxlJobProp = loadProperties("xxl-job-executor.properties");
// 2、初始化XxlJobSimpleExecutor对象
xxlJobExecutor = new XxlJobSimpleExecutor();
// 2.1、调度中心部署根地址 [选填]：如调度中心集群部署存在多个地址则用逗号分隔。执行器将会使用该地址进行"执行器心跳注册"和"任务结果回调"；为空则关闭自动注册；
// 执行器回调地址（xxl.job.admin.addresses）需要保持一致；执行器根据该配置进行执行器自动注册等操作。
xxlJobExecutor.setAdminAddresses(xxlJobProp.getProperty("xxl.job.admin.addresses"));
// 2.2、为提升系统安全性，调度中心和执行器进行安全性校验，双方AccessToken匹配才允许通讯；
// 调度中心和执行器，可通过配置项 "xxl.job.accessToken" 进行AccessToken的设置。
// 调度中心和执行器，如果需要正常通讯，只有两种设置；
// - 设置一：调度中心和执行器，均不设置AccessToken；关闭安全性校验；
// - 设置二：调度中心和执行器，设置了相同的AccessToken；
xxlJobExecutor.setAccessToken(xxlJobProp.getProperty("xxl.job.accessToken"));
// 2.3、执行器AppName [选填]：执行器心跳注册分组依据；为空则关闭自动注册
// 同一个执行器集群内AppName（xxl.job.executor.appname）需要保持一致；调度中心根据该配置动态发现不同集群的在线执行器列表。
xxlJobExecutor.setAppname(xxlJobProp.getProperty("xxl.job.executor.appname"));
// 2.4、”注册地址 / xxl.job.executor.address“，优先使用该配置作为注册地址，为空时使用内嵌服务 ”IP:PORT“ 作为注册地址。
// 从而更灵活的支持容器类型执行器动态IP和动态映射端口问题。
xxlJobExecutor.setAddress(xxlJobProp.getProperty("xxl.job.executor.address"));
// 2.5、执行器IP [选填]：默认为空表示自动获取IP，多网卡时可手动设置指定IP，该IP不会绑定Host仅作为通讯实用；地址信息用于 "执行器注册" 和 "调度中心请求并触发任务"；
xxlJobExecutor.setIp(xxlJobProp.getProperty("xxl.job.executor.ip"));
// 2.6、执行器端口号 [选填]：小于等于0则自动获取；默认端口为9999，单机部署多个执行器时，注意要配置不同执行器端口；
// 执行器实际上是一个内嵌的Server，默认端口9999
xxlJobExecutor.setPort(Integer.valueOf(xxlJobProp.getProperty("xxl.job.executor.port")));
// 2.7、执行器运行日志文件存储磁盘路径 [选填] ：需要对该路径拥有读写权限；为空则使用默认路径；
xxlJobExecutor.setLogPath(xxlJobProp.getProperty("xxl.job.executor.logpath"));
// 2.8、执行器日志文件保存天数 [选填] ： 过期日志自动清理, 限制值大于等于3时生效; 否则, 如-1, 关闭自动清理功能；
xxlJobExecutor.setLogRetentionDays(Integer.valueOf(xxlJobProp.getProperty("xxl.job.executor.logretentiondays")));
// 3、注册xxl执行器bean对象
xxlJobExecutor.setXxlJobBeanList(Arrays.asList(new SampleXxlJob()));
try {
// 4、执行启动方法
xxlJobExecutor.start();
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
}
public void destroyXxlJobExecutor() {
if (xxlJobExecutor != null) {
xxlJobExecutor.destroy();
}
}
public static Properties loadProperties(String propertyFileName) {
InputStreamReader in = null;
try {
ClassLoader loder = Thread.currentThread().getContextClassLoader();
in = new InputStreamReader(loder.getResourceAsStream(propertyFileName), "UTF-8");;
if (in != null) {
Properties prop = new Properties();
prop.load(in);
return prop;
}
} catch (IOException e) {
logger.error(e.getMessage(), e);
} finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
logger.error(e.getMessage(), e);
}
}
}
return null;
}
}
3）step 3
1、com.xxl.job.executor.sample.frameless.config.FrameLessXxlJobConfig#initXxlJobExecutor方法，内部调用com.xxl.job.core.executor.impl.XxlJobSimpleExecutor#start方法，具体实现如下
package com.xxl.job.core.executor.impl;
import *;
public class XxlJobSimpleExecutor extends XxlJobExecutor {
private static final Logger logger = LoggerFactory.getLogger(XxlJobSimpleExecutor.class);
private List<Object> xxlJobBeanList = new ArrayList<>();
public List<Object> getXxlJobBeanList() {
return xxlJobBeanList;
}
public void setXxlJobBeanList(List<Object> xxlJobBeanList) {
this.xxlJobBeanList = xxlJobBeanList;
}
@Override
public void start() {
// 初始化 xxl执行器处理方法 仓库
initJobHandlerMethodRepository(xxlJobBeanList);
// 执行启动方法
try {
super.start();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
@Override
public void destroy() {
super.destroy();
}
/**
* 初始化 xxl执行器处理方法 仓库
*/
private void initJobHandlerMethodRepository(List<Object> xxlJobBeanList) {
if (xxlJobBeanList==null || xxlJobBeanList.size()==0) {
return;
}
for (Object bean: xxlJobBeanList) {
Method[] methods = bean.getClass().getDeclaredMethods();
if (methods.length == 0) {
continue;
}
for (Method executeMethod : methods) {
XxlJob xxlJob = executeMethod.getAnnotation(XxlJob.class);
// 注册xxl执行器处理方法
// 将@XxlJob 上注册的 执行器方法、初始化方法、销毁方法 存储到变量com.xxl.job.core.executor.XxlJobExecutor#jobHandlerRepository
registJobHandler(xxlJob, bean, executeMethod);
}
}
}
}
4）step 4
1、com.xxl.job.core.executor.impl.XxlJobSimpleExecutor#start方法，内部com.xxl.job.core.executor.impl.XxlJobSimpleExecutor#initJobHandlerMethodRepository方法，调用基础类的com.xxl.job.core.executor.XxlJobExecutor#registJobHandler方法，具体代码实现如下
2、com.xxl.job.core.executor.impl.XxlJobSimpleExecutor#start方法，内部调用基础类的com.xxl.job.core.executor.XxlJobExecutor#start方法，具体代码实现如下
package com.xxl.job.core.executor;
import *;
public class XxlJobExecutor {
private static final Logger logger = LoggerFactory.getLogger(XxlJobExecutor.class);
private String adminAddresses;
private String accessToken;
private String appname;
private String address;
private String ip;
private int port;
private String logPath;
private int logRetentionDays;
public void setAdminAddresses(String adminAddresses) {
this.adminAddresses = adminAddresses;
}
public void setAccessToken(String accessToken) {
this.accessToken = accessToken;
}
public void setAppname(String appname) {
this.appname = appname;
}
public void setAddress(String address) {
this.address = address;
}
public void setIp(String ip) {
this.ip = ip;
}
public void setPort(int port) {
this.port = port;
}
public void setLogPath(String logPath) {
this.logPath = logPath;
}
public void setLogRetentionDays(int logRetentionDays) {
this.logRetentionDays = logRetentionDays;
}
public void start() throws Exception {
// 1、初始化 执行器运行日志文件存储磁盘路径（xxl.job.executor.logpath）、及gluesource子目录
XxlJobFileAppender.initLogPath(logPath);
// 2、初始化 调度中心client请求对象（xxl.job.admin.addresses，xxl.job.accessToken）
// 并赋值给com.xxl.job.core.executor.XxlJobExecutor.adminBizList对象
initAdminBizList(adminAddresses, accessToken);
// 3、启动 过期日志自动清理 监控线程
JobLogFileCleanThread.getInstance().start(logRetentionDays);
// 4.1、启动 任务回调 监控线程，并赋值给com.xxl.job.core.thread.TriggerCallbackThread.triggerCallbackThread
// 主要负责消耗com.xxl.job.core.thread.TriggerCallbackThread.callBackQueue队列中的数据，进行任务回调
// 4.2、启动 失败回调重试 监控线程，并赋值给com.xxl.job.core.thread.TriggerCallbackThread.triggerRetryCallbackThread
// 主要负责 加载失败回调参数（磁盘文件反序列化），并主动执行回调方法
TriggerCallbackThread.getInstance().start();
// 5、初始化内嵌服务，用户注册中心调用（netty实现）
initEmbedServer(address, ip, port, appname, accessToken);
}
public void destroy(){
// destroy executor-server
stopEmbedServer();
// destroy jobThreadRepository
if (jobThreadRepository.size() > 0) {
for (Map.Entry<Integer, JobThread> item: jobThreadRepository.entrySet()) {
JobThread oldJobThread = removeJobThread(item.getKey(), "web container destroy and kill the job.");
// wait for job thread push result to callback queue
if (oldJobThread != null) {
try {
oldJobThread.join();
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
}
jobThreadRepository.clear();
}
jobHandlerRepository.clear();
JobLogFileCleanThread.getInstance().toStop();
TriggerCallbackThread.getInstance().toStop();
}
private static List<AdminBiz> adminBizList;// 调度中心client请求对象集合
/**
* 初始化 调度中心client请求对象,并赋值给变量com.xxl.job.core.executor.XxlJobExecutor.adminBizList
*/
private void initAdminBizList(String adminAddresses, String accessToken) throws Exception {
if (adminAddresses!=null && adminAddresses.trim().length()>0) {
for (String address: adminAddresses.trim().split(",")) { // 多个逗号分割
if (address!=null && address.trim().length()>0) {
AdminBiz adminBiz = new AdminBizClient(address.trim(), accessToken);
if (adminBizList == null) {
adminBizList = new ArrayList<AdminBiz>();
}
adminBizList.add(adminBiz);
}
}
}
}
public static List<AdminBiz> getAdminBizList(){
return adminBizList;
}
private EmbedServer embedServer = null;
private void initEmbedServer(String address, String ip, int port, String appname, String accessToken) throws Exception {
// 1、获取ip、port（赋予初始值）
port = port>0?port: NetUtil.findAvailablePort(9999);// 获取可用的端口，默认端口9999
ip = (ip!=null&&ip.trim().length()>0)?ip: IpUtil.getIp();// 获取本机地址
// 2、默认使用address来注册，如果address为空则使用ip:port
if (address==null || address.trim().length()==0) {
String ip_port_address = IpUtil.getIpPort(ip, port);
address = "http://{ip_port}/".replace("{ip_port}", ip_port_address);
}
if (accessToken==null || accessToken.trim().length()==0) {
logger.warn("xxl-job accessToken is empty.");
}
// 3、启动netty服务 监控线程
embedServer = new EmbedServer();
embedServer.start(address, port, appname, accessToken);
}
private void stopEmbedServer() {
// stop provider factory
if (embedServer != null) {
try {
embedServer.stop();
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
}
}
private static ConcurrentMap<String, IJobHandler> jobHandlerRepository = new ConcurrentHashMap<String, IJobHandler>();
public static IJobHandler loadJobHandler(String name){
return jobHandlerRepository.get(name);
}
/**
* 存储到变量com.xxl.job.core.executor.XxlJobExecutor#jobHandlerRepository
*/
public static IJobHandler registJobHandler(String name, IJobHandler jobHandler){
return jobHandlerRepository.put(name, jobHandler);
}
/**
* 注册xxl执行器处理方法（执行器方法、初始化方法、销毁方法）
*/
protected void registJobHandler(XxlJob xxlJob, Object bean, Method executeMethod){
// 1、跳过未加@XxlJob注解方法
if (xxlJob == null) {
return;
}
// 2、获取@XxlJob注解 上配置的 执行器方法、init方法、destory方法配置
String name = xxlJob.value();// 获取执行器名称
Class<?> clazz = bean.getClass();// 获取bean全类名
String methodName = executeMethod.getName();// 获取方法名称
if (name.trim().length() == 0) {
throw new RuntimeException("xxl-job method-jobhandler name invalid");
}
if (loadJobHandler(name) != null) {
throw new RuntimeException("xxl-job jobhandler[" + name + "] naming conflicts.");
}
executeMethod.setAccessible(true);// 打开反射访问权限
// 获取@XxlJob注解上配置的init初始化方法、destroy销毁方法
Method initMethod = null;
Method destroyMethod = null;
if (xxlJob.init().trim().length() > 0) {
try {
initMethod = clazz.getDeclaredMethod(xxlJob.init());
initMethod.setAccessible(true);
} catch (NoSuchMethodException e) {
throw new RuntimeException(e);
}
}
if (xxlJob.destroy().trim().length() > 0) {
try {
destroyMethod = clazz.getDeclaredMethod(xxlJob.destroy());
destroyMethod.setAccessible(true);
} catch (NoSuchMethodException e) {
throw new RuntimeException(e);
}
}
// 注册jobhandler（存储到变量com.xxl.job.core.executor.XxlJobExecutor#jobHandlerRepository）
registJobHandler(name, new MethodJobHandler(bean, executeMethod, initMethod, destroyMethod));// MethodJobHandler extends IJobHandler
}
private static ConcurrentMap<Integer, JobThread> jobThreadRepository = new ConcurrentHashMap<Integer, JobThread>();
public static JobThread registJobThread(int jobId, IJobHandler handler, String removeOldReason){
JobThread newJobThread = new JobThread(jobId, handler);
newJobThread.start();// 线程启动
JobThread oldJobThread = jobThreadRepository.put(jobId, newJobThread);
if (oldJobThread != null) {
oldJobThread.toStop(removeOldReason);
oldJobThread.interrupt();
}
return newJobThread;
}
/**
* 从com.xxl.job.core.executor.XxlJobExecutor.jobThreadRepository缓存中移除JobThread，并且停止线程
*/
public static JobThread removeJobThread(int jobId, String removeOldReason){
JobThread oldJobThread = jobThreadRepository.remove(jobId);
if (oldJobThread != null) {
oldJobThread.toStop(removeOldReason);
oldJobThread.interrupt();
return oldJobThread;
}
return null;
}
public static JobThread loadJobThread(int jobId){
return jobThreadRepository.get(jobId);
}
}
5）step 5
1、咱们对com.xxl.job.core.executor.XxlJobExecutor#start实现进行逐行分析，首先执行com.xxl.job.core.log.XxlJobFileAppender#initLogPath方法，具体代码实现如下
package com.xxl.job.core.log;
import *;
public class XxlJobFileAppender {
private static Logger logger = LoggerFactory.getLogger(XxlJobFileAppender.class);
private static String logBasePath = "/data/applogs/xxl-job/jobhandler";
private static String glueSrcPath = logBasePath.concat("/gluesource");
public static void initLogPath(String logPath){
// logPath值不为空，则覆盖com.xxl.job.core.log.XxlJobFileAppender.logBasePath的值
if (logPath!=null && logPath.trim().length()>0) {
logBasePath = logPath;
}
// 创建目录
File logPathDir = new File(logBasePath);
if (!logPathDir.exists()) {
logPathDir.mkdirs();
}
logBasePath = logPathDir.getPath();// 获取真实路径，并覆盖logBasePath的值
// 创建gluesource子目录
File glueBaseDir = new File(logPathDir, "gluesource");
if (!glueBaseDir.exists()) {
glueBaseDir.mkdirs();
}
glueSrcPath = glueBaseDir.getPath();// 获取真实路径，并覆盖glueSrcPath的值
}
public static String getLogPath() {
return logBasePath;
}
public static String getGlueSrcPath() {
return glueSrcPath;
}
/**
* 获取日志文件 路径名称（目录不存在则先创建目录）
*/
public static String makeLogFileName(Date triggerDate, long logId) {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
File logFilePath = new File(getLogPath(), sdf.format(triggerDate));
if (!logFilePath.exists()) {
logFilePath.mkdir();
}
String logFileName = logFilePath.getPath()
.concat(File.separator)
.concat(String.valueOf(logId))
.concat(".log");
return logFileName;
}
public static void appendLog(String logFileName, String appendLog) {
if (logFileName==null || logFileName.trim().length()==0) {
return;
}
File logFile = new File(logFileName);
if (!logFile.exists()) {
try {
logFile.createNewFile();
} catch (IOException e) {
logger.error(e.getMessage(), e);
return;
}
}
if (appendLog == null) {
appendLog = "";
}
appendLog += "\r\n";
FileOutputStream fos = null;
try {
fos = new FileOutputStream(logFile, true);
fos.write(appendLog.getBytes("utf-8"));
fos.flush();
} catch (Exception e) {
logger.error(e.getMessage(), e);
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
logger.error(e.getMessage(), e);
}
}
}
}
public static LogResult readLog(String logFileName, int fromLineNum){
if (logFileName==null || logFileName.trim().length()==0) {
return new LogResult(fromLineNum, 0, "readLog fail, logFile not found", true);
}
File logFile = new File(logFileName);
if (!logFile.exists()) {
return new LogResult(fromLineNum, 0, "readLog fail, logFile not exists", true);
}
StringBuffer logContentBuffer = new StringBuffer();
int toLineNum = 0;
LineNumberReader reader = null;
try {
reader = new LineNumberReader(new InputStreamReader(new FileInputStream(logFile), "utf-8"));
String line = null;
while ((line = reader.readLine())!=null) {
toLineNum = reader.getLineNumber();
if (toLineNum >= fromLineNum) {
logContentBuffer.append(line).append("\n");
}
}
} catch (IOException e) {
logger.error(e.getMessage(), e);
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException e) {
logger.error(e.getMessage(), e);
}
}
}
LogResult logResult = new LogResult(fromLineNum, toLineNum, logContentBuffer.toString(), false);
return logResult;
}
public static String readLines(File logFile){
BufferedReader reader = null;
try {
reader = new BufferedReader(new InputStreamReader(new FileInputStream(logFile), "utf-8"));
if (reader != null) {
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line).append("\n");
}
return sb.toString();
}
} catch (IOException e) {
logger.error(e.getMessage(), e);
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException e) {
logger.error(e.getMessage(), e);
}
}
}
return null;
}
}
6）step 6
1、com.xxl.job.core.executor.XxlJobExecutor#start内部执行com.xxl.job.core.executor.XxlJobExecutor#initAdminBizList方法，具体现实见step 4
7）step 7
1、com.xxl.job.core.executor.XxlJobExecutor#start内部调用com.xxl.job.core.thread.JobLogFileCleanThread#start方法，具体代码实现如下
package com.xxl.job.core.thread;
import *;
public class JobLogFileCleanThread {
private static Logger logger = LoggerFactory.getLogger(JobLogFileCleanThread.class);
private static JobLogFileCleanThread instance = new JobLogFileCleanThread();
public static JobLogFileCleanThread getInstance(){
return instance;
}
private Thread localThread;
private volatile boolean toStop = false;
public void start(final long logRetentionDays){
// 执行器日志文件保存天数xxl.job.executor.logretentiondays必须大于3，否则关闭自动清理功能
if (logRetentionDays < 3 ) {
return;
}
localThread = new Thread(new Runnable() {
@Override
public void run() {
while (!toStop) {
try {
// 清理执行器运行日志文件存储磁盘路径（xxl.job.executor.logpath）下，所有过期（xxl.job.executor.logretentiondays）日志
File[] childDirs = new File(XxlJobFileAppender.getLogPath()).listFiles();
if (childDirs!=null && childDirs.length>0) {
Calendar todayCal = Calendar.getInstance();
todayCal.set(Calendar.HOUR_OF_DAY,0);
todayCal.set(Calendar.MINUTE,0);
todayCal.set(Calendar.SECOND,0);
todayCal.set(Calendar.MILLISECOND,0);
Date todayDate = todayCal.getTime();
for (File childFile: childDirs) {
// 不是目录 或者 文件夹不包含-，则返回
if (!childFile.isDirectory()) {
continue;
}
if (childFile.getName().indexOf("-") == -1) {
continue;
}
// 获取文件夹创建时间（形如yyyy-MM-dd）
Date logFileCreateDate = null;
try {
SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyyy-MM-dd");
logFileCreateDate = simpleDateFormat.parse(childFile.getName());
} catch (ParseException e) {
logger.error(e.getMessage(), e);
}
if (logFileCreateDate == null) {
continue;
}
// 过期，则递归删除此目录
if ((todayDate.getTime()-logFileCreateDate.getTime()) >= logRetentionDays * (24 * 60 * 60 * 1000) ) {
FileUtil.deleteRecursively(childFile);
}
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
// 睡眠一天
try {
TimeUnit.DAYS.sleep(1);
} catch (InterruptedException e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
}
});
localThread.setDaemon(true);
localThread.setName("xxl-job, executor JobLogFileCleanThread");
localThread.start();
}
public void toStop() {
toStop = true;
if (localThread == null) {
return;
}
localThread.interrupt();
try {
localThread.join();
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
}
8）step 8
1、com.xxl.job.core.executor.XxlJobExecutor#start内部调用com.xxl.job.core.thread.TriggerCallbackThread#start方法，具体代码实现如下
package com.xxl.job.core.thread;
import *;
public class TriggerCallbackThread {
private static Logger logger = LoggerFactory.getLogger(TriggerCallbackThread.class);
private static TriggerCallbackThread instance = new TriggerCallbackThread();
public static TriggerCallbackThread getInstance(){
return instance;
}
private LinkedBlockingQueue<HandleCallbackParam> callBackQueue = new LinkedBlockingQueue<HandleCallbackParam>();
public static void pushCallBack(HandleCallbackParam callback){
getInstance().callBackQueue.add(callback);
}
private Thread triggerCallbackThread;
private Thread triggerRetryCallbackThread;
private volatile boolean toStop = false;
public void start() {
// 执行器回调 配置错误，直接返回（xxl.job.admin.addresses）
if (XxlJobExecutor.getAdminBizList() == null) {
return;
}
//1、任务回调 监控线程
triggerCallbackThread = new Thread(new Runnable() {
@Override
public void run() {
// 1、正常回调
while(!toStop){
try {
// 取阻塞队列com.xxl.job.core.thread.TriggerCallbackThread.callBackQueue中取值
// put()方法向队列中生产数据，当队列满时，线程阻塞; take()方法从队列中消费数据，当队列为空是，线程阻塞;
HandleCallbackParam callback = getInstance().callBackQueue.take();
if (callback != null) {
// 将队列中的数据 传递给 集合callbackParamList
List<HandleCallbackParam> callbackParamList = new ArrayList<HandleCallbackParam>();
int drainToNum = getInstance().callBackQueue.drainTo(callbackParamList);
callbackParamList.add(callback);// 把第一个参数 补齐
// 执行回调
if (callbackParamList!=null && callbackParamList.size()>0) {
doCallback(callbackParamList);
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
// 2、任务结束，执行最后一次回调
try {
List<HandleCallbackParam> callbackParamList = new ArrayList<HandleCallbackParam>();
int drainToNum = getInstance().callBackQueue.drainTo(callbackParamList);
if (callbackParamList!=null && callbackParamList.size()>0) {
doCallback(callbackParamList);
}
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
});
triggerCallbackThread.setDaemon(true);
triggerCallbackThread.setName("xxl-job, executor TriggerCallbackThread");
triggerCallbackThread.start();
//2、失败回调重试 监控线程
triggerRetryCallbackThread = new Thread(new Runnable() {
@Override
public void run() {
while(!toStop){
try {
//2、失败回调重试
retryFailCallbackFile();
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
//2、睡眠30s
try {
TimeUnit.SECONDS.sleep(RegistryConfig.BEAT_TIMEOUT);
} catch (InterruptedException e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
}
});
triggerRetryCallbackThread.setDaemon(true);
triggerRetryCallbackThread.start();
}
public void toStop(){
toStop = true;
if (triggerCallbackThread != null) {
triggerCallbackThread.interrupt();
try {
triggerCallbackThread.join();
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
if (triggerRetryCallbackThread != null) {
triggerRetryCallbackThread.interrupt();
try {
triggerRetryCallbackThread.join();
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
}
private void doCallback(List<HandleCallbackParam> callbackParamList){
boolean callbackRet = false;
for (AdminBiz adminBiz: XxlJobExecutor.getAdminBizList()) {// 调度中心client请求对象集合（xxl.job.admin.addresses）
try {
// 执行回调方法（向调度中心发请求）
ReturnT<String> callbackResult = adminBiz.callback(callbackParamList);
if (callbackResult!=null && ReturnT.SUCCESS_CODE == callbackResult.getCode()) {// 执行成功
callbackLog(callbackParamList, "<br>- xxl-job job callback finish.");
callbackRet = true;
break;
} else {// 执行失败
callbackLog(callbackParamList, "<br>- xxl-job job callback fail, callbackResult:" + callbackResult);
}
} catch (Exception e) {// 执行异常
callbackLog(callbackParamList, "<br>- xxl-job job callback error, errorMsg:" + e.getMessage());
}
}
if (!callbackRet) {// 执行失败
// 追加回调失败文件（序列化回调参数）
appendFailCallbackFile(callbackParamList);
}
}
/**
* 记录任务执行日志
*/
private void callbackLog(List<HandleCallbackParam> callbackParamList, String logContent){
for (HandleCallbackParam callbackParam: callbackParamList) {
// 查找 任务执行日志文件所在 路径名称
String logFileName = XxlJobFileAppender.makeLogFileName(new Date(callbackParam.getLogDateTim()), callbackParam.getLogId());
// 线程间传递XxlJobContext对象（日志文件 路径名称）
XxlJobContext.setXxlJobContext(new XxlJobContext(
-1,
null,
logFileName,
-1,
-1));
// 记录日志
XxlJobHelper.log(logContent);
}
}
/**
* 回调失败文件所在目录
*/
private static String failCallbackFilePath = XxlJobFileAppender.getLogPath().concat(File.separator).concat("callbacklog").concat(File.separator);
/**
* 回调失败文件地址
*/
private static String failCallbackFileName = failCallbackFilePath.concat("xxl-job-callback-{x}").concat(".log");
/**
* 追加回调失败文件
*/
private void appendFailCallbackFile(List<HandleCallbackParam> callbackParamList){
if (callbackParamList==null || callbackParamList.size()==0) {
return;
}
byte[] callbackParamList_bytes = JdkSerializeTool.serialize(callbackParamList);
// 获取回调日志文件名称
File callbackLogFile = new File(failCallbackFileName.replace("{x}", String.valueOf(System.currentTimeMillis())));
if (callbackLogFile.exists()) {
// 文件存在就往文件名后加-i，直到找到一个不存在的文件（最大为-99）
for (int i = 0; i < 100; i++) {
callbackLogFile = new File(failCallbackFileName.replace("{x}", String.valueOf(System.currentTimeMillis()).concat("-").concat(String.valueOf(i)) ));
if (!callbackLogFile.exists()) {
break;
}
}
}
// 写入磁盘（序列化）
FileUtil.writeFileContent(callbackLogFile, callbackParamList_bytes);
}
private void retryFailCallbackFile(){
//1、目录不存在则结束
File callbackLogPath = new File(failCallbackFilePath);
if (!callbackLogPath.exists()) {
return;
}
//2、是文件则直接删除
if (callbackLogPath.isFile()) {
callbackLogPath.delete();
}
//3、不是目录 或者 子目录为空 或者 子目录数量不为空 则直接结束
if (!(callbackLogPath.isDirectory() && callbackLogPath.list()!=null && callbackLogPath.list().length>0)) {
return;
}
//4、加载失败任务回调 参数，重试
for (File callbaclLogFile: callbackLogPath.listFiles()) {
//反序列化callbackParamList对象
byte[] callbackParamList_bytes = FileUtil.readFileContent(callbaclLogFile);
//内容为空，则删除
if(callbackParamList_bytes == null || callbackParamList_bytes.length < 1){
callbaclLogFile.delete();
continue;
}
List<HandleCallbackParam> callbackParamList = (List<HandleCallbackParam>) JdkSerializeTool.deserialize(callbackParamList_bytes, List.class);
//删除此文件
callbaclLogFile.delete();
//主动执行回调方法
doCallback(callbackParamList);
}
}
}
9）step 9
1、com.xxl.job.core.executor.XxlJobExecutor#start方法，内部调用方法com.xxl.job.core.executor.XxlJobExecutor#initEmbedServer，具体代码见step 4
2、com.xxl.job.core.executor.XxlJobExecutor#initEmbedServer内部调用方法com.xxl.job.core.server.EmbedServer#start，具体代码如下
package com.xxl.job.core.server;
import *;
public class EmbedServer {
private static final Logger logger = LoggerFactory.getLogger(EmbedServer.class);
private ExecutorBiz executorBiz;
private Thread thread;
public void start(final String address, final int port, final String appname, final String accessToken) {
//1、初始化com.xxl.job.core.biz.ExecutorBiz对象
executorBiz = new ExecutorBizImpl();
//2、启动netty服务 监控线程
thread = new Thread(new Runnable() {
@Override
public void run() {
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
//初始化业务请求线程池
ThreadPoolExecutor bizThreadPool = new ThreadPoolExecutor(
0,
200,
60L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<Runnable>(2000),
new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "xxl-job, EmbedServer bizThreadPool-" + r.hashCode());
}
},
new RejectedExecutionHandler() {
@Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
throw new RuntimeException("xxl-job, EmbedServer bizThreadPool is EXHAUSTED!");
}
});
try {
//启动服务
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
public void initChannel(SocketChannel channel) throws Exception {
channel.pipeline()
/**
* 心跳检测处理器
*
* 对Netty的IdleStateHandler的认识
* 首先，此Handler位于：io.netty.handler.timeout包下。
* 说明：
* 当一个channel在一定时间内没有发生：读、写操作时或读写都未发生时，就会触发⼀个IdleStateEvent事件。事件的状态共有3种，对应的枚举类是：IdleState。
*
* 属性和说明：
* readerIdleTime：如果在指定的时间周期内，没有读操作发⽣，就会触发⼀个状态为IdleState.READER_IDLE的IdleStateEvent事件。如果要禁⽤它的话，就设置readerIdleTime=0。
* writerIdleTime：如果在指定的时间周期内，没有写操作发⽣的话，就会触发⼀个状态为IdleState.WRITER_IDLE的IdleStateEvent事件。如果要禁⽤它的话，设置其writerIdleTime=0即可。
* allIdleTime：如果在指定的时间周期内，当既没有读也没有写操作发⽣的话，就触发⼀个状态为IdleState.ALL_IDLE事件。如果要禁⽤它的话，就设置allIdleTime=0。
*/
.addLast(new IdleStateHandler(0, 0, 30 * 3, TimeUnit.SECONDS)) // beat 3N, close if idle
/**
* http编解码器处理器
*/
.addLast(new HttpServerCodec())
/**
* 合并request、reponse请求，保证消息的完整性（最大长度5 * 1024 * 1024 防止拆包、粘包、半包等导致数据发送、接收不完整）
*/
.addLast(new HttpObjectAggregator(5 * 1024 * 1024))
/**
* 自定义处理器
*/
.addLast(new EmbedHttpServerHandler(executorBiz, accessToken, bizThreadPool));
}
})
.childOption(ChannelOption.SO_KEEPALIVE, true);
//绑定端口
ChannelFuture future = bootstrap.bind(port).sync();
//开始注册
startRegistry(appname, address);
//等待直到关闭服务
future.channel().closeFuture().sync();
} catch (InterruptedException e) {
logger.info("xxl-job remoting server stop.");
} catch (Exception e) {
logger.error(e.getMessage(), e);
} finally {
//关闭时间循环组
try {
workerGroup.shutdownGracefully();
bossGroup.shutdownGracefully();
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
}
}
});
thread.setDaemon(true);
thread.start();
}
public void stop() throws Exception {
if (thread != null && thread.isAlive()) {
thread.interrupt();
}
stopRegistry();
}
public static class EmbedHttpServerHandler extends SimpleChannelInboundHandler<FullHttpRequest> {
private static final Logger logger = LoggerFactory.getLogger(EmbedHttpServerHandler.class);
private ExecutorBiz executorBiz;
private String accessToken;
private ThreadPoolExecutor bizThreadPool;
public EmbedHttpServerHandler(ExecutorBiz executorBiz, String accessToken, ThreadPoolExecutor bizThreadPool) {
this.executorBiz = executorBiz;
this.accessToken = accessToken;
this.bizThreadPool = bizThreadPool;
}
/**
* 接收请求时
*/
@Override
protected void channelRead0(final ChannelHandlerContext ctx, FullHttpRequest msg) throws Exception {
// request parse
//final byte[] requestBytes = ByteBufUtil.getBytes(msg.content()); // byteBuf.toString(io.netty.util.CharsetUtil.UTF_8);
String requestData = msg.content().toString(CharsetUtil.UTF_8);
String uri = msg.uri();
HttpMethod httpMethod = msg.method();
boolean keepAlive = HttpUtil.isKeepAlive(msg);
String accessTokenReq = msg.headers().get(XxlJobRemotingUtil.XXL_JOB_ACCESS_TOKEN);
// 异步处理请求
bizThreadPool.execute(new Runnable() {
@Override
public void run() {
//1、处理请求
Object responseObj = process(httpMethod, uri, requestData, accessTokenReq);
//2、处理结果转json
String responseJson = GsonTool.toJson(responseObj);
//3、写出到响应体response
writeResponse(ctx, keepAlive, responseJson);
}
});
}
private Object process(HttpMethod httpMethod, String uri, String requestData, String accessTokenReq) {
//参数校验：POST请求 && uri不为空 && （accessToken为空 || accessToken一致）
if (HttpMethod.POST != httpMethod) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "invalid request, HttpMethod not support.");
}
if (uri == null || uri.trim().length() == 0) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "invalid request, uri-mapping empty.");
}
if (accessToken != null
&& accessToken.trim().length() > 0
&& !accessToken.equals(accessTokenReq)) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "The access token is wrong.");
}
//api服务映射
try {
switch (uri) {
case "/beat"://心跳检测
return executorBiz.beat();
case "/idleBeat"://闲置检测
IdleBeatParam idleBeatParam = GsonTool.fromJson(requestData, IdleBeatParam.class);
return executorBiz.idleBeat(idleBeatParam);
case "/run"://执行任务
TriggerParam triggerParam = GsonTool.fromJson(requestData, TriggerParam.class);
return executorBiz.run(triggerParam);
case "/kill"://关闭
KillParam killParam = GsonTool.fromJson(requestData, KillParam.class);
return executorBiz.kill(killParam);
case "/log"://记录日志
LogParam logParam = GsonTool.fromJson(requestData, LogParam.class);
return executorBiz.log(logParam);
default:
return new ReturnT<String>(ReturnT.FAIL_CODE, "invalid request, uri-mapping(" + uri + ") not found.");
}
} catch (Exception e) {
logger.error(e.getMessage(), e);
return new ReturnT<String>(ReturnT.FAIL_CODE, "request error:" + ThrowableUtil.toString(e));
}
}
/**
* 写出到请求体
*/
private void writeResponse(ChannelHandlerContext ctx, boolean keepAlive, String responseJson) {
FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK, Unpooled.copiedBuffer(responseJson, CharsetUtil.UTF_8));
response.headers().set(HttpHeaderNames.CONTENT_TYPE, "text/html;charset=UTF-8");
response.headers().set(HttpHeaderNames.CONTENT_LENGTH, response.content().readableBytes());
if (keepAlive) {
response.headers().set(HttpHeaderNames.CONNECTION, HttpHeaderValues.KEEP_ALIVE);
}
ctx.writeAndFlush(response);
}
@Override
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
ctx.flush();
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
logger.error(e.getMessage(), e);
ctx.close();
}
/**
* 3轮心跳周期（90s）未发生读写则关闭通道，否则放行
*/
@Override
public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception {
if (evt instanceof IdleStateEvent) {
ctx.channel().close();// beat 3N, close if idle
} else {
super.userEventTriggered(ctx, evt);
}
}
}
/**
* 向调度中心，注册本执行节点的appName、address
*/
public void startRegistry(final String appname, final String address) {
//1、启动 执行节点注册 监控线程，赋值给com.xxl.job.core.thread.ExecutorRegistryThread.registryThread
// 每隔30s执行一次，主要负责向调度中心，注册本执行节点的appName、address
ExecutorRegistryThread.getInstance().start(appname, address);
}
public void stopRegistry() {
ExecutorRegistryThread.getInstance().toStop();
}
}
10）step 10
1、com.xxl.job.core.server.EmbedServer#start内部调用方法com.xxl.job.core.server.EmbedServer#startRegistry，具体代码实现见step 9
2、com.xxl.job.core.server.EmbedServer#startRegistry内部调用方法com.xxl.job.core.thread.ExecutorRegistryThread#start，具体代码实现如下
package com.xxl.job.core.thread;
import *;
public class ExecutorRegistryThread {
private static Logger logger = LoggerFactory.getLogger(ExecutorRegistryThread.class);
private static ExecutorRegistryThread instance = new ExecutorRegistryThread();
public static ExecutorRegistryThread getInstance(){
return instance;
}
private Thread registryThread;
private volatile boolean toStop = false;
public void start(final String appname, final String address){
// 参数校验
if (appname==null || appname.trim().length()==0) {
return;
}
if (XxlJobExecutor.getAdminBizList() == null) {
return;
}
registryThread = new Thread(new Runnable() {
@Override
public void run() {
//1、每隔30s执行一次注册
while (!toStop) {
try {
RegistryParam registryParam = new RegistryParam(RegistryConfig.RegistType.EXECUTOR.name(), appname, address);
for (AdminBiz adminBiz: XxlJobExecutor.getAdminBizList()) {
try {
//2、调用注册方法
ReturnT<String> registryResult = adminBiz.registry(registryParam);
if (registryResult!=null && ReturnT.SUCCESS_CODE == registryResult.getCode()) {//注册成功
registryResult = ReturnT.SUCCESS;
break;
} else {//注册失败
logger.info("xxl-job registry fail"});
}
} catch (Exception e) {//注册异常
logger.info("xxl-job registry error", e);
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
try {
if (!toStop) {
TimeUnit.SECONDS.sleep(RegistryConfig.BEAT_TIMEOUT);// 睡眠30s
}
} catch (InterruptedException e) {
if (!toStop) {
logger.warn("{}", e.getMessage());
}
}
}
//2、移除注册信息
try {
RegistryParam registryParam = new RegistryParam(RegistryConfig.RegistType.EXECUTOR.name(), appname, address);
for (AdminBiz adminBiz: XxlJobExecutor.getAdminBizList()) {
try {
ReturnT<String> registryResult = adminBiz.registryRemove(registryParam);
if (registryResult!=null && ReturnT.SUCCESS_CODE == registryResult.getCode()) {
registryResult = ReturnT.SUCCESS;
logger.info("xxl-job registry-remove success");
break;
} else {
logger.info("xxl-job registry-remove fail");
}
} catch (Exception e) {
if (!toStop) {
logger.info("xxl-job registry-remove error", e);
}
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
});
registryThread.setDaemon(true);
registryThread.setName("xxl-job, executor ExecutorRegistryThread");
registryThread.start();
}
public void toStop() {
toStop = true;
if (registryThread != null) {
registryThread.interrupt();
try {
registryThread.join();
} catch (InterruptedException e) {
logger.error(e.getMessage(), e);
}
}
}
}
2.springboot、frameless执行器区别
（三）关键流程分析
1.节点注册流程
1）step 1
1、执行节点启动时调用方法com.xxl.job.core.thread.ExecutorRegistryThread#start，具体代码参见执行器启动流程的step 10；方法内部 启动 节点注册 监控线程：每30s执行一次，主要负责向调度中心注册本执行节点的appName、address信息
2、com.xxl.job.core.thread.ExecutorRegistryThread#start内部调用com.xxl.job.core.biz.AdminBiz#registry方法，向调度中心发起节点注册rpc请求，具体代码实现如下
package com.xxl.job.core.biz.client;
import *;
public class AdminBizClient implements AdminBiz {
public AdminBizClient() {
}
public AdminBizClient(String addressUrl, String accessToken) {
this.addressUrl = addressUrl;// xxl.job.admin.addresses
this.accessToken = accessToken;// xxl.job.accessToken
if (!this.addressUrl.endsWith("/")) {
this.addressUrl = this.addressUrl + "/";
}
}
private String addressUrl ;
private String accessToken;
private int timeout = 3;
@Override
public ReturnT<String> callback(List<HandleCallbackParam> callbackParamList) {
return XxlJobRemotingUtil.postBody(addressUrl+"api/callback", accessToken, timeout, callbackParamList, String.class);
}
@Override
public ReturnT<String> registry(RegistryParam registryParam) {
return XxlJobRemotingUtil.postBody(addressUrl + "api/registry", accessToken, timeout, registryParam, String.class);
}
@Override
public ReturnT<String> registryRemove(RegistryParam registryParam) {
return XxlJobRemotingUtil.postBody(addressUrl + "api/registryRemove", accessToken, timeout, registryParam, String.class);
}
}
2）step 2
3、对应调度中心请求接收端代码实现如下
package com.xxl.job.admin.controller;
import *;
@Controller
@RequestMapping("/api")
public class JobApiController {
@Resource
private AdminBiz adminBiz;
@RequestMapping("/{uri}")
@ResponseBody
@PermissionLimit(limit=false)
public ReturnT<String> api(HttpServletRequest request, @PathVariable("uri") String uri, @RequestBody(required = false) String data) {
// 参数校验
if (!"POST".equalsIgnoreCase(request.getMethod())) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "invalid request, HttpMethod not support.");
}
if (uri==null || uri.trim().length()==0) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "invalid request, uri-mapping empty.");
}
// accessToken为空 或者 相等，否则不放行
if (XxlJobAdminConfig.getAdminConfig().getAccessToken()!=null
&& XxlJobAdminConfig.getAdminConfig().getAccessToken().trim().length()>0
&& !XxlJobAdminConfig.getAdminConfig().getAccessToken().equals(request.getHeader(XxlJobRemotingUtil.XXL_JOB_ACCESS_TOKEN))) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "The access token is wrong.");
}
if ("callback".equals(uri)) {// 任务回调
List<HandleCallbackParam> callbackParamList = GsonTool.fromJson(data, List.class, HandleCallbackParam.class);
return adminBiz.callback(callbackParamList);
} else if ("registry".equals(uri)) {// 执行节点注册
RegistryParam registryParam = GsonTool.fromJson(data, RegistryParam.class);
return adminBiz.registry(registryParam);
} else if ("registryRemove".equals(uri)) {// 移除执行节点信息
RegistryParam registryParam = GsonTool.fromJson(data, RegistryParam.class);
return adminBiz.registryRemove(registryParam);
} else {
return new ReturnT<String>(ReturnT.FAIL_CODE, "invalid request, uri-mapping("+ uri +") not found.");
}
}
}
3）step 3
4、com.xxl.job.core.biz.AdminBiz#registry对应实现代码如下
package com.xxl.job.admin.service.impl;
import *;
@Service
public class AdminBizImpl implements AdminBiz {
@Override
public ReturnT<String> callback(List<HandleCallbackParam> callbackParamList) {
return JobCompleteHelper.getInstance().callback(callbackParamList);
}
@Override
public ReturnT<String> registry(RegistryParam registryParam) {
return JobRegistryHelper.getInstance().registry(registryParam);
}
@Override
public ReturnT<String> registryRemove(RegistryParam registryParam) {
return JobRegistryHelper.getInstance().registryRemove(registryParam);
}
}
5、由com.xxl.job.admin.core.thread.JobRegistryHelper#registry方法，完成节点信息的新增和更新操作，具体代码参见调度中心启动流程的step 5
6、同时任务节点注册后台监控线程com.xxl.job.admin.core.thread.JobRegistryHelper#registryMonitorThread每30s执行一次，实现 失效执行节点的移除操作 并 刷新所有在线执行节点绑定信息，具体代码参见调度中心启动流程的step 5
2.新增执行器流程
1）step 1
1、操作流程：访问调度中心页面http://localhost:8080/xxl-job-admin -> 执行器管理 -> 新增 -> 保存
2、接口地址：http://127.0.0.1:8080/xxl-job-admin/jobgroup/save
3、接口描述：执行器 新增接口
4、代码说明：对应controller代码实现如下
package com.xxl.job.admin.controller;
import *;
@Controller
@RequestMapping("/jobgroup")
public class JobGroupController {
@Resource
public XxlJobInfoDao xxlJobInfoDao;
@Resource
public XxlJobGroupDao xxlJobGroupDao;
@Resource
private XxlJobRegistryDao xxlJobRegistryDao;
@RequestMapping
@PermissionLimit(adminuser = true)
public String index(Model model) {
return "jobgroup/jobgroup.index";
}
@RequestMapping("/pageList")
@ResponseBody
@PermissionLimit(adminuser = true)
public Map<String, Object> pageList(HttpServletRequest request,
@RequestParam(required = false, defaultValue = "0") int start,
@RequestParam(required = false, defaultValue = "10") int length,
String appname, String title) {
List<XxlJobGroup> list = xxlJobGroupDao.pageList(start, length, appname, title);
int list_count = xxlJobGroupDao.pageListCount(start, length, appname, title);
Map<String, Object> maps = new HashMap<String, Object>();
maps.put("recordsTotal", list_count);
maps.put("recordsFiltered", list_count);
maps.put("data", list);
return maps;
}
/**
* 执行器新增接口
*/
@RequestMapping("/save")
@ResponseBody
@PermissionLimit(adminuser = true)
public ReturnT<String> save(XxlJobGroup xxlJobGroup){
//1、参数校验
if (xxlJobGroup.getAppname()==null || xxlJobGroup.getAppname().trim().length()==0) {
return new ReturnT<String>(500, (I18nUtil.getString("system_please_input")+"AppName") );
}
if (xxlJobGroup.getAppname().length()<4 || xxlJobGroup.getAppname().length()>64) {
return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_appname_length") );
}
if (xxlJobGroup.getAppname().contains(">") || xxlJobGroup.getAppname().contains("<")) {
return new ReturnT<String>(500, "AppName"+I18nUtil.getString("system_unvalid") );
}
if (xxlJobGroup.getTitle()==null || xxlJobGroup.getTitle().trim().length()==0) {
return new ReturnT<String>(500, (I18nUtil.getString("system_please_input") + I18nUtil.getString("jobgroup_field_title")) );
}
if (xxlJobGroup.getTitle().contains(">") || xxlJobGroup.getTitle().contains("<")) {
return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_title")+I18nUtil.getString("system_unvalid") );
}
if (xxlJobGroup.getAddressType()!=0) {
if (xxlJobGroup.getAddressList()==null || xxlJobGroup.getAddressList().trim().length()==0) {
return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_addressType_limit") );
}
if (xxlJobGroup.getAddressList().contains(">") || xxlJobGroup.getAddressList().contains("<")) {
return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_registryList")+I18nUtil.getString("system_unvalid") );
}
String[] addresss = xxlJobGroup.getAddressList().split(",");
for (String item: addresss) {
if (item==null || item.trim().length()==0) {
return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_registryList_unvalid") );
}
}
}
//2、设置更新时间
xxlJobGroup.setUpdateTime(new Date());
//3、存储xxl_job_group记录
int ret = xxlJobGroupDao.save(xxlJobGroup);
return (ret>0)?ReturnT.SUCCESS:ReturnT.FAIL;
}
@RequestMapping("/update")
@ResponseBody
@PermissionLimit(adminuser = true)
public ReturnT<String> update(XxlJobGroup xxlJobGroup){
// valid
if (xxlJobGroup.getAppname()==null || xxlJobGroup.getAppname().trim().length()==0) {
return new ReturnT<String>(500, (I18nUtil.getString("system_please_input")+"AppName") );
}
if (xxlJobGroup.getAppname().length()<4 || xxlJobGroup.getAppname().length()>64) {
return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_appname_length") );
}
if (xxlJobGroup.getTitle()==null || xxlJobGroup.getTitle().trim().length()==0) {
return new ReturnT<String>(500, (I18nUtil.getString("system_please_input") + I18nUtil.getString("jobgroup_field_title")) );
}
if (xxlJobGroup.getAddressType() == 0) {
// 0=自动注册
List<String> registryList = findRegistryByAppName(xxlJobGroup.getAppname());
String addressListStr = null;
if (registryList!=null && !registryList.isEmpty()) {
Collections.sort(registryList);
addressListStr = "";
for (String item:registryList) {
addressListStr += item + ",";
}
addressListStr = addressListStr.substring(0, addressListStr.length()-1);
}
xxlJobGroup.setAddressList(addressListStr);
} else {
// 1=手动录入
if (xxlJobGroup.getAddressList()==null || xxlJobGroup.getAddressList().trim().length()==0) {
return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_addressType_limit") );
}
String[] addresss = xxlJobGroup.getAddressList().split(",");
for (String item: addresss) {
if (item==null || item.trim().length()==0) {
return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_registryList_unvalid") );
}
}
}
xxlJobGroup.setUpdateTime(new Date());
int ret = xxlJobGroupDao.update(xxlJobGroup);
return (ret>0)?ReturnT.SUCCESS:ReturnT.FAIL;
}
private List<String> findRegistryByAppName(String appnameParam){
HashMap<String, List<String>> appAddressMap = new HashMap<String, List<String>>();
List<XxlJobRegistry> list = xxlJobRegistryDao.findAll(RegistryConfig.DEAD_TIMEOUT, new Date());
if (list != null) {
for (XxlJobRegistry item: list) {
if (RegistryConfig.RegistType.EXECUTOR.name().equals(item.getRegistryGroup())) {
String appname = item.getRegistryKey();
List<String> registryList = appAddressMap.get(appname);
if (registryList == null) {
registryList = new ArrayList<String>();
}
if (!registryList.contains(item.getRegistryValue())) {
registryList.add(item.getRegistryValue());
}
appAddressMap.put(appname, registryList);
}
}
}
return appAddressMap.get(appnameParam);
}
@RequestMapping("/remove")
@ResponseBody
@PermissionLimit(adminuser = true)
public ReturnT<String> remove(int id){
// valid
int count = xxlJobInfoDao.pageListCount(0, 10, id, -1, null, null, null);
if (count > 0) {
return new ReturnT<String>(500, I18nUtil.getString("jobgroup_del_limit_0") );
}
List<XxlJobGroup> allList = xxlJobGroupDao.findAll();
if (allList.size() == 1) {
return new ReturnT<String>(500, I18nUtil.getString("jobgroup_del_limit_1") );
}
int ret = xxlJobGroupDao.remove(id);
return (ret>0)?ReturnT.SUCCESS:ReturnT.FAIL;
}
@RequestMapping("/loadById")
@ResponseBody
@PermissionLimit(adminuser = true)
public ReturnT<XxlJobGroup> loadById(int id){
XxlJobGroup jobGroup = xxlJobGroupDao.load(id);
return jobGroup!=null?new ReturnT<XxlJobGroup>(jobGroup):new ReturnT<XxlJobGroup>(ReturnT.FAIL_CODE, null);
}
}
2）step 2
5、调度中心启动的时候，启动任务执行节点注册线程com.xxl.job.admin.core.thread.JobRegistryHelper#registryMonitorThread，完成所有执行器对应执行节点信息更新，具体代码参见调度中心启动流程的step 5
3.新增任务配置流程
1）step 1
1、操作流程：访问调度中心页面http://localhost:8080/xxl-job-admin -> 任务管理 -> 新增 -> 保存
2、接口地址：http://127.0.0.1:8080/xxl-job-admin/jobinfo/add
3、接口描述：任务配置新增接口
4、代码说明：对应controller代码实现如下
package com.xxl.job.admin.controller;
import *;
@Controller
@RequestMapping("/jobinfo")
public class JobInfoController {
private static Logger logger = LoggerFactory.getLogger(JobInfoController.class);
@Resource
private XxlJobGroupDao xxlJobGroupDao;
@Resource
private XxlJobService xxlJobService;
@RequestMapping
public String index(HttpServletRequest request, Model model, @RequestParam(required = false, defaultValue = "-1") int jobGroup) {
// 枚举-字典
model.addAttribute("ExecutorRouteStrategyEnum", ExecutorRouteStrategyEnum.values());// 路由策略-列表
model.addAttribute("GlueTypeEnum", GlueTypeEnum.values());// Glue类型-字典
model.addAttribute("ExecutorBlockStrategyEnum", ExecutorBlockStrategyEnum.values());// 阻塞处理策略-字典
model.addAttribute("ScheduleTypeEnum", ScheduleTypeEnum.values());// 调度类型
model.addAttribute("MisfireStrategyEnum", MisfireStrategyEnum.values());// 调度过期策略
// 执行器列表
List<XxlJobGroup> jobGroupList_all = xxlJobGroupDao.findAll();
List<XxlJobGroup> jobGroupList = filterJobGroupByRole(request, jobGroupList_all);
if (jobGroupList==null || jobGroupList.size()==0) {
throw new XxlJobException(I18nUtil.getString("jobgroup_empty"));
}
model.addAttribute("JobGroupList", jobGroupList);
model.addAttribute("jobGroup", jobGroup);
return "jobinfo/jobinfo.index";
}
public static List<XxlJobGroup> filterJobGroupByRole(HttpServletRequest request, List<XxlJobGroup> jobGroupList_all){
List<XxlJobGroup> jobGroupList = new ArrayList<>();
if (jobGroupList_all!=null && jobGroupList_all.size()>0) {
XxlJobUser loginUser = (XxlJobUser) request.getAttribute(LoginService.LOGIN_IDENTITY_KEY);
if (loginUser.getRole() == 1) {
jobGroupList = jobGroupList_all;
} else {
List<String> groupIdStrs = new ArrayList<>();
if (loginUser.getPermission()!=null && loginUser.getPermission().trim().length()>0) {
groupIdStrs = Arrays.asList(loginUser.getPermission().trim().split(","));
}
for (XxlJobGroup groupItem:jobGroupList_all) {
if (groupIdStrs.contains(String.valueOf(groupItem.getId()))) {
jobGroupList.add(groupItem);
}
}
}
}
return jobGroupList;
}
public static void validPermission(HttpServletRequest request, int jobGroup) {
XxlJobUser loginUser = (XxlJobUser) request.getAttribute(LoginService.LOGIN_IDENTITY_KEY);
if (!loginUser.validPermission(jobGroup)) {
throw new RuntimeException(I18nUtil.getString("system_permission_limit") + "[username="+ loginUser.getUsername() +"]");
}
}
@RequestMapping("/pageList")
@ResponseBody
public Map<String, Object> pageList(@RequestParam(required = false, defaultValue = "0") int start,
@RequestParam(required = false, defaultValue = "10") int length,
int jobGroup, int triggerStatus, String jobDesc, String executorHandler, String author) {
return xxlJobService.pageList(start, length, jobGroup, triggerStatus, jobDesc, executorHandler, author);
}
@RequestMapping("/add")
@ResponseBody
public ReturnT<String> add(XxlJobInfo jobInfo) {
return xxlJobService.add(jobInfo);
}
@RequestMapping("/update")
@ResponseBody
public ReturnT<String> update(XxlJobInfo jobInfo) {
return xxlJobService.update(jobInfo);
}
@RequestMapping("/remove")
@ResponseBody
public ReturnT<String> remove(int id) {
return xxlJobService.remove(id);
}
@RequestMapping("/stop")
@ResponseBody
public ReturnT<String> pause(int id) {
return xxlJobService.stop(id);
}
@RequestMapping("/start")
@ResponseBody
public ReturnT<String> start(int id) {
return xxlJobService.start(id);
}
@RequestMapping("/trigger")
@ResponseBody
//@PermissionLimit(limit = false)
public ReturnT<String> triggerJob(int id, String executorParam, String addressList) {
if (executorParam == null) {
executorParam = "";
}
//任务触发类型：手动触发（TriggerTypeEnum.MANUAL）
JobTriggerPoolHelper.trigger(id, TriggerTypeEnum.MANUAL, -1, null, executorParam, addressList);
return ReturnT.SUCCESS;
}
@RequestMapping("/nextTriggerTime")
@ResponseBody
public ReturnT<List<String>> nextTriggerTime(String scheduleType, String scheduleConf) {
XxlJobInfo paramXxlJobInfo = new XxlJobInfo();
paramXxlJobInfo.setScheduleType(scheduleType);
paramXxlJobInfo.setScheduleConf(scheduleConf);
List<String> result = new ArrayList<>();
try {
Date lastTime = new Date();
for (int i = 0; i < 5; i++) {
lastTime = JobScheduleHelper.generateNextValidTime(paramXxlJobInfo, lastTime);
if (lastTime != null) {
result.add(DateUtil.formatDateTime(lastTime));
} else {
break;
}
}
} catch (Exception e) {
logger.error(e.getMessage(), e);
return new ReturnT<List<String>>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) + e.getMessage());
}
return new ReturnT<List<String>>(result);
}
}
2）step 2
5、com.xxl.job.admin.service.XxlJobService#add方法，具体代码实现如下
package com.xxl.job.admin.service.impl;
import *;
@Service
public class XxlJobServiceImpl implements XxlJobService {
private static Logger logger = LoggerFactory.getLogger(XxlJobServiceImpl.class);
@Resource
private XxlJobGroupDao xxlJobGroupDao;
@Resource
private XxlJobInfoDao xxlJobInfoDao;
@Resource
public XxlJobLogDao xxlJobLogDao;
@Resource
private XxlJobLogGlueDao xxlJobLogGlueDao;
@Resource
private XxlJobLogReportDao xxlJobLogReportDao;
@Override
public Map<String, Object> pageList(int start, int length, int jobGroup, int triggerStatus, String jobDesc, String executorHandler, String author) {
List<XxlJobInfo> list = xxlJobInfoDao.pageList(start, length, jobGroup, triggerStatus, jobDesc, executorHandler, author);
int list_count = xxlJobInfoDao.pageListCount(start, length, jobGroup, triggerStatus, jobDesc, executorHandler, author);
Map<String, Object> maps = new HashMap<String, Object>();
maps.put("recordsTotal", list_count);
maps.put("recordsFiltered", list_count);
maps.put("data", list);
return maps;
}
@Override
public ReturnT<String> add(XxlJobInfo jobInfo) {
//1、基础 参数 校验
XxlJobGroup group = xxlJobGroupDao.load(jobInfo.getJobGroup());
if (group == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("system_please_choose")+I18nUtil.getString("jobinfo_field_jobgroup")) );
}
if (jobInfo.getJobDesc()==null || jobInfo.getJobDesc().trim().length()==0) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("system_please_input")+I18nUtil.getString("jobinfo_field_jobdesc")) );
}
if (jobInfo.getAuthor()==null || jobInfo.getAuthor().trim().length()==0) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("system_please_input")+I18nUtil.getString("jobinfo_field_author")) );
}
//2、任务调度 类型 校验
ScheduleTypeEnum scheduleTypeEnum = ScheduleTypeEnum.match(jobInfo.getScheduleType(), null);
if (scheduleTypeEnum == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) );
}
if (scheduleTypeEnum == ScheduleTypeEnum.CRON) {
if (jobInfo.getScheduleConf()==null || !CronExpression.isValidExpression(jobInfo.getScheduleConf())) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "Cron"+I18nUtil.getString("system_unvalid"));
}
} else if (scheduleTypeEnum == ScheduleTypeEnum.FIX_RATE/* || scheduleTypeEnum == ScheduleTypeEnum.FIX_DELAY*/) {
if (jobInfo.getScheduleConf() == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")) );
}
try {
int fixSecond = Integer.valueOf(jobInfo.getScheduleConf());
if (fixSecond < 1) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) );
}
} catch (Exception e) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) );
}
}
//3、运行模式 类型 校验
if (GlueTypeEnum.match(jobInfo.getGlueType()) == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("jobinfo_field_gluetype")+I18nUtil.getString("system_unvalid")) );
}
if (GlueTypeEnum.BEAN==GlueTypeEnum.match(jobInfo.getGlueType()) && (jobInfo.getExecutorHandler()==null || jobInfo.getExecutorHandler().trim().length()==0) ) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("system_please_input")+"JobHandler") );
}
// 》fix "\r" in shell
if (GlueTypeEnum.GLUE_SHELL==GlueTypeEnum.match(jobInfo.getGlueType()) && jobInfo.getGlueSource()!=null) {
jobInfo.setGlueSource(jobInfo.getGlueSource().replaceAll("\r", ""));
}
//4、路由策略 类型 校验
if (ExecutorRouteStrategyEnum.match(jobInfo.getExecutorRouteStrategy(), null) == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("jobinfo_field_executorRouteStrategy")+I18nUtil.getString("system_unvalid")) );
}
if (MisfireStrategyEnum.match(jobInfo.getMisfireStrategy(), null) == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("misfire_strategy")+I18nUtil.getString("system_unvalid")) );
}
if (ExecutorBlockStrategyEnum.match(jobInfo.getExecutorBlockStrategy(), null) == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("jobinfo_field_executorBlockStrategy")+I18nUtil.getString("system_unvalid")) );
}
//5、子任务校验
if (jobInfo.getChildJobId()!=null && jobInfo.getChildJobId().trim().length()>0) {
String[] childJobIds = jobInfo.getChildJobId().split(",");
for (String childJobIdItem: childJobIds) {
if (childJobIdItem!=null && childJobIdItem.trim().length()>0 && isNumeric(childJobIdItem)) {
//6、子任务必须存在
XxlJobInfo childJobInfo = xxlJobInfoDao.loadById(Integer.parseInt(childJobIdItem));
if (childJobInfo==null) {
return new ReturnT<String>(ReturnT.FAIL_CODE,
MessageFormat.format((I18nUtil.getString("jobinfo_field_childJobId")+"({0})"+I18nUtil.getString("system_not_found")), childJobIdItem));
}
} else {//子任务格式校验不正确
return new ReturnT<String>(ReturnT.FAIL_CODE,
MessageFormat.format((I18nUtil.getString("jobinfo_field_childJobId")+"({0})"+I18nUtil.getString("system_unvalid")), childJobIdItem));
}
}
// 拼接子任务id
String temp = "";
for (String item:childJobIds) {
temp += item + ",";
}
temp = temp.substring(0, temp.length()-1);
jobInfo.setChildJobId(temp);
}
//6、新增xxl_job_info表记录
jobInfo.setAddTime(new Date());
jobInfo.setUpdateTime(new Date());
jobInfo.setGlueUpdatetime(new Date());
xxlJobInfoDao.save(jobInfo);
if (jobInfo.getId() < 1) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("jobinfo_field_add")+I18nUtil.getString("system_fail")) );
}
//7、返回记录id
return new ReturnT<String>(String.valueOf(jobInfo.getId()));
}
private boolean isNumeric(String str){
try {
int result = Integer.valueOf(str);
return true;
} catch (NumberFormatException e) {
return false;
}
}
@Override
public ReturnT<String> update(XxlJobInfo jobInfo) {
if (jobInfo.getJobDesc()==null || jobInfo.getJobDesc().trim().length()==0) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("system_please_input")+I18nUtil.getString("jobinfo_field_jobdesc")) );
}
if (jobInfo.getAuthor()==null || jobInfo.getAuthor().trim().length()==0) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("system_please_input")+I18nUtil.getString("jobinfo_field_author")) );
}
ScheduleTypeEnum scheduleTypeEnum = ScheduleTypeEnum.match(jobInfo.getScheduleType(), null);
if (scheduleTypeEnum == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) );
}
if (scheduleTypeEnum == ScheduleTypeEnum.CRON) {
if (jobInfo.getScheduleConf()==null || !CronExpression.isValidExpression(jobInfo.getScheduleConf())) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "Cron"+I18nUtil.getString("system_unvalid") );
}
} else if (scheduleTypeEnum == ScheduleTypeEnum.FIX_RATE) {
if (jobInfo.getScheduleConf() == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) );
}
try {
int fixSecond = Integer.valueOf(jobInfo.getScheduleConf());
if (fixSecond < 1) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) );
}
} catch (Exception e) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) );
}
}
if (ExecutorRouteStrategyEnum.match(jobInfo.getExecutorRouteStrategy(), null) == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("jobinfo_field_executorRouteStrategy")+I18nUtil.getString("system_unvalid")) );
}
if (MisfireStrategyEnum.match(jobInfo.getMisfireStrategy(), null) == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("misfire_strategy")+I18nUtil.getString("system_unvalid")) );
}
if (ExecutorBlockStrategyEnum.match(jobInfo.getExecutorBlockStrategy(), null) == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("jobinfo_field_executorBlockStrategy")+I18nUtil.getString("system_unvalid")) );
}
if (jobInfo.getChildJobId()!=null && jobInfo.getChildJobId().trim().length()>0) {
String[] childJobIds = jobInfo.getChildJobId().split(",");
for (String childJobIdItem: childJobIds) {
if (childJobIdItem!=null && childJobIdItem.trim().length()>0 && isNumeric(childJobIdItem)) {
XxlJobInfo childJobInfo = xxlJobInfoDao.loadById(Integer.parseInt(childJobIdItem));
if (childJobInfo==null) {
return new ReturnT<String>(ReturnT.FAIL_CODE,
MessageFormat.format((I18nUtil.getString("jobinfo_field_childJobId")+"({0})"+I18nUtil.getString("system_not_found")), childJobIdItem));
}
} else {
return new ReturnT<String>(ReturnT.FAIL_CODE,
MessageFormat.format((I18nUtil.getString("jobinfo_field_childJobId")+"({0})"+I18nUtil.getString("system_unvalid")), childJobIdItem));
}
}
String temp = "";
for (String item:childJobIds) {
temp += item + ",";
}
temp = temp.substring(0, temp.length()-1);
jobInfo.setChildJobId(temp);
}
// group valid
XxlJobGroup jobGroup = xxlJobGroupDao.load(jobInfo.getJobGroup());
if (jobGroup == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("jobinfo_field_jobgroup")+I18nUtil.getString("system_unvalid")) );
}
XxlJobInfo exists_jobInfo = xxlJobInfoDao.loadById(jobInfo.getId());
if (exists_jobInfo == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("jobinfo_field_id")+I18nUtil.getString("system_not_found")) );
}
// next trigger time (5s后生效，避开预读周期)
long nextTriggerTime = exists_jobInfo.getTriggerNextTime();
boolean scheduleDataNotChanged = jobInfo.getScheduleType().equals(exists_jobInfo.getScheduleType()) && jobInfo.getScheduleConf().equals(exists_jobInfo.getScheduleConf());
if (exists_jobInfo.getTriggerStatus() == 1 && !scheduleDataNotChanged) {
try {
Date nextValidTime = JobScheduleHelper.generateNextValidTime(jobInfo, new Date(System.currentTimeMillis() + JobScheduleHelper.PRE_READ_MS));
if (nextValidTime == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) );
}
nextTriggerTime = nextValidTime.getTime();
} catch (Exception e) {
logger.error(e.getMessage(), e);
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) );
}
}
exists_jobInfo.setJobGroup(jobInfo.getJobGroup());
exists_jobInfo.setJobDesc(jobInfo.getJobDesc());
exists_jobInfo.setAuthor(jobInfo.getAuthor());
exists_jobInfo.setAlarmEmail(jobInfo.getAlarmEmail());
exists_jobInfo.setScheduleType(jobInfo.getScheduleType());
exists_jobInfo.setScheduleConf(jobInfo.getScheduleConf());
exists_jobInfo.setMisfireStrategy(jobInfo.getMisfireStrategy());
exists_jobInfo.setExecutorRouteStrategy(jobInfo.getExecutorRouteStrategy());
exists_jobInfo.setExecutorHandler(jobInfo.getExecutorHandler());
exists_jobInfo.setExecutorParam(jobInfo.getExecutorParam());
exists_jobInfo.setExecutorBlockStrategy(jobInfo.getExecutorBlockStrategy());
exists_jobInfo.setExecutorTimeout(jobInfo.getExecutorTimeout());
exists_jobInfo.setExecutorFailRetryCount(jobInfo.getExecutorFailRetryCount());
exists_jobInfo.setChildJobId(jobInfo.getChildJobId());
exists_jobInfo.setTriggerNextTime(nextTriggerTime);
exists_jobInfo.setUpdateTime(new Date());
xxlJobInfoDao.update(exists_jobInfo);
return ReturnT.SUCCESS;
}
@Override
public ReturnT<String> remove(int id) {
XxlJobInfo xxlJobInfo = xxlJobInfoDao.loadById(id);
if (xxlJobInfo == null) {
return ReturnT.SUCCESS;
}
xxlJobInfoDao.delete(id);
xxlJobLogDao.delete(id);
xxlJobLogGlueDao.deleteByJobId(id);
return ReturnT.SUCCESS;
}
@Override
public ReturnT<String> start(int id) {
//加载执行器 任务配置 信息
XxlJobInfo xxlJobInfo = xxlJobInfoDao.loadById(id);
//获取调度类型，默认值为无
ScheduleTypeEnum scheduleTypeEnum = ScheduleTypeEnum.match(xxlJobInfo.getScheduleType(), ScheduleTypeEnum.NONE);
if (ScheduleTypeEnum.NONE == scheduleTypeEnum) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type_none_limit_start")) );
}
// next trigger time (5s后生效，避开预读周期)
long nextTriggerTime = 0;
try {
//生成下次执行时间
Date nextValidTime = JobScheduleHelper.generateNextValidTime(xxlJobInfo, new Date(System.currentTimeMillis() + JobScheduleHelper.PRE_READ_MS));
if (nextValidTime == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) );
}
nextTriggerTime = nextValidTime.getTime();
} catch (Exception e) {
logger.error(e.getMessage(), e);
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_unvalid")) );
}
//更新xxl_job_info表记录
xxlJobInfo.setTriggerStatus(1);//调度状态：0-停止，1-运行
xxlJobInfo.setTriggerLastTime(0);//上次调度时间
xxlJobInfo.setTriggerNextTime(nextTriggerTime);
xxlJobInfo.setUpdateTime(new Date());//下次调度时间
xxlJobInfoDao.update(xxlJobInfo);
return ReturnT.SUCCESS;
}
@Override
public ReturnT<String> stop(int id) {
XxlJobInfo xxlJobInfo = xxlJobInfoDao.loadById(id);
xxlJobInfo.setTriggerStatus(0);
xxlJobInfo.setTriggerLastTime(0);
xxlJobInfo.setTriggerNextTime(0);
xxlJobInfo.setUpdateTime(new Date());
xxlJobInfoDao.update(xxlJobInfo);
return ReturnT.SUCCESS;
}
@Override
public Map<String, Object> dashboardInfo() {
int jobInfoCount = xxlJobInfoDao.findAllCount();
int jobLogCount = 0;
int jobLogSuccessCount = 0;
XxlJobLogReport xxlJobLogReport = xxlJobLogReportDao.queryLogReportTotal();
if (xxlJobLogReport != null) {
jobLogCount = xxlJobLogReport.getRunningCount() + xxlJobLogReport.getSucCount() + xxlJobLogReport.getFailCount();
jobLogSuccessCount = xxlJobLogReport.getSucCount();
}
Set<String> executorAddressSet = new HashSet<String>();
List<XxlJobGroup> groupList = xxlJobGroupDao.findAll();
if (groupList!=null && !groupList.isEmpty()) {
for (XxlJobGroup group: groupList) {
if (group.getRegistryList()!=null && !group.getRegistryList().isEmpty()) {
executorAddressSet.addAll(group.getRegistryList());
}
}
}
int executorCount = executorAddressSet.size();
Map<String, Object> dashboardMap = new HashMap<String, Object>();
dashboardMap.put("jobInfoCount", jobInfoCount);
dashboardMap.put("jobLogCount", jobLogCount);
dashboardMap.put("jobLogSuccessCount", jobLogSuccessCount);
dashboardMap.put("executorCount", executorCount);
return dashboardMap;
}
@Override
public ReturnT<Map<String, Object>> chartInfo(Date startDate, Date endDate) {
List<String> triggerDayList = new ArrayList<String>();
List<Integer> triggerDayCountRunningList = new ArrayList<Integer>();
List<Integer> triggerDayCountSucList = new ArrayList<Integer>();
List<Integer> triggerDayCountFailList = new ArrayList<Integer>();
int triggerCountRunningTotal = 0;
int triggerCountSucTotal = 0;
int triggerCountFailTotal = 0;
List<XxlJobLogReport> logReportList = xxlJobLogReportDao.queryLogReport(startDate, endDate);
if (logReportList!=null && logReportList.size()>0) {
for (XxlJobLogReport item: logReportList) {
String day = DateUtil.formatDate(item.getTriggerDay());
int triggerDayCountRunning = item.getRunningCount();
int triggerDayCountSuc = item.getSucCount();
int triggerDayCountFail = item.getFailCount();
triggerDayList.add(day);
triggerDayCountRunningList.add(triggerDayCountRunning);
triggerDayCountSucList.add(triggerDayCountSuc);
triggerDayCountFailList.add(triggerDayCountFail);
triggerCountRunningTotal += triggerDayCountRunning;
triggerCountSucTotal += triggerDayCountSuc;
triggerCountFailTotal += triggerDayCountFail;
}
} else {
for (int i = -6; i <= 0; i++) {
triggerDayList.add(DateUtil.formatDate(DateUtil.addDays(new Date(), i)));
triggerDayCountRunningList.add(0);
triggerDayCountSucList.add(0);
triggerDayCountFailList.add(0);
}
}
Map<String, Object> result = new HashMap<String, Object>();
result.put("triggerDayList", triggerDayList);
result.put("triggerDayCountRunningList", triggerDayCountRunningList);
result.put("triggerDayCountSucList", triggerDayCountSucList);
result.put("triggerDayCountFailList", triggerDayCountFailList);
result.put("triggerCountRunningTotal", triggerCountRunningTotal);
result.put("triggerCountSucTotal", triggerCountSucTotal);
result.put("triggerCountFailTotal", triggerCountFailTotal);
return new ReturnT<Map<String, Object>>(result);
}
}
4.执行一次任务流程
1）step 1
1、操作流程：访问调度中心页面http://localhost:8080/xxl-job-admin -> 任务管理 -> 选择某条任务配置记录 -> 操作 -> 执行一次
2、接口地址：http://127.0.0.1:8080/xxl-job-admin/jobinfo/trigger
3、接口描述：执行一次任务接口
4、代码说明：对应controller具体代码参见新增任务配置流程 step 1
2）step 2
5、controller内部调用com.xxl.job.admin.core.thread.JobTriggerPoolHelper#trigger实现立即触发一次任务执行，具体代码参见 调度中心启动流程 step 4
3）step 3
5、com.xxl.job.admin.core.thread.JobTriggerPoolHelper#trigger内部继续调用方法com.xxl.job.admin.core.thread.JobTriggerPoolHelper#addTrigger，其实现逻辑为：
① 根据当前任务jobId，去分钟内任务调度超时次数统计变量jobTimeoutCountMap中取值，判断使用那个任务调度线程池；默认使用快任务调度线程池fastTriggerPool，仅当值大于10时使用慢任务调度线程池slowTriggerPool
② 线程池异步调用，触发任务执行
③ 记录任务开始执行时间
④ 调用方法com.xxl.job.admin.core.trigger.XxlJobTrigger#trigger，触发任务执行，内部实现流程分析参见执行一次任务流程 step 4
⑤ 计算当前时间分钟刻度minTim_now，并与全局的分钟刻度com.xxl.job.admin.core.thread.JobTriggerPoolHelper#minTim做比对。仅当不一致时，刷新全局分钟刻度的值；并清空分钟内任务调度超时次数统计变量jobTimeoutCountMap的值
⑥ 结合步骤③的值，计算任务执行消耗时长。仅当耗时大于500ms时，分钟内任务调度超时次数统计变量jobTimeoutCountMap对应当前任务jobId的值，自动+1
具体代码参见 调度中心启动流程 step 4
4）step 4
6、方法com.xxl.job.admin.core.trigger.XxlJobTrigger#trigger，其实现逻辑为：
① 根据入参jobId，去数据库加载任务配置信息XxlJobInfo记录，查询结果为空则直接返回
② 入参executorParam不为空，则覆盖XxlJobInfo记录的执行器任务参数executorParam字段
③ 根据入参failRetryCount，获取当前任务失败重试次数finalFailRetryCount；默认值为XxlJobInfo记录的executorFailRetryCount，仅当failRetryCount大于等于0时值为failRetryCount
④ 入参addressList不为空，则覆盖XxlJobInfo记录的执行器地址列表addressList字段，并设置执行器地址类型addressType字段为1（0=自动注册、1=手动录入）
注：即优先外部传入执行器地址，也即优先手动录入执行器地址
⑤ 入参executorShardingParam不为空时，将分片参数按照/符号拆分为index，total两条数据并装入临时变量shardingParam，便于后续方法引用
⑥ 根据XxlJobInfo记录的执行器路由策略类型executorRouteStrategy字段，分情况调用方法com.xxl.job.admin.core.trigger.XxlJobTrigger#processTrigger实现指定节点执行器的任务调度：
1）当路由策略 为 分片广播 且 XxlJobInfo记录的执行器地址列表registryList字段不为空时，循环registryList进行com.xxl.job.admin.core.trigger.XxlJobTrigger#processTrigger调用，内部实现流程分析参见 执行一次任务流程 step 5
2）当路由策略 为 其他情况时，此时若shardingParam为空则赋初值{0, 1}；然后继续调用com.xxl.job.admin.core.trigger.XxlJobTrigger#processTrigger方法，内部实现流程分析参见 执行一次任务流程 step 5
具体代码参见 调度中心启动流程 step 4
5）step 5
7、方法com.xxl.job.admin.core.trigger.XxlJobTrigger#processTrigger，其实现逻辑为：
① 根据入参XxlJobInfo记录阻塞处理策略executorBlockStrategy字段，获取对应的阻塞处理策略枚举，枚举默认值为单机串行
②
③
④
⑤
⑥
⑦
⑧
⑨
⑩
具体代码参见 调度中心启动流程 step 4
6）step 6
8、com.xxl.job.core.server.EmbedServer#thread内部调用com.xxl.job.core.biz.impl.ExecutorBizImpl#run方法，具体代码实现如下
package com.xxl.job.core.biz.impl;
import *;
public class ExecutorBizImpl implements ExecutorBiz {
private static Logger logger = LoggerFactory.getLogger(ExecutorBizImpl.class);
@Override
public ReturnT<String> beat() {
return ReturnT.SUCCESS;
}
@Override
public ReturnT<String> idleBeat(IdleBeatParam idleBeatParam) {
boolean isRunningOrHasQueue = false;
JobThread jobThread = XxlJobExecutor.loadJobThread(idleBeatParam.getJobId());
if (jobThread != null && jobThread.isRunningOrHasQueue()) {
isRunningOrHasQueue = true;
}
if (isRunningOrHasQueue) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "job thread is running or has trigger queue.");
}
return ReturnT.SUCCESS;
}
@Override
public ReturnT<String> run(TriggerParam triggerParam) {
//1、从com.xxl.job.core.executor.XxlJobExecutor.jobThreadRepository缓存中获取 任务执行线程，首次为空
JobThread jobThread = XxlJobExecutor.loadJobThread(triggerParam.getJobId());
//2、获取IJobHandler信息（Ctrl+H），首次为空
IJobHandler jobHandler = jobThread!=null?jobThread.getHandler():null;
String removeOldReason = null;
//3、获取 运行模式 类型
GlueTypeEnum glueTypeEnum = GlueTypeEnum.match(triggerParam.getGlueType());
if (GlueTypeEnum.BEAN == glueTypeEnum) {//BEAN模式
//获取@XxlJob注解的配置信息对象，IJobHandler
IJobHandler newJobHandler = XxlJobExecutor.loadJobHandler(triggerParam.getExecutorHandler());
//参数校验：缓存信息存在情况下 缓存信息 和 节点注册 的 不一致（？？）
if (jobThread!=null && jobHandler != newJobHandler) {
removeOldReason = "change jobhandler or glue type, and terminate the old job thread.";
jobThread = null;
jobHandler = null;
}
//jobHandler首次为空，则将加载到的newJobHandler赋值给jobHandler
if (jobHandler == null) {
jobHandler = newJobHandler;
//如果本执行节点未找到newJobHandler信息，则直接返回执行失败
if (jobHandler == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "job handler [" + triggerParam.getExecutorHandler() + "] not found.");
}
}
} else if (GlueTypeEnum.GLUE_GROOVY == glueTypeEnum) {//GLUE模式(Java)
//参数校验：缓存信息存在情况下 缓存信息所属类型 不为GlueJobHandler 并且 更新时间与传入的参数 一致
if (jobThread != null &&
!(jobThread.getHandler() instanceof GlueJobHandler
&& ((GlueJobHandler) jobThread.getHandler()).getGlueUpdatetime()==triggerParam.getGlueUpdatetime() )) {
removeOldReason = "change job source or glue type, and terminate the old job thread.";
jobThread = null;
jobHandler = null;
}
if (jobHandler == null) {
try {
IJobHandler originJobHandler = GlueFactory.getInstance().loadNewInstance(triggerParam.getGlueSource());
jobHandler = new GlueJobHandler(originJobHandler, triggerParam.getGlueUpdatetime());
} catch (Exception e) {
logger.error(e.getMessage(), e);
return new ReturnT<String>(ReturnT.FAIL_CODE, e.getMessage());
}
}
} else if (glueTypeEnum!=null && glueTypeEnum.isScript()) {//运行模式 类型 为系统指定的脚本语言
//参数校验：缓存信息存在情况下 缓存信息所属类型 不为ScriptJobHandler 并且 更新时间与传入的参数 一致
if (jobThread != null &&
!(jobThread.getHandler() instanceof ScriptJobHandler
&& ((ScriptJobHandler) jobThread.getHandler()).getGlueUpdatetime()==triggerParam.getGlueUpdatetime() )) {
removeOldReason = "change job source or glue type, and terminate the old job thread.";
jobThread = null;
jobHandler = null;
}
if (jobHandler == null) {
jobHandler = new ScriptJobHandler(triggerParam.getJobId(), triggerParam.getGlueUpdatetime(), triggerParam.getGlueSource(), GlueTypeEnum.match(triggerParam.getGlueType()));
}
} else {//运行模式类型 不存在
//则直接返回执行失败
return new ReturnT<String>(ReturnT.FAIL_CODE, "glueType[" + triggerParam.getGlueType() + "] is not valid.");
}
//4、获取 阻塞策略 类型
if (jobThread != null) {
ExecutorBlockStrategyEnum blockStrategy = ExecutorBlockStrategyEnum.match(triggerParam.getExecutorBlockStrategy(), null);
if (ExecutorBlockStrategyEnum.DISCARD_LATER == blockStrategy) {//丢弃后续调度
if (jobThread.isRunningOrHasQueue()) {
// 如果运行中或者任务队列不为空 则直接返回
return new ReturnT<String>(ReturnT.FAIL_CODE, "block strategy effect："+ExecutorBlockStrategyEnum.DISCARD_LATER.getTitle());
}
} else if (ExecutorBlockStrategyEnum.COVER_EARLY == blockStrategy) {//覆盖之前调度
if (jobThread.isRunningOrHasQueue()) {
removeOldReason = "block strategy effect：" + ExecutorBlockStrategyEnum.COVER_EARLY.getTitle();
jobThread = null;
}
} else {// 单机串行 获取 未设置
// just queue trigger
}
}
//新建任务线程JobThread实例并启动（缓存到com.xxl.job.core.executor.XxlJobExecutor.jobThreadRepository），并且停止旧的线程实例
if (jobThread == null) {
jobThread = XxlJobExecutor.registJobThread(triggerParam.getJobId(), jobHandler, removeOldReason);
}
// 推送数据到队列中
ReturnT<String> pushResult = jobThread.pushTriggerQueue(triggerParam);
return pushResult;
}
@Override
public ReturnT<String> kill(KillParam killParam) {
JobThread jobThread = XxlJobExecutor.loadJobThread(killParam.getJobId());
if (jobThread != null) {
XxlJobExecutor.removeJobThread(killParam.getJobId(), "scheduling center kill job.");
return ReturnT.SUCCESS;
}
return new ReturnT<String>(ReturnT.SUCCESS_CODE, "job thread already killed.");
}
@Override
public ReturnT<LogResult> log(LogParam logParam) {
String logFileName = XxlJobFileAppender.makeLogFileName(new Date(logParam.getLogDateTim()), logParam.getLogId());
LogResult logResult = XxlJobFileAppender.readLog(logFileName, logParam.getFromLineNum());
return new ReturnT<LogResult>(logResult);
}
}
7、com.xxl.job.core.biz.impl.ExecutorBizImpl#run内部调用方法com.xxl.job.core.executor.XxlJobExecutor#registJobThread，具体代码参见 执行器启动流程 step 4
7）step 7
8、com.xxl.job.core.executor.XxlJobExecutor#registJobThread内部调用方法com.xxl.job.core.thread.JobThread#start方法，具体代码实现如下
package com.xxl.job.core.thread;
import *;
public class JobThread extends Thread{
private static Logger logger = LoggerFactory.getLogger(JobThread.class);
private int jobId;
private IJobHandler handler;
private LinkedBlockingQueue<TriggerParam> triggerQueue;//任务队列
private Set<Long> triggerLogIdSet;//避免对同一个TRIGGER_LOG_ID重复触发
private volatile boolean toStop = false;
private String stopReason;
private boolean running = false;//是否运行中
private int idleTimes = 0;//闲置次数
public JobThread(int jobId, IJobHandler handler) {
this.jobId = jobId;
this.handler = handler;
this.triggerQueue = new LinkedBlockingQueue<TriggerParam>();
this.triggerLogIdSet = Collections.synchronizedSet(new HashSet<Long>());
// assign job thread name
this.setName("xxl-job, JobThread-"+jobId+"-"+System.currentTimeMillis());
}
public IJobHandler getHandler() {
return handler;
}
/**
* 推送数据至任务队列中
*/
public ReturnT<String> pushTriggerQueue(TriggerParam triggerParam) {
//避免任务重复执行
if (triggerLogIdSet.contains(triggerParam.getLogId())) {
logger.info(">>>>>>>>>>> repeate trigger job, logId:{}", triggerParam.getLogId());
return new ReturnT<String>(ReturnT.FAIL_CODE, "repeate trigger job, logId:" + triggerParam.getLogId());
}
//数据加入任务队列中
triggerLogIdSet.add(triggerParam.getLogId());
triggerQueue.add(triggerParam);
return ReturnT.SUCCESS;
}
public void toStop(String stopReason) {
/**
* Thread.interrupt只支持终止线程的阻塞状态(wait、join、sleep)，
* 在阻塞出抛出InterruptedException异常,但是并不会终止运行的线程本身；
* 所以需要注意，此处彻底销毁本线程，需要通过共享变量方式；
*/
this.toStop = true;
this.stopReason = stopReason;
}
/**
* 是否运行中 或者 任务队列不为空
*/
public boolean isRunningOrHasQueue() {
return running || triggerQueue.size()>0;
}
@Override
public void run() {
//1、执行初始化方法
try {
handler.init();
} catch (Throwable e) {
logger.error(e.getMessage(), e);
}
//2、执行任务
while(!toStop){
running = false;//重置running状态
idleTimes++;//闲置次数+1
TriggerParam triggerParam = null;
try {
//要检查toStop信号，我们需要循环，所以我们使用poll(timeout)，不能使用queue.take()
//poll(timeout)删除队首元素，当队列为空的时候，线程休眠一定时间。休眠时，当前线程被中断抛出InterruptedException异常。
triggerParam = triggerQueue.poll(3L, TimeUnit.SECONDS);
if (triggerParam!=null) {
running = true;//设置running状态为运行中
idleTimes = 0;//重置闲置次数
triggerLogIdSet.remove(triggerParam.getLogId());//移除logId
//获取日志文件 路径名称（目录不存在则先创建目录） log filename, like "logPath/yyyy-MM-dd/9999.log"
String logFileName = XxlJobFileAppender.makeLogFileName(new Date(triggerParam.getLogDateTime()), triggerParam.getLogId());
//线程间传递XxlJobContext对象（日志文件 路径名称）
XxlJobContext xxlJobContext = new XxlJobContext(
triggerParam.getJobId(),
triggerParam.getExecutorParams(),
logFileName,
triggerParam.getBroadcastIndex(),
triggerParam.getBroadcastTotal());
XxlJobContext.setXxlJobContext(xxlJobContext);
//记录日志
XxlJobHelper.log("<br>----------- xxl-job job execute start -----------<br>----------- Param:" + xxlJobContext.getJobParam());
if (triggerParam.getExecutorTimeout() > 0) {
//设置了执行超时时间
Thread futureThread = null;
try {
FutureTask<Boolean> futureTask = new FutureTask<Boolean>(new Callable<Boolean>() {
@Override
public Boolean call() throws Exception {
//设置xxlJobContext变量
XxlJobContext.setXxlJobContext(xxlJobContext);
//执行任务方法
handler.execute();
return true;
}
});
futureThread = new Thread(futureTask);
futureThread.start();
//线程同步等待，超时则返回TimeoutException
Boolean tempResult = futureTask.get(triggerParam.getExecutorTimeout(), TimeUnit.SECONDS);
} catch (TimeoutException e) {
//记录任务超时日志
XxlJobHelper.log("<br>----------- xxl-job job execute timeout");
XxlJobHelper.log(e);
//超时处理（带日志）
XxlJobHelper.handleTimeout("job execute timeout ");
} finally {
//中断本线程
futureThread.interrupt();
}
} else {
//执行任务方法
handler.execute();
}
if (XxlJobContext.getXxlJobContext().getHandleCode() <= 0) {//handleCode小于等于0
XxlJobHelper.handleFail("job handle result lost.");//任务处理结果丢失
} else {
//重设处理消息（超长则直接截取）
String tempHandleMsg = XxlJobContext.getXxlJobContext().getHandleMsg();
tempHandleMsg = (tempHandleMsg!=null&&tempHandleMsg.length()>50000)
?tempHandleMsg.substring(0, 50000).concat("...")
:tempHandleMsg;
XxlJobContext.getXxlJobContext().setHandleMsg(tempHandleMsg);
}
//记录日志
XxlJobHelper.log("<br>----------- xxl-job job execute end(finish) -----------<br>----------- Result: handleCode="
+ XxlJobContext.getXxlJobContext().getHandleCode()
+ ", handleMsg = "
+ XxlJobContext.getXxlJobContext().getHandleMsg()
);
} else {
// 闲置次数超过30次，即30次没有取到数据
if (idleTimes > 30) {
if(triggerQueue.size() == 0) {// 避免并发触发导致jobId丢失
//从com.xxl.job.core.executor.XxlJobExecutor.jobThreadRepository缓存中移除JobThread，并且停止线程
XxlJobExecutor.removeJobThread(jobId, "excutor idel times over limit.");
}
}
}
} catch (Throwable e) {
if (toStop) {
XxlJobHelper.log("<br>----------- JobThread toStop, stopReason:" + stopReason);
}
//获取堆栈信息，并存入XxlJobContext变量
StringWriter stringWriter = new StringWriter();
e.printStackTrace(new PrintWriter(stringWriter));
String errorMsg = stringWriter.toString();
XxlJobHelper.handleFail(errorMsg);
//异常记录日志
XxlJobHelper.log("<br>----------- JobThread Exception:" + errorMsg + "<br>----------- xxl-job job execute end(error) -----------");
} finally {
if(triggerParam != null) {
if (!toStop) {
//执行回调，向com.xxl.job.core.thread.TriggerCallbackThread.callBackQueue队列存入数据
TriggerCallbackThread.pushCallBack(new HandleCallbackParam(
triggerParam.getLogId(),
triggerParam.getLogDateTime(),
XxlJobContext.getXxlJobContext().getHandleCode(),
XxlJobContext.getXxlJobContext().getHandleMsg() )
);
} else {
//toStop is true, is killed
//执行当前任务回调，向com.xxl.job.core.thread.TriggerCallbackThread.callBackQueue队列存入数据
TriggerCallbackThread.pushCallBack(new HandleCallbackParam(
triggerParam.getLogId(),
triggerParam.getLogDateTime(),
XxlJobContext.HANDLE_CODE_FAIL,
stopReason + " [job running, killed]" )
);
}
}
}
}
//任务队列不为空(触发剩余任务回调)
while(triggerQueue !=null && triggerQueue.size()>0){
//poll()方法用来获取并移除队列头部元素的方法，如果队列为空，则返回null。
TriggerParam triggerParam = triggerQueue.poll();
if (triggerParam!=null) {
// is killed
TriggerCallbackThread.pushCallBack(new HandleCallbackParam(
triggerParam.getLogId(),
triggerParam.getLogDateTime(),
XxlJobContext.HANDLE_CODE_FAIL,
stopReason + " [job not executed, in the job queue, killed.]")
);
}
}
//执行销毁方法
try {
handler.destroy();
} catch (Throwable e) {
logger.error(e.getMessage(), e);
}
logger.info(">>>>>>>>>>> xxl-job JobThread stoped, hashCode:{}", Thread.currentThread());
}
}
5.启动任务流程
1）step 1
1、操作流程：访问调度中心页面http://localhost:8080/xxl-job-admin -> 任务管理 -> 选择某条任务配置记录 -> 操作 -> 启动
2、接口地址：http://127.0.0.1:8080/xxl-job-admin/jobinfo/start
3、接口描述：执行启动任务接口
4、代码说明：对应controller具体代码参见 新增任务配置流程 step 1
2）step 2
5、com.xxl.job.admin.controller.JobInfoController#start内部调用方法com.xxl.job.admin.service.XxlJobService#start，其实现逻辑为：
① 调度类型参数校验（支持调度类型：CRON表达式、固定速度FIX_RATE）
② 以当前时间 + 5000ms 为 基础时间，根据任务配置规则生成下次执行时间
③ 更新xxl_job_info表记录 调度状态trigger_status为1（0-停止，1-运行）、更新时间update_time为当前时间、上次调度时间trigger_last_time为0、下次调度时间trigger_next_time(见步骤②)
具体代码参见 新增任务配置流程 step 2
3）step 3
6、调度中心启动时，会启动 定时任务调度线程 以及 时间轮任务消费线程，实现对任务的定时调用，具体代码参见 调度中心启动流程 step 9。
定时任务调度线程com.xxl.job.admin.core.thread.JobScheduleHelper#scheduleThread，其实现逻辑为：
①时间对其，相对本注册中心上一轮执行结束时间开始睡眠5s
②根据xxl.job.triggerpool.fast.max、xxl.job.triggerpool.slow.max配置计算当次预读任务的数量（xxl预计每个任务耗时50ms，即单个线程qps为20，那么预读任务数量 等于 (快任务线程池数量 + 慢任务线程池数量) * 20）
③记录任务开始执行时间
④关闭数据库自动事务提交，并开启事务
⑤对表xxl_job_lock的记录schedule_lock加锁（集群模式下xxl-job进行任务调度时，避免任务被重复调度）
⑥查询 当次预读任务的数量条 xxl_job_info表记录，满足 调度状态trigger_status为1（0-停止，1-运行）并且下次调度时间trigger_next_time小于等于 当前时间 + 5000ms，就是接下来 待处理 的 任务配置列表
⑦查询到的记录按照trigger_next_time值分三种情况进行处理：
1）当前时间 大于 trigger_next_time + 5000ms（对应图中的调度过期）
1~当调度过期策略为立即执行一次FIRE_ONCE_NOW时，立即触发任务执行参见 执行一次任务流程 step 2，否则直接执行下一行代码；
2~然后计算下次执行时间，为空时，设置XxlJobInfo记录的调度状态trigger_status为0（0-停止，1-运行）、上次调度时间trigger_last_time为0、下次调度时间trigger_next_time为0；非空时，刷新XxlJobInfo记录的上次调度时间trigger_last_time、下次调度时间trigger_next_time
2）当前时间 大于 trigger_next_time（对应图中的立即执行）
1~立即触发任务执行参见 执行一次任务流程 step 2
2~然后计算下次执行时间，为空时，设置XxlJobInfo记录的调度状态trigger_status为0（0-停止，1-运行）、上次调度时间trigger_last_time为0、下次调度时间trigger_next_time为0；非空时，刷新XxlJobInfo记录的上次调度时间trigger_last_time、下次调度时间trigger_next_time
3~如果trigger_status为1（0-停止，1-运行）且当前时间 + 5000ms 大于trigger_next_time，就把XxlJobInfo记录的id放入时间轮对象com.xxl.job.admin.core.thread.JobScheduleHelper#ringData（存放规则为：trigger_next_time/1000)%60，即下次执行时间的秒刻度时间）
3）当前时间 小于等于 trigger_next_time（对应图中的待执行）
1~直接将XxlJobInfo记录的id放入时间轮对象com.xxl.job.admin.core.thread.JobScheduleHelper#ringData（存放规则为：trigger_next_time/1000)%60，即下次执行时间的秒刻度时间）
⑧持久化xxl_job_info记录至数据库（更新调度状态trigger_status、上次调度时间trigger_last_time、下次调度时间trigger_next_time）
⑨事务提交，释放链接
⑩记录任务结束执行时间，并计算任务耗时
⑪如果任务耗时小于1秒，则执行线程睡眠（时间对其），睡眠时间按照查询到的xxl_job_info表记录数量为准，为0时睡眠5000s；否则睡眠1000s（一方面控制数据库访问频次，另外一方面防止任务频繁被执行）
⑫回到任务①进入下一轮任务执行
时间轮任务消费线程com.xxl.job.admin.core.thread.JobScheduleHelper#ringThread，其实现逻辑为：
①时间对其, 相对本注册中心上一轮执行结束时间睡眠1秒（由于本线程核心逻辑走的异步线程池，所以执行完一轮的速度很快，直接理解为程序读秒，即每秒执行一次即可）
②计算当前时间刻度（当前时间秒数值）
③从时间轮缓存对象com.xxl.job.admin.core.thread.JobScheduleHelper#ringData中取两个刻度的数据（避免处理耗时太长，跨过刻度，向前校验一个刻度），并赋值给临时变量，即当前刻度待执行任务列表
④循环刻度待执行任务列表，立即触发任务执行参见 执行一次任务流程 step 2
，每次执行完成后从com.xxl.job.admin.core.thread.JobScheduleHelper#ringData移除这部分数据
⑤回到任务①进行下一轮任务执行
6.主动标记执行结果丢失流程？？
1）step 1
1、执行结果丢失任务的定义为：调度记录停留在 “运行中” 状态超过10min，且对应执行器心跳注册失败不在线，则将本地调度主动标记失败
2、调度中心启动时，会启动执行结果丢失任务后台监控线程com.xxl.job.admin.core.thread.JobCompleteHelper#monitorThread，扫描xxl_job_log表，将所有符合条件的记录主动标记为失败，具体代码见 调度中心启动流程 step 7
3、
7.任务回调流程
1、任务处理结束时，方法com.xxl.job.core.thread.JobThread#run进入finally代码块，主动调用com.xxl.job.core.thread.TriggerCallbackThread#pushCallBack方法，将方法回调参数缓存至队列com.xxl.job.core.thread.TriggerCallbackThread.callBackQueue，具体代码参见 执行一次任务流程 step 4
2、执行器启动时会启动 任务回调 监控线程com.xxl.job.core.thread.TriggerCallbackThread#triggerCallbackThread，主要负责消耗com.xxl.job.core.thread.TriggerCallbackThread.callBackQueue队列中的数据，进行任务回调处理
3、回调执行成功时，主动调用调度中心/api/callback接口更新xxl_job_log表handle_time、handle_code、handle_msg状态，并触发子任务执行
8.过期日志清理流程
（四）扩展内容
1.自定义报警实现
1、先说结论：com.xxl.job.admin.core.alarm.JobAlarm为报警接口，自定义报警实现，只需要实现此接口即可。
2、一方面，项目启动时会主动执行回调方法com.xxl.job.admin.core.alarm.JobAlarmer#afterPropertiesSet，内部实现逻辑为去ioc容器中获取所有JobAlarm接口实例，并赋值给集合com.xxl.job.admin.core.alarm.JobAlarmer#jobAlarmList
com.xxl.job.admin.core.alarm.JobAlarmer具体实现代码如下
package com.xxl.job.admin.core.alarm;
import *;
@Component
public class JobAlarmer implements ApplicationContextAware, InitializingBean {
private static Logger logger = LoggerFactory.getLogger(JobAlarmer.class);
private ApplicationContext applicationContext;
private List<JobAlarm> jobAlarmList;
@Override
public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
this.applicationContext = applicationContext;
}
@Override
public void afterPropertiesSet() throws Exception {
// 获取所有com.xxl.job.admin.core.alarm.JobAlarm实例，并赋值给变量jobAlarmList
Map<String, JobAlarm> serviceBeanMap = applicationContext.getBeansOfType(JobAlarm.class);
if (serviceBeanMap != null && serviceBeanMap.size() > 0) {
jobAlarmList = new ArrayList<JobAlarm>(serviceBeanMap.values());
}
}
public boolean alarm(XxlJobInfo info, XxlJobLog jobLog) {
// 遍历所有com.xxl.job.admin.core.alarm.JobAlarm实例，调用com.xxl.job.admin.core.alarm.JobAlarm#doAlarm方法
boolean result = false;
if (jobAlarmList!=null && jobAlarmList.size()>0) {
result = true; // success means all-success
for (JobAlarm alarm: jobAlarmList) {
boolean resultItem = false;
try {
resultItem = alarm.doAlarm(info, jobLog);
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
if (!resultItem) {
result = false;
}
}
}
return result;
}
}
3、另一方面，参见 调度中心启动流程 step 6，系统启动时会启动 失败任务 后台监控线程，每10s执行一次，主要负责失败任务的重试、已经失败任务的报警工作。任务报警部分代码 具体代码参见 这一行：
boolean alarmResult = XxlJobAdminConfig.getAdminConfig().getJobAlarmer().alarm(info, log);
方法com.xxl.job.admin.core.alarm.JobAlarmer#alarm主要实现逻辑为，遍历集合com.xxl.job.admin.core.alarm.JobAlarmer#jobAlarmList，调用接口的com.xxl.job.admin.core.alarm.JobAlarm#doAlarm实现
2.路由策略
五、补充
（一）初始化数据库脚本
sql脚本位置：https://gitee.com/xuxueli0323/xxl-job/blob/master/doc/db/tables_xxl_job.sql
（二）重置数据库脚本
delete from xxl_job_group;
delete from xxl_job_info;
delete from xxl_job_log;
delete from xxl_job_log_report;
delete from xxl_job_logglue;
delete from xxl_job_registry;
INSERT INTO `xxl_job_group`(`id`, `app_name`, `title`, `address_type`, `address_list`, `update_time`) VALUES (1, 'xxl-job-executor-sample', '示例执行器', 0, NULL, '2018-11-03 22:21:31' );
INSERT INTO `xxl_job_info`(`id`, `job_group`, `job_desc`, `add_time`, `update_time`, `author`, `alarm_email`, `schedule_type`, `schedule_conf`, `misfire_strategy`, `executor_route_strategy`, `executor_handler`, `executor_param`, `executor_block_strategy`, `executor_timeout`, `executor_fail_retry_count`, `glue_type`, `glue_source`, `glue_remark`, `glue_updatetime`, `child_jobid`) VALUES (1, 1, '测试任务1', '2018-11-03 22:21:31', '2018-11-03 22:21:31', 'XXL', '', 'CRON', '0 0 0 * * ? *', 'DO_NOTHING', 'FIRST', 'demoJobHandler', '', 'SERIAL_EXECUTION', 0, 0, 'BEAN', '', 'GLUE代码初始化', '2018-11-03 22:21:31', '');
1303
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
