最近经过一段时间的源码系统学习,写一篇博客总结一下。本文记录并整理了学习XXL-JOB框架的全部过程,要求阅读前对XXL-JOB掌握基础的使用方法和基础原理,可以先参考博主之前的XXL-JOB搭建笔记博客:
https://blog.csdn.net/nannan7777/article/details/107337464?spm=1001.2014.3001.5501
分布式任务调度平台XXL-JOB源码解析
技术栈:
spring
+spring security
+jetty
+quartz
+mysql
+自研RPC
环境:idea:2018.3
,win10
,maven: 3.6.3
,jdk:1.8
,spring cloud:Finchley.RELEASE
,spring boot: 2.0.8
,quartz-2.2.3
,xxl-job:2.0.1
,xxl-rpc: 1.4.0
XXL-JOB 是一个轻量级分布式任务调度框架,主要基于spring quartz
框架搭建,它的核心设计理念是把任务调度分为两个核心部分:调度中心(xxl-admin)和执行器(executor),隔离成两个部分。这是一种中心化的设计,由调度中心来统一管理和调度各个接入的业务模块(执行器)。
接入的执行器只需要接收调度信号,然后去执行具体的业务逻辑,两者可以各自的进行扩容,采用自研RPC协议调用来实现执行器的注册和任务的调度,整体框架原理如图所示:
下面对XXL-JOB框架源码进行解析,按照使用逻辑顺序从如下三个方面进行总结:
一、 执行器注册至任务调度中心
XXL-JOB框架添加执行器到任务调度中心有两种方式:
(1)客户端执行器自动将名称和机器地址注册到任务调度中心
(2)在任务调度中心手动录入执行器名称和相关的机器地址(多个机器地址用逗号隔开)
下面对自动注册流程进行源码实现过程总结:
首先在在执行器客户端配置执行器名称和任务调度中心地址:
### xxl-job admin address list, such as "http://address" or "http://address01,http://address02"
##任务调度中心地址
xxl.job.admin.addresses=http://127.0.0.1:8080
### xxl-job executor address
##任务调度器名称和机器信息
xxl.job.executor.appname=xxl-job-executor-sample
xxl.job.executor.ip=127.0.0.1
xxl.job.executor.port=9998
执行器注册至任务调度中心的逻辑流程如图所示:
STEP1 执行器注册
在执行器启动时会读取上述配置,当存在任务调度中心地址时,会依次向任务调度中心注册其地址。
XxlJobExecutor类
在进行初始化时,通过XxlRpcReferenceBean
构建AdminBiz
的客户端,通过XxxlJobExecutor#getAdminBizList
来获取AdminBiz的Client
,调用任务调度中心AdMinBiz的相关方法。
具体代码如下:
private static void initAdminBizList(String adminAddresses, String accessToken) throws Exception {
if (adminAddresses!=null && adminAddresses.trim().length()>0) {
for (String address: adminAddresses.trim().split(",")) {
if (address!=null && address.trim().length()>0) {
String addressUrl = address.concat(AdminBiz.MAPPING);
AdminBiz adminBiz = (AdminBiz) new NetComClientProxy(AdminBiz.class, addressUrl, accessToken).getObject();
if (adminBizList == null) {
adminBizList = new ArrayList<AdminBiz>();
}
adminBizList.add(adminBiz);
}
}
}
}
其中,在XxlJobExecutor类
被调用时执行getObject()
方法,完成向任务调度中心发送请求进行执行器注册的操作。
具体代码如下:
@Override
public Object getObject() throws Exception {
return Proxy.newProxyInstance(Thread.currentThread()
.getContextClassLoader(), new Class[] { iface },
new InvocationHandler() {
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
// filter method like "Object.toString()"
if (Object.class.getName().equals(method.getDeclaringClass().getName())) {
logger.error(">>>>>>>>>>> xxl-rpc proxy class-method not support [{}.{}]", method.getDeclaringClass().getName(), method.getName());
throw new RuntimeException("xxl-rpc proxy class-method not support");
}
// request
RpcRequest request = new RpcRequest();
request.setServerAddress(serverAddress);
request.setCreateMillisTime(System.currentTimeMillis());
request.setAccessToken(accessToken);
request.setClassName(method.getDeclaringClass().getName());
request.setMethodName(method.getName());
request.setParameterTypes(method.getParameterTypes());
request.setParameters(args);
// send
//向任务调度中心发送请求进行服务注册
RpcResponse response = client.send(request);
// valid response
if (response == null) {
throw new Exception("Network request fail, response not found.");
}
if (response.isError()) {
throw new RuntimeException(response.getError());
} else {
return response.getResult();
}
}
});
最后,调用send()
方法,完成执行器的注册操作。
具体代码如下:
public RpcResponse send(RpcRequest request) throws Exception {
try {
// serialize request
byte[] requestBytes = HessianSerializer.serialize(request);
// reqURL
String reqURL = request.getServerAddress();
if (reqURL!=null && reqURL.toLowerCase().indexOf("http")==-1) {
reqURL = "http://" + request.getServerAddress() + "/"; // IP:PORT, need parse to url
}
//发送post请求进行服务注册,简单注册一下IP和端口信息等
// remote invoke
byte[] responseBytes = HttpClientUtil.postRequest(reqURL, requestBytes);
if (responseBytes == null || responseBytes.length==0) {
RpcResponse rpcResponse = new RpcResponse();
rpcResponse.setError("Network request fail, RpcResponse byte[] is null");
return rpcResponse;
}
// deserialize response
RpcResponse rpcResponse = (RpcResponse) HessianSerializer.deserialize(responseBytes, RpcResponse.class);
return rpcResponse;
} catch (Exception e) {
logger.error(e.getMessage(), e);
RpcResponse rpcResponse = new RpcResponse();
rpcResponse.setError("Network request error: " + e.getMessage());
return rpcResponse;
}
}
总结:STEP1中,执行器注册时,注册到任务调度中心的信息仅仅包括一些基本信息,IP、端口和应用名称等等,并不用将具体的任务类等信息注册到任务调度中心,所以任务调度中心无法感知执行器的具体业务逻辑信息,可以通过添加标识或打印日志详细信息进行监管。
STEP2 任务调度中心接收注册
任务调度中心对外提供注册地址http://127.0.0.1:8080/api
,通过JobApiController#api
接口用来接受注册执行器相关信息。具体代码如下:
@Controller
public class JobApiController {
private static Logger logger = LoggerFactory.getLogger(JobApiController.class);
private RpcResponse doInvoke(HttpServletRequest request) {
try {
// deserialize request
byte[] requestBytes = HttpClientUtil.readBytes(request);
if (requestBytes == null || requestBytes.length==0) {
RpcResponse rpcResponse = new RpcResponse();
rpcResponse.setError("RpcRequest byte[] is null");
return rpcResponse;
}
//反序列化数据
RpcRequest rpcRequest = (RpcRequest) HessianSerializer.deserialize(requestBytes, RpcRequest.class);
// invoke
//调用执行器注册方法
RpcResponse rpcResponse = NetComServerFactory.invokeService(rpcRequest, null);
return rpcResponse;
} catch (Exception e) {
logger.error(e.getMessage(), e);
RpcResponse rpcResponse = new RpcResponse();
rpcResponse.setError("Server-error:" + e.getMessage());
return rpcResponse;
}
}
//对外提供api接口
@RequestMapping(AdminBiz.MAPPING)
@PermessionLimit(limit=false)
public void api(HttpServletRequest request, HttpServletResponse response) throws IOException {
// invoke
RpcResponse rpcResponse = doInvoke(request);
// serialize response
byte[] responseBytes = HessianSerializer.serialize(rpcResponse);
response.setContentType("text/html;charset=utf-8");
response.setStatus(HttpServletResponse.SC_OK);
//baseRequest.setHandled(true);
OutputStream out = response.getOutputStream();
out.write(responseBytes);
out.flush();
}
}
其中,在NetComServerFactory类
中调用invokeService()方法
,根据反射调用AdminBiz接口
,实现AdminBizImpl类
的register()方法
,完成执行器注册操作。
具体代码如下:
public static RpcResponse invokeService(RpcRequest request, Object serviceBean) {
if (serviceBean==null) {
serviceBean = serviceMap.get(request.getClassName());
}
if (serviceBean == null) {
// TODO
}
RpcResponse response = new RpcResponse();
if (System.currentTimeMillis() - request.getCreateMillisTime() > 180000) {
response.setResult(new ReturnT<String>(ReturnT.FAIL_CODE, "The timestamp difference between admin and executor exceeds the limit."));
return response;
}
if (accessToken!=null && accessToken.trim().length()>0 && !accessToken.trim().equals(request.getAccessToken())) {
response.setResult(new ReturnT<String>(ReturnT.FAIL_CODE, "The access token[" + request.getAccessToken() + "] is wrong."));
return response;
}
try {
//接口AdminBiz的实现类AdminBizImpl
Class<?> serviceClass = serviceBean.getClass();
//AdminBiz的register方法
String methodName = request.getMethodName();
Class<?>[] parameterTypes = request.getParameterTypes();
Object[] parameters = request.getParameters();
FastClass serviceFastClass = FastClass.create(serviceClass);
//调用AdminBizImpl的register方法
FastMethod serviceFastMethod = serviceFastClass.getMethod(methodName, parameterTypes);
Object result = serviceFastMethod.invoke(serviceBean, parameters);
response.setResult(result);
} catch (Throwable t) {
t.printStackTrace();
response.setError(t.getMessage());
}
return response;
}
最后,在AdminBiz
的实现类AdminBizImpl
中,调用dao层
完成注册信息的入库操作。
具体代码如下:
@Override
public ReturnT<String> registry(RegistryParam registryParam) {
//在xxl_job_qrtz_trigger_registry表中添加或更新数据
int ret = xxlJobRegistryDao.registryUpdate(registryParam.getRegistGroup(), registryParam.getRegistryKey(), registryParam.getRegistryValue());
if (ret < 1) {
xxlJobRegistryDao.registrySave(registryParam.getRegistGroup(), registryParam.getRegistryKey(), registryParam.getRegistryValue());
}
return ReturnT.SUCCESS;
}
此时表中数据如下所示:
此时在任务调度中心的前端页面中会展示相关执行器数据,如下所示:
总结:STEP2中,执行器将其服务器信息通过http请求发送至任务调度中心,任务调度中心将相关注册信息插入或更新到xxl_job_qrtz_trigger_registry表中,在任务调度中心的前端页面上通过列表展示,接下来相关任务执行时会根据执行器信息向某个指定执行器发送调用请求。
二、 任务调度中心任务管理
XXL-JOB框架是基于quartz
来实现定时任务,其中任务调度中心任务执行基于quartz,任务执行器也是一个被调用执行的服务。
1、任务创建
调用JobInfoController
的add()方法
添加任务。
具体代码如下:
@RequestMapping("/add")
@ResponseBody
public ReturnT<String> add(XxlJobInfo jobInfo) {
//添加定时器
return xxlJobService.add(jobInfo);
}
其中,调用XxlJobService
的add()方法
,新建quartz任务。
具体代码如下:
@Override
public ReturnT<String> add(XxlJobInfo jobInfo) {
// valid
//获取任务分组
XxlJobGroup group = xxlJobGroupDao.load(jobInfo.getJobGroup());
//省略部分任务相关信息检查的代码
// add in db
//将任务持久化到数据库的xxl_job_qrtz_trigger_info
xxlJobInfoDao.save(jobInfo);
if (jobInfo.getId() < 1) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("jobinfo_field_add")+I18nUtil.getString("system_fail")) );
}
// add in quartz
String qz_group = String.valueOf(jobInfo.getJobGroup());
String qz_name = String.valueOf(jobInfo.getId());
try {
//添加任务到quartz中
XxlJobDynamicScheduler.addJob(qz_name, qz_group, jobInfo.getJobCron());
//XxlJobDynamicScheduler.pauseJob(qz_name, qz_group);
return ReturnT.SUCCESS;
} catch (SchedulerException e) {
logger.error(e.getMessage(), e);
try {
xxlJobInfoDao.delete(jobInfo.getId());
XxlJobDynamicScheduler.removeJob(qz_name, qz_group);
} catch (SchedulerException e1) {
logger.error(e.getMessage(), e1);
}
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("jobinfo_field_add")+I18nUtil.getString("system_fail"))+":" + e.getMessage());
}
}
其中,在XxlJobDynamicScheduler
中调用quartz的创建任务方法构建定时任务,定时任务执行类为QuartzJobBean
的子类RemoteHttpJobBean
,在定时任务执行时会调用RemoteHttpJobBean中任务执行器的接口,执行相关任务。
具体代码如下:
public static boolean addJob(String jobName, String jobGroup, String cronExpression) throws SchedulerException {
// TriggerKey : name + group
//创建定时器别名
TriggerKey triggerKey = TriggerKey.triggerKey(jobName, jobGroup);
JobKey jobKey = new JobKey(jobName, jobGroup);
// TriggerKey valid if_exists
if (checkExists(jobName, jobGroup)) {
logger.info(">>>>>>>>> addJob fail, job already exist, jobGroup:{}, jobName:{}", jobGroup, jobName);
return false;
}
// CronTrigger : TriggerKey + cronExpression // withMisfireHandlingInstructionDoNothing 忽略掉调度终止过程中忽略的调度
CronScheduleBuilder cronScheduleBuilder = CronScheduleBuilder.cronSchedule(cronExpression).withMisfireHandlingInstructionDoNothing();
//创建cron定时器
CronTrigger cronTrigger = TriggerBuilder.newTrigger().withIdentity(triggerKey).withSchedule(cronScheduleBuilder).build();
// JobDetail : jobClass
//定时执行类
Class<? extends Job> jobClass_ = RemoteHttpJobBean.class; // Class.forName(jobInfo.getJobClass());
//创建任务
JobDetail jobDetail = JobBuilder.newJob(jobClass_).withIdentity(jobKey).build();
/*if (jobInfo.getJobData()!=null) {
JobDataMap jobDataMap = jobDetail.getJobDataMap();
jobDataMap.putAll(JacksonUtil.readValue(jobInfo.getJobData(), Map.class));
// JobExecutionContext context.getMergedJobDataMap().get("mailGuid");
}*/
// schedule : jobDetail + cronTrigger
//添加定时任务到quartz
Date date = scheduler.scheduleJob(jobDetail, cronTrigger);
logger.info(">>>>>>>>>>> addJob success, jobDetail:{}, cronTrigger:{}, date:{}", jobDetail, cronTrigger, date);
return true;
}
2、任务删除
将任务从quartz中移除,不再执行。JobInfoController
中提供了删除任务的接口。
具体代码如下:
@RequestMapping("/remove")
@ResponseBody
public ReturnT<String> remove(int id) {
return xxlJobService.remove(id);
}
在XxlJobService中根据逻辑id删除任务,并且删除数据库中相关持久化数据
@Override
public ReturnT<String> remove(int id) {
XxlJobInfo xxlJobInfo = xxlJobInfoDao.loadById(id);
String group = String.valueOf(xxlJobInfo.getJobGroup());
String name = String.valueOf(xxlJobInfo.getId());
try {
//从quartz中移除任务,并将数据库中相关数据删除
XxlJobDynamicScheduler.removeJob(name, group);
xxlJobInfoDao.delete(id);
xxlJobLogDao.delete(id);
xxlJobLogGlueDao.deleteByJobId(id);
return ReturnT.SUCCESS;
} catch (SchedulerException e) {
logger.error(e.getMessage(), e);
}
return ReturnT.FAIL;
}
同理,在XxlJobDynamicScheduler
中调用removeJob()方法
删除相关任务。
具体代码如下:
public static boolean removeJob(String jobName, String jobGroup) throws SchedulerException {
// TriggerKey : name + group
TriggerKey triggerKey = TriggerKey.triggerKey(jobName, jobGroup);
boolean result = false;
if (checkExists(jobName, jobGroup)) {
//删除任务
result = scheduler.unscheduleJob(triggerKey);
logger.info(">>>>>>>>>>> removeJob, triggerKey:{}, result [{}]", triggerKey, result);
}
return true;
}
3、任务暂停
调用quartz接口进行任务暂停处理,在JobInfoController
中提供接口处理方法。
具体代码如下:
@RequestMapping("/pause")
@ResponseBody
public ReturnT<String> pause(int id) {
return xxlJobService.pause(id);
}
其中,调用XxlJobService
的paus()方法
进行任务停止。
具体代码如下:
@Override
public ReturnT<String> pause(int id) {
XxlJobInfo xxlJobInfo = xxlJobInfoDao.loadById(id);
String group = String.valueOf(xxlJobInfo.getJobGroup());
String name = String.valueOf(xxlJobInfo.getId());
try {
boolean ret = XxlJobDynamicScheduler.pauseJob(name, group); // jobStatus do not store
return ret?ReturnT.SUCCESS:ReturnT.FAIL;
} catch (SchedulerException e) {
logger.error(e.getMessage(), e);
return ReturnT.FAIL;
}
}
同理,调用XxlJobDynamicScheduler
的pauseJob()方法
进行任务停止,执行quartz的接口完成。
具体代码如下:
public static boolean pauseJob(String jobName, String jobGroup) throws SchedulerException {
// TriggerKey : name + group
TriggerKey triggerKey = TriggerKey.triggerKey(jobName, jobGroup);
boolean result = false;
if (checkExists(jobName, jobGroup)) {
//任务暂停
scheduler.pauseTrigger(triggerKey);
result = true;
logger.info(">>>>>>>>>>> pauseJob success, triggerKey:{}", triggerKey);
} else {
logger.info(">>>>>>>>>>> pauseJob fail, triggerKey:{}", triggerKey);
}
return result;
}
4、任务恢复
将暂停的任务继续重新执行,在JobInfoController
中提供接口重新执行任务。
具体代码如下:
@RequestMapping("/resume")
@ResponseBody
public ReturnT<String> resume(int id) {
return xxlJobService.resume(id);
}
其中,调用XxlJobService
的resume()
接口继续任务。
具体代码如下:
@Override
public ReturnT<String> resume(int id) {
XxlJobInfo xxlJobInfo = xxlJobInfoDao.loadById(id);
String group = String.valueOf(xxlJobInfo.getJobGroup());
String name = String.valueOf(xxlJobInfo.getId());
try {
//继续任务
boolean ret = XxlJobDynamicScheduler.resumeJob(name, group);
return ret?ReturnT.SUCCESS:ReturnT.FAIL;
} catch (SchedulerException e) {
logger.error(e.getMessage(), e);
return ReturnT.FAIL;
}
}
其中,在XllJobDynamicScheduler
中调用quartz的接口resumeJob()
重新开启任务。
具体代码如下:
public static boolean resumeJob(String jobName, String jobGroup) throws SchedulerException {
// TriggerKey : name + group
TriggerKey triggerKey = TriggerKey.triggerKey(jobName, jobGroup);
boolean result = false;
if (checkExists(jobName, jobGroup)) {
//继续任务
scheduler.resumeTrigger(triggerKey);
result = true;
logger.info(">>>>>>>>>>> resumeJob success, triggerKey:{}", triggerKey);
} else {
logger.info(">>>>>>>>>>> resumeJob fail, triggerKey:{}", triggerKey);
}
return result;
}
5、任务更新
更新任务相关参数,在XxlJobController
中提供update()
接口。
具体代码如下:
@RequestMapping("/update")
@ResponseBody
public ReturnT<String> update(XxlJobInfo jobInfo) {
return xxlJobService.update(jobInfo);
}
其中,在XxlJobService
中调用update()方法
更新任务,并更新相关数据库中的数据。
具体代码如下:
@Override
public ReturnT<String> update(XxlJobInfo jobInfo) {
//省略部分判断代码
// stage job info
XxlJobInfo exists_jobInfo = xxlJobInfoDao.loadById(jobInfo.getId());
if (exists_jobInfo == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, (I18nUtil.getString("jobinfo_field_id")+I18nUtil.getString("system_not_found")) );
}
exists_jobInfo.setJobCron(jobInfo.getJobCron());
exists_jobInfo.setJobDesc(jobInfo.getJobDesc());
exists_jobInfo.setAuthor(jobInfo.getAuthor());
exists_jobInfo.setAlarmEmail(jobInfo.getAlarmEmail());
exists_jobInfo.setExecutorRouteStrategy(jobInfo.getExecutorRouteStrategy());
exists_jobInfo.setExecutorHandler(jobInfo.getExecutorHandler());
exists_jobInfo.setExecutorParam(jobInfo.getExecutorParam());
exists_jobInfo.setExecutorBlockStrategy(jobInfo.getExecutorBlockStrategy());
exists_jobInfo.setExecutorFailStrategy(jobInfo.getExecutorFailStrategy());
exists_jobInfo.setExecutorTimeout(jobInfo.getExecutorTimeout());
exists_jobInfo.setChildJobId(jobInfo.getChildJobId());
xxlJobInfoDao.update(exists_jobInfo);
// fresh quartz
String qz_group = String.valueOf(exists_jobInfo.getJobGroup());
String qz_name = String.valueOf(exists_jobInfo.getId());
try {
//重新配置定时任务
boolean ret = XxlJobDynamicScheduler.rescheduleJob(qz_group, qz_name, exists_jobInfo.getJobCron());
return ret?ReturnT.SUCCESS:ReturnT.FAIL;
} catch (SchedulerException e) {
logger.error(e.getMessage(), e);
}
return ReturnT.FAIL;
}
其中,在XxlJobDynamicScheduler
中调用quartz相关方法重新配置定时任务。
具体代码如下:
public static boolean rescheduleJob(String jobGroup, String jobName, String cronExpression) throws SchedulerException {
// TriggerKey valid if_exists
if (!checkExists(jobName, jobGroup)) {
logger.info(">>>>>>>>>>> rescheduleJob fail, job not exists, JobGroup:{}, JobName:{}", jobGroup, jobName);
return false;
}
// TriggerKey : name + group
TriggerKey triggerKey = TriggerKey.triggerKey(jobName, jobGroup);
CronTrigger oldTrigger = (CronTrigger) scheduler.getTrigger(triggerKey);
//任务存在直接更新一下
if (oldTrigger != null) {
// avoid repeat
String oldCron = oldTrigger.getCronExpression();
if (oldCron.equals(cronExpression)){
return true;
}
// CronTrigger : TriggerKey + cronExpression
CronScheduleBuilder cronScheduleBuilder = CronScheduleBuilder.cronSchedule(cronExpression).withMisfireHandlingInstructionDoNothing();
oldTrigger = oldTrigger.getTriggerBuilder().withIdentity(triggerKey).withSchedule(cronScheduleBuilder).build();
// rescheduleJob
scheduler.rescheduleJob(triggerKey, oldTrigger);
} else {
//不存在和直接创建新的任务类似
// CronTrigger : TriggerKey + cronExpression
CronScheduleBuilder cronScheduleBuilder = CronScheduleBuilder.cronSchedule(cronExpression).withMisfireHandlingInstructionDoNothing();
CronTrigger cronTrigger = TriggerBuilder.newTrigger().withIdentity(triggerKey).withSchedule(cronScheduleBuilder).build();
// JobDetail-JobDataMap fresh
JobKey jobKey = new JobKey(jobName, jobGroup);
JobDetail jobDetail = scheduler.getJobDetail(jobKey);
/*JobDataMap jobDataMap = jobDetail.getJobDataMap();
jobDataMap.clear();
jobDataMap.putAll(JacksonUtil.readValue(jobInfo.getJobData(), Map.class));*/
// Trigger fresh
HashSet<Trigger> triggerSet = new HashSet<Trigger>();
triggerSet.add(cronTrigger);
scheduler.scheduleJob(jobDetail, triggerSet, true);
}
logger.info(">>>>>>>>>>> resumeJob success, JobGroup:{}, JobName:{}", jobGroup, jobName);
return true;
}
6、立即执行任务
在任务调度中心的前端页面可以立即进行任务调用,在JobInfoController
中提供接口直接进行任务调用。
具体代码如下:
@RequestMapping("/trigger")
@ResponseBody
public ReturnT<String> triggerJob(int id) {
return xxlJobService.triggerJob(id);
}
其中,在XxlJobService
中调用接口triggerJob()
可以直接进行任务调度。
具体代码如下:
@Override
public ReturnT<String> triggerJob(int id) {
JobTriggerPoolHelper.trigger(id);
return ReturnT.SUCCESS;
最后,在JobTriggerPoolHelper
的trigger()方法
会将任务id添加至多线程中,最终的实现同样是调用RemoteHttpJobBean
的executeInternal()方法
进行任务调度,不是依赖quartz实现,具体的业务逻辑见下文第三部分。
三、 任务调度中心触发任务流程
执行器提供web服务,任务调度中心根据任务分组发送http请求,执行器收到请求后,根据请求中的数据调用对应的任务进行具体业务的执行,逻辑流程图图所示:
具体实施步骤如下:
STEP1 任务调度中心发送任务执行请求
任务调度中心发送任务的方式有两种:
(1)根据配置的cron表达式周期性执行相关任务;
(2)在任务调度中心中主动执行任务。
由于在注册
quartz定时任务
时,已经注册执行类为RemoteHttpJobBean
,所以周期性执行定时任务时会调用RemoteHttpJobBean类中的executeInternal()方法
,在executeInternal中调用方法JobTriggerPoolHelper.trigger(jobId)
;同时,通过任务调度中心主动执行任务时也会调用该方法,因此重点可以放在该方法中做的逻辑处理。
executeInternal()方法
的具体代码如下:
public class RemoteHttpJobBean extends QuartzJobBean {
private static Logger logger = LoggerFactory.getLogger(RemoteHttpJobBean.class);
@Override
protected void executeInternal(JobExecutionContext context)
throws JobExecutionException {
// load jobId
JobKey jobKey = context.getTrigger().getJobKey();
Integer jobId = Integer.valueOf(jobKey.getName());
// trigger
//XxlJobTrigger.trigger(jobId);
JobTriggerPoolHelper.trigger(jobId);
}
}
其中,JobTriggerPoolHelper.trigger
所做的操作是将任务提交给一个线程池,在线程池中调用XxlJobTrigger.trigger
。
具体代码如下:
public void addTrigger(final int jobId){
triggerPool.execute(new Runnable() {
@Override
public void run() {
XxlJobTrigger.trigger(jobId);
}
});
}
之后,在XxlJobTrigger.trigger
中会根据jobId
获取任务的基本配置信息(阻塞策略、路由策略、失败重试测试、分组服务器列表等等),然后可以根据路由策略选择是广播还是单播等等,接下来组装消息体调用runExecutor()方法
发送http请求到任务执行器。
具体代码如下:
public static void trigger(int jobId) {
//获取任务信息
// load data
XxlJobInfo jobInfo = XxlJobDynamicScheduler.xxlJobInfoDao.loadById(jobId); // job info
if (jobInfo == null) {
logger.warn(">>>>>>>>>>>> trigger fail, jobId invalid,jobId={}", jobId);
return;
}
//根据任务的分组信息找到分组,分组中存在服务器的IP和端口地址等
XxlJobGroup group = XxlJobDynamicScheduler.xxlJobGroupDao.load(jobInfo.getJobGroup()); // group info
//阻塞策略
ExecutorBlockStrategyEnum blockStrategy = ExecutorBlockStrategyEnum.match(jobInfo.getExecutorBlockStrategy(), ExecutorBlockStrategyEnum.SERIAL_EXECUTION); // block strategy
//失败策略
ExecutorFailStrategyEnum failStrategy = ExecutorFailStrategyEnum.match(jobInfo.getExecutorFailStrategy(), ExecutorFailStrategyEnum.NULL); // fail strategy
//执行路由测试
ExecutorRouteStrategyEnum executorRouteStrategyEnum = ExecutorRouteStrategyEnum.match(jobInfo.getExecutorRouteStrategy(), null); // route strategy
//服务器地址
ArrayList<String> addressList = (ArrayList<String>) group.getRegistryList();
//广播模式
// broadcast
if (ExecutorRouteStrategyEnum.SHARDING_BROADCAST == executorRouteStrategyEnum && CollectionUtils.isNotEmpty(addressList)) {
//依次调用所有的服务器
for (int i = 0; i < addressList.size(); i++) {
String address = addressList.get(i);
// 1、save log-id
XxlJobLog jobLog = new XxlJobLog();
jobLog.setJobGroup(jobInfo.getJobGroup());
jobLog.setJobId(jobInfo.getId());
XxlJobDynamicScheduler.xxlJobLogDao.save(jobLog);
logger.debug(">>>>>>>>>>> xxl-job trigger start, jobId:{}", jobLog.getId());
// 2、prepare trigger-info
//jobLog.setExecutorAddress(executorAddress);
jobLog.setGlueType(jobInfo.getGlueType());
jobLog.setExecutorHandler(jobInfo.getExecutorHandler());
jobLog.setExecutorParam(jobInfo.getExecutorParam());
jobLog.setTriggerTime(new Date());
ReturnT<String> triggerResult = new ReturnT<String>(null);
StringBuffer triggerMsgSb = new StringBuffer();
triggerMsgSb.append(I18nUtil.getString("jobconf_trigger_admin_adress")).append(":").append(IpUtil.getIp());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_exe_regtype")).append(":")
.append( (group.getAddressType() == 0)?I18nUtil.getString("jobgroup_field_addressType_0"):I18nUtil.getString("jobgroup_field_addressType_1") );
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_exe_regaddress")).append(":").append(group.getRegistryList());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorRouteStrategy")).append(":").append(executorRouteStrategyEnum.getTitle()).append("("+i+"/"+addressList.size()+")"); // update01
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorBlockStrategy")).append(":").append(blockStrategy.getTitle());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorFailStrategy")).append(":").append(failStrategy.getTitle());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_timeout")).append(":").append(jobInfo.getExecutorTimeout());
// 3、trigger-valid
if (triggerResult.getCode()==ReturnT.SUCCESS_CODE && CollectionUtils.isEmpty(addressList)) {
triggerResult.setCode(ReturnT.FAIL_CODE);
triggerMsgSb.append("<br>----------------------<br>").append(I18nUtil.getString("jobconf_trigger_address_empty"));
}
if (triggerResult.getCode() == ReturnT.SUCCESS_CODE) {
// 4.1、trigger-param
TriggerParam triggerParam = new TriggerParam();
triggerParam.setJobId(jobInfo.getId());
triggerParam.setExecutorHandler(jobInfo.getExecutorHandler());
triggerParam.setExecutorParams(jobInfo.getExecutorParam());
triggerParam.setExecutorBlockStrategy(jobInfo.getExecutorBlockStrategy());
triggerParam.setExecutorTimeout(jobInfo.getExecutorTimeout());
triggerParam.setLogId(jobLog.getId());
triggerParam.setLogDateTim(jobLog.getTriggerTime().getTime());
triggerParam.setGlueType(jobInfo.getGlueType());
triggerParam.setGlueSource(jobInfo.getGlueSource());
triggerParam.setGlueUpdatetime(jobInfo.getGlueUpdatetime().getTime());
triggerParam.setBroadcastIndex(i);
triggerParam.setBroadcastTotal(addressList.size()); // update02
// 4.2、trigger-run (route run / trigger remote executor)
//远程调用服务接口,执行任务
triggerResult = runExecutor(triggerParam, address); // update03
triggerMsgSb.append("<br><br><span style=\"color:#00c0ef;\" > >>>>>>>>>>>"+ I18nUtil.getString("jobconf_trigger_run") +"<<<<<<<<<<< </span><br>").append(triggerResult.getMsg());
// 4.3、trigger (fail retry)
if (triggerResult.getCode()!=ReturnT.SUCCESS_CODE && failStrategy == ExecutorFailStrategyEnum.FAIL_TRIGGER_RETRY) {
triggerResult = runExecutor(triggerParam, address); // update04
triggerMsgSb.append("<br><br><span style=\"color:#F39C12;\" > >>>>>>>>>>>"+ I18nUtil.getString("jobconf_fail_trigger_retry") +"<<<<<<<<<<< </span><br>").append(triggerResult.getMsg());
}
}
// 5、save trigger-info
jobLog.setExecutorAddress(triggerResult.getContent());
jobLog.setTriggerCode(triggerResult.getCode());
jobLog.setTriggerMsg(triggerMsgSb.toString());
XxlJobDynamicScheduler.xxlJobLogDao.updateTriggerInfo(jobLog);
// 6、monitor trigger
JobFailMonitorHelper.monitor(jobLog.getId());
logger.debug(">>>>>>>>>>> xxl-job trigger end, jobId:{}", jobLog.getId());
}
} else {
//单播模式
// 1、save log-id
XxlJobLog jobLog = new XxlJobLog();
jobLog.setJobGroup(jobInfo.getJobGroup());
jobLog.setJobId(jobInfo.getId());
XxlJobDynamicScheduler.xxlJobLogDao.save(jobLog);
logger.debug(">>>>>>>>>>> xxl-job trigger start, jobId:{}", jobLog.getId());
// 2、prepare trigger-info
//jobLog.setExecutorAddress(executorAddress);
jobLog.setGlueType(jobInfo.getGlueType());
jobLog.setExecutorHandler(jobInfo.getExecutorHandler());
jobLog.setExecutorParam(jobInfo.getExecutorParam());
jobLog.setTriggerTime(new Date());
ReturnT<String> triggerResult = new ReturnT<String>(null);
StringBuffer triggerMsgSb = new StringBuffer();
triggerMsgSb.append(I18nUtil.getString("jobconf_trigger_admin_adress")).append(":").append(IpUtil.getIp());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_exe_regtype")).append(":")
.append( (group.getAddressType() == 0)?I18nUtil.getString("jobgroup_field_addressType_0"):I18nUtil.getString("jobgroup_field_addressType_1") );
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_exe_regaddress")).append(":").append(group.getRegistryList());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorRouteStrategy")).append(":").append(executorRouteStrategyEnum.getTitle());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorBlockStrategy")).append(":").append(blockStrategy.getTitle());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorFailStrategy")).append(":").append(failStrategy.getTitle());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_timeout")).append(":").append(jobInfo.getExecutorTimeout());
// 3、trigger-valid
if (triggerResult.getCode()==ReturnT.SUCCESS_CODE && CollectionUtils.isEmpty(addressList)) {
triggerResult.setCode(ReturnT.FAIL_CODE);
triggerMsgSb.append("<br>----------------------<br>").append(I18nUtil.getString("jobconf_trigger_address_empty"));
}
if (triggerResult.getCode() == ReturnT.SUCCESS_CODE) {
// 4.1、trigger-param
TriggerParam triggerParam = new TriggerParam();
triggerParam.setJobId(jobInfo.getId());
triggerParam.setExecutorHandler(jobInfo.getExecutorHandler());
triggerParam.setExecutorParams(jobInfo.getExecutorParam());
triggerParam.setExecutorBlockStrategy(jobInfo.getExecutorBlockStrategy());
triggerParam.setExecutorTimeout(jobInfo.getExecutorTimeout());
triggerParam.setLogId(jobLog.getId());
triggerParam.setLogDateTim(jobLog.getTriggerTime().getTime());
triggerParam.setGlueType(jobInfo.getGlueType());
triggerParam.setGlueSource(jobInfo.getGlueSource());
triggerParam.setGlueUpdatetime(jobInfo.getGlueUpdatetime().getTime());
triggerParam.setBroadcastIndex(0);
triggerParam.setBroadcastTotal(1);
// 4.2、trigger-run (route run / trigger remote executor)
//路由后远程调用服务接口,执行任务
triggerResult = executorRouteStrategyEnum.getRouter().routeRun(triggerParam, addressList);
triggerMsgSb.append("<br><br><span style=\"color:#00c0ef;\" > >>>>>>>>>>>"+ I18nUtil.getString("jobconf_trigger_run") +"<<<<<<<<<<< </span><br>").append(triggerResult.getMsg());
// 4.3、trigger (fail retry)
if (triggerResult.getCode()!=ReturnT.SUCCESS_CODE && failStrategy == ExecutorFailStrategyEnum.FAIL_TRIGGER_RETRY) {
triggerResult = executorRouteStrategyEnum.getRouter().routeRun(triggerParam, addressList);
triggerMsgSb.append("<br><br><span style=\"color:#F39C12;\" > >>>>>>>>>>>"+ I18nUtil.getString("jobconf_fail_trigger_retry") +"<<<<<<<<<<< </span><br>").append(triggerResult.getMsg());
}
}
// 5、save trigger-info
jobLog.setExecutorAddress(triggerResult.getContent());
jobLog.setTriggerCode(triggerResult.getCode());
jobLog.setTriggerMsg(triggerMsgSb.toString());
XxlJobDynamicScheduler.xxlJobLogDao.updateTriggerInfo(jobLog);
// 6、monitor trigger
JobFailMonitorHelper.monitor(jobLog.getId());
logger.debug(">>>>>>>>>>> xxl-job trigger end, jobId:{}", jobLog.getId());
}
}
其中,在这里的runExecutor()方法
中,根据address服务器地址,XxlJobDynamicScheduler.getExecutorBiz
中会获取代理类最终调用JettyClient的send()方法
。
具体代码如下:
public static ReturnT<String> runExecutor(TriggerParam triggerParam, String address){
ReturnT<String> runResult = null;
try {
//获取代理对象
ExecutorBiz executorBiz = XxlJobDynamicScheduler.getExecutorBiz(address);
//最终调用执行
runResult = executorBiz.run(triggerParam);
} catch (Exception e) {
logger.error(">>>>>>>>>>> xxl-job trigger error, please check if the executor[{}] is running.", address, e);
runResult = new ReturnT<String>(ReturnT.FAIL_CODE, ""+e );
}
StringBuffer runResultSB = new StringBuffer(I18nUtil.getString("jobconf_trigger_run") + ":");
runResultSB.append("<br>address:").append(address);
runResultSB.append("<br>code:").append(runResult.getCode());
runResultSB.append("<br>msg:").append(runResult.getMsg());
runResult.setMsg(runResultSB.toString());
runResult.setContent(address);
return runResult;
}
其中,getExecutorBiz()方法
中会通过NetComClientProxy
生成代理对象,在执行时会调用其方法。
具体代码如下:
public static ExecutorBiz getExecutorBiz(String address) throws Exception {
// valid
if (address==null || address.trim().length()==0) {
return null;
}
// load-cache
address = address.trim();
ExecutorBiz executorBiz = executorBizRepository.get(address);
if (executorBiz != null) {
return executorBiz;
}
// set-cache
executorBiz = (ExecutorBiz) new NetComClientProxy(ExecutorBiz.class, address, accessToken).getObject();
executorBizRepository.put(address, executorBiz);
return executorBiz;
}
最后,在getObject()方法
中生成代理对象,执行JettyClient的send()方法
。
具体代码如下:
@Override
public Object getObject() throws Exception {
return Proxy.newProxyInstance(Thread.currentThread()
.getContextClassLoader(), new Class[] { iface },
new InvocationHandler() {
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
// filter method like "Object.toString()"
if (Object.class.getName().equals(method.getDeclaringClass().getName())) {
logger.error(">>>>>>>>>>> xxl-rpc proxy class-method not support [{}.{}]", method.getDeclaringClass().getName(), method.getName());
throw new RuntimeException("xxl-rpc proxy class-method not support");
}
// request
RpcRequest request = new RpcRequest();
request.setServerAddress(serverAddress);
request.setCreateMillisTime(System.currentTimeMillis());
request.setAccessToken(accessToken);
request.setClassName(method.getDeclaringClass().getName());
request.setMethodName(method.getName());
request.setParameterTypes(method.getParameterTypes());
request.setParameters(args);
//发起http调用,执行任务
// send
RpcResponse response = client.send(request);
// valid response
if (response == null) {
throw new Exception("Network request fail, response not found.");
}
if (response.isError()) {
throw new RuntimeException(response.getError());
} else {
return response.getResult();
}
}
});
}
STEP2 执行器接收任务后执行
执行器会根据内置的jetty提供web服务,提供请求处理器JettyServerHandler
,从而接收任务调度中心发送过来的任务进行处理。
具体代码如下:
public class JettyServerHandler extends AbstractHandler {
private static Logger logger = LoggerFactory.getLogger(JettyServerHandler.class);
//接收请求,处理任务
@Override
public void handle(String target, Request baseRequest, HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException {
// invoke
//调用任务执行
RpcResponse rpcResponse = doInvoke(request);
// serialize response
byte[] responseBytes = HessianSerializer.serialize(rpcResponse);
response.setContentType("text/html;charset=utf-8");
response.setStatus(HttpServletResponse.SC_OK);
baseRequest.setHandled(true);
OutputStream out = response.getOutputStream();
out.write(responseBytes);
out.flush();
}
private RpcResponse doInvoke(HttpServletRequest request) {
try {
// deserialize request
byte[] requestBytes = HttpClientUtil.readBytes(request);
if (requestBytes == null || requestBytes.length==0) {
RpcResponse rpcResponse = new RpcResponse();
rpcResponse.setError("RpcRequest byte[] is null");
return rpcResponse;
}
//反序列化数据
RpcRequest rpcRequest = (RpcRequest) HessianSerializer.deserialize(requestBytes, RpcRequest.class);
// invoke
//通过反射调用任务
RpcResponse rpcResponse = NetComServerFactory.invokeService(rpcRequest, null);
return rpcResponse;
} catch (Exception e) {
logger.error(e.getMessage(), e);
RpcResponse rpcResponse = new RpcResponse();
rpcResponse.setError("Server-error:" + e.getMessage());
return rpcResponse;
}
}
}
其中,在invokeService()方法
中,根据发送过来的类名com.xxl.job.core.biz.ExecutorBiz
和方法run()
,通过反射机制调用真正的实现类。
具体代码如下:
public static RpcResponse invokeService(RpcRequest request, Object serviceBean) {
if (serviceBean==null) {
serviceBean = serviceMap.get(request.getClassName());
}
if (serviceBean == null) {
// TODO
}
RpcResponse response = new RpcResponse();
if (System.currentTimeMillis() - request.getCreateMillisTime() > 180000) {
response.setResult(new ReturnT<String>(ReturnT.FAIL_CODE, "The timestamp difference between admin and executor exceeds the limit."));
return response;
}
if (accessToken!=null && accessToken.trim().length()>0 && !accessToken.trim().equals(request.getAccessToken())) {
response.setResult(new ReturnT<String>(ReturnT.FAIL_CODE, "The access token[" + request.getAccessToken() + "] is wrong."));
return response;
}
try {
Class<?> serviceClass = serviceBean.getClass();
String methodName = request.getMethodName();
Class<?>[] parameterTypes = request.getParameterTypes();
Object[] parameters = request.getParameters();
FastClass serviceFastClass = FastClass.create(serviceClass);
FastMethod serviceFastMethod = serviceFastClass.getMethod(methodName, parameterTypes);
Object result = serviceFastMethod.invoke(serviceBean, parameters);
response.setResult(result);
} catch (Throwable t) {
t.printStackTrace();
response.setError(t.getMessage());
}
return response;
}
最后,在com.xxl.job.core.biz.ExecutorBiz类
里的run()
方法中,根据demoJobHandler
找到接口的实现类,接下来就可以新起线程去执行该实现类DemoJobHandler。
具体代码如下:
@Override
public ReturnT<String> run(TriggerParam triggerParam) {
// load old:jobHandler + jobThread
//创建执行线程
JobThread jobThread = XxlJobExecutor.loadJobThread(triggerParam.getJobId());
//如果存在则直接使用老的线程
IJobHandler jobHandler = jobThread!=null?jobThread.getHandler():null;
String removeOldReason = null;
// valid:jobHandler + jobThread
GlueTypeEnum glueTypeEnum = GlueTypeEnum.match(triggerParam.getGlueType());
if (GlueTypeEnum.BEAN == glueTypeEnum) {
// new jobhandler
//根据类名找到任务执行类
IJobHandler newJobHandler = XxlJobExecutor.loadJobHandler(triggerParam.getExecutorHandler());
// valid old jobThread
if (jobThread!=null && jobHandler != newJobHandler) {
// change handler, need kill old thread
removeOldReason = "change jobhandler or glue type, and terminate the old job thread.";
jobThread = null;
jobHandler = null;
}
// valid handler
if (jobHandler == null) {
jobHandler = newJobHandler;
if (jobHandler == null) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "job handler [" + triggerParam.getExecutorHandler() + "] not found.");
}
}
} else if (GlueTypeEnum.GLUE_GROOVY == glueTypeEnum) {
// valid old jobThread
if (jobThread != null &&
!(jobThread.getHandler() instanceof GlueJobHandler
&& ((GlueJobHandler) jobThread.getHandler()).getGlueUpdatetime()==triggerParam.getGlueUpdatetime() )) {
// change handler or gluesource updated, need kill old thread
removeOldReason = "change job source or glue type, and terminate the old job thread.";
jobThread = null;
jobHandler = null;
}
// valid handler
if (jobHandler == null) {
try {
IJobHandler originJobHandler = GlueFactory.getInstance().loadNewInstance(triggerParam.getGlueSource());
jobHandler = new GlueJobHandler(originJobHandler, triggerParam.getGlueUpdatetime());
} catch (Exception e) {
logger.error(e.getMessage(), e);
return new ReturnT<String>(ReturnT.FAIL_CODE, e.getMessage());
}
}
} else if (glueTypeEnum!=null && glueTypeEnum.isScript()) {
// valid old jobThread
if (jobThread != null &&
!(jobThread.getHandler() instanceof ScriptJobHandler
&& ((ScriptJobHandler) jobThread.getHandler()).getGlueUpdatetime()==triggerParam.getGlueUpdatetime() )) {
// change script or gluesource updated, need kill old thread
removeOldReason = "change job source or glue type, and terminate the old job thread.";
jobThread = null;
jobHandler = null;
}
// valid handler
if (jobHandler == null) {
jobHandler = new ScriptJobHandler(triggerParam.getJobId(), triggerParam.getGlueUpdatetime(), triggerParam.getGlueSource(), GlueTypeEnum.match(triggerParam.getGlueType()));
}
} else {
return new ReturnT<String>(ReturnT.FAIL_CODE, "glueType[" + triggerParam.getGlueType() + "] is not valid.");
}
// executor block strategy
if (jobThread != null) {
ExecutorBlockStrategyEnum blockStrategy = ExecutorBlockStrategyEnum.match(triggerParam.getExecutorBlockStrategy(), null);
if (ExecutorBlockStrategyEnum.DISCARD_LATER == blockStrategy) {
// discard when running
if (jobThread.isRunningOrHasQueue()) {
return new ReturnT<String>(ReturnT.FAIL_CODE, "block strategy effect:"+ExecutorBlockStrategyEnum.DISCARD_LATER.getTitle());
}
} else if (ExecutorBlockStrategyEnum.COVER_EARLY == blockStrategy) {
// kill running jobThread
if (jobThread.isRunningOrHasQueue()) {
removeOldReason = "block strategy effect:" + ExecutorBlockStrategyEnum.COVER_EARLY.getTitle();
jobThread = null;
}
} else {
// just queue trigger
}
}
// replace thread (new or exists invalid)
//起线程执行任务
if (jobThread == null) {
jobThread = XxlJobExecutor.registJobThread(triggerParam.getJobId(), jobHandler, removeOldReason);
}
// push data to queue
ReturnT<String> pushResult = jobThread.pushTriggerQueue(triggerParam);
return pushResult;
}
总的来说,任务调度平台和远程执行器通过自研RPC通信模块进行交互,任务调度平台在XxlJobScheduler
中初始化Adminbiz Server和 Executorbiz Client,远程执行器在XxxlJobExecutor
中初始化Adminbiz Client和 Executorbiz Server。
执行器新建一个XxlRpcReferenceBean,通过一个NettyClient发送给任务调度平台信息(AdminBiz类名称、method信息),任务调度平台拿到这个信息之后,执行相应AdminBiz的method,这个是一个RPC通信流程;
任务调度平台新建一个XxlRpcReferenceBean,通过一个NettyClient发送给执行器信息(ExecutorBiz类名称、method信息),执行器拿到这个信息之后,执行相应ExecutorBiz的method,这个是一个RPC通信流程。
具体可以参考大佬技术贴,对RPC总结的简单到位:http://www.heartthinkdo.com/?p=3181#221
后续会随着项目的开展对XXL-JOB有更深入的理解,继续更新细化XXL-JOB源码解析的相关博客。
又在驻场开发高温假可能泡汤了的坚强打工人乔木小姐
2021.07.01