1. 利用Spring进行配置,配置自己的分布式作业的命名空间处理器,这里主要以simple为主,前提还是要配置META-INF/spring.handlers和spring.schemas
public final class JobNamespaceHandler extends NamespaceHandlerSupport {
@Override
public void init() {
registerBeanDefinitionParser("simple", new SimpleJobBeanDefinitionParser());
registerBeanDefinitionParser("dataflow", new DataflowJobBeanDefinitionParser());
registerBeanDefinitionParser("script", new ScriptJobBeanDefinitionParser());
}
}
开始解析xml文件,主要解析代码在AbstractJobBeanDefinitionParser,主要解析类SpringJobScheduler构造函数的属性。属性分别为我们主要实现的job处理器,注册中心即zk,job的配置文件,job记录事件配置,job监听器
public SpringJobScheduler(final ElasticJob elasticJob, final CoordinatorRegistryCenter regCenter, final LiteJobConfiguration jobConfig,
final JobEventConfiguration jobEventConfig, final ElasticJobListener... elasticJobListeners) {
super(regCenter, jobConfig, jobEventConfig, getTargetElasticJobListeners(elasticJobListeners));
this.elasticJob = elasticJob;
}
2. 实例化JobScheduler,先在任务注册类中注册该类的实例信息,设置监听器的保障服务类
private JobScheduler(final CoordinatorRegistryCenter regCenter, final LiteJobConfiguration liteJobConfig, final JobEventBus jobEventBus, final ElasticJobListener... elasticJobListeners) {
JobRegistry.getInstance().addJobInstance(liteJobConfig.getJobName(), new JobInstance());
this.liteJobConfig = liteJobConfig;
this.regCenter = regCenter;
List<ElasticJobListener> elasticJobListenerList = Arrays.asList(elasticJobListeners);
setGuaranteeServiceForElasticJobListeners(regCenter, elasticJobListenerList);
schedulerFacade = new SchedulerFacade(regCenter, liteJobConfig.getJobName(), elasticJobListenerList);
jobFacade = new LiteJobFacade(regCenter, liteJobConfig.getJobName(), Arrays.asList(elasticJobListeners), jobEventBus);
}
实例化调度程序的门面类SchedulerFacade,里面会初始化各种作用的类,配置服务类,主节点服务类,服务节点类,实例服务类,分片服务类,执行服务类,监控服务类,调解分布式作业不一致状态服务类,各种监听器管理类,
public SchedulerFacade(final CoordinatorRegistryCenter regCenter, final String jobName, final List<ElasticJobListener> elasticJobListeners) {
this.jobName = jobName;
configService = new ConfigurationService(regCenter, jobName);
leaderService = new LeaderService(regCenter, jobName);
serverService = new ServerService(regCenter, jobName);
instanceService = new InstanceService(regCenter, jobName);
shardingService = new ShardingService(regCenter, jobName);
executionService = new ExecutionService(regCenter, jobName);
monitorService = new MonitorService(regCenter, jobName);
reconcileService = new ReconcileService(regCenter, jobName);
listenerManager = new ListenerManager(regCenter, jobName, elasticJobListeners);
}
实例化任务门面类LiteJobFacade,和SchedulerFacade的内部属性类似
public LiteJobFacade(final CoordinatorRegistryCenter regCenter, final String jobName, final List<ElasticJobListener> elasticJobListeners, final JobEventBus jobEventBus) {
configService = new ConfigurationService(regCenter, jobName);
shardingService = new ShardingService(regCenter, jobName);
executionContextService = new ExecutionContextService(regCenter, jobName);
executionService = new ExecutionService(regCenter, jobName);
failoverService = new FailoverService(regCenter, jobName);
this.elasticJobListeners = elasticJobListeners;
this.jobEventBus = jobEventBus;
}
增加了发布订阅组件JobEventBus,主要是通过开源框架guava的EventBus来实现。ExecutorServiceObject是个线程池,
public JobEventBus(final JobEventConfiguration jobEventConfig) {
this.jobEventConfig = jobEventConfig;
executorServiceObject = new ExecutorServiceObject("job-event", Runtime.getRuntime().availableProcessors() * 2);
eventBus = new AsyncEventBus(executorServiceObject.createExecutorService());
register();
}
private void register() {
try {
eventBus.register(jobEventConfig.createJobEventListener());
isRegistered = true;
} catch (final JobEventListenerConfigurationException ex) {
log.error("Elastic job: create JobEventListener failure, error is: ", ex);
}
}
jobEventConfig为JobEventRdbConfiguration,创建事件监听器JobEventRdbListener,JobEventRdbStorage会初始化表。
public final class JobEventRdbListener extends JobEventRdbIdentity implements JobEventListener {
private final JobEventRdbStorage repository;
public JobEventRdbListener(final DataSource dataSource) throws SQLException {
repository = new JobEventRdbStorage(dataSource);
}
@Override
public void listen(final JobExecutionEvent executionEvent) {
repository.addJobExecutionEvent(executionEvent);
}
@Override
public void listen(final JobStatusTraceEvent jobStatusTraceEvent) {
repository.addJobStatusTraceEvent(jobStatusTraceEvent);
}
}
实现的接口是JobEventListener,EventBus主要是根据注解来解析@Subscribe@AllowConcurrentEvents匹配对应的方法。
public interface JobEventListener extends JobEventIdentity {
/**
* 作业执行事件监听执行.
*
* @param jobExecutionEvent 作业执行事件
*/
@Subscribe
@AllowConcurrentEvents
void listen(JobExecutionEvent jobExecutionEvent);
/**
* 作业状态痕迹事件监听执行.
*
* @param jobStatusTraceEvent 作业状态痕迹事件
*/
@Subscribe
@AllowConcurrentEvents
void listen(JobStatusTraceEvent jobStatusTraceEvent);
}
3. 因为在解析xml文件的时候factory.setInitMethodName("init");设置了init方法。实例化完成后会执行该方法。
/**
* 初始化作业.
*/
public void init() {
LiteJobConfiguration liteJobConfigFromRegCenter = schedulerFacade.updateJobConfiguration(liteJobConfig);
JobRegistry.getInstance().setCurrentShardingTotalCount(liteJobConfigFromRegCenter.getJobName(), liteJobConfigFromRegCenter.getTypeConfig().getCoreConfig().getShardingTotalCount());
JobScheduleController jobScheduleController = new JobScheduleController(
createScheduler(), createJobDetail(liteJobConfigFromRegCenter.getTypeConfig().getJobClass()), liteJobConfigFromRegCenter.getJobName());
JobRegistry.getInstance().registerJob(liteJobConfigFromRegCenter.getJobName(), jobScheduleController, regCenter);
schedulerFacade.registerStartUpInfo(!liteJobConfigFromRegCenter.isDisabled());
jobScheduleController.scheduleJob(liteJobConfigFromRegCenter.getTypeConfig().getCoreConfig().getCron());
}
首先把任务配置存储到zk注册中心,具体流程是先检查该节点是否存在,节点的数据是否存在,节点数据的执行类是否一致,配置是否允许重写,最后选择是否替换为新的配置。所有的节点的父节点都是jobName。最后从注册中心中查找最新的配置信息。
public LiteJobConfiguration updateJobConfiguration(final LiteJobConfiguration liteJobConfig) {
configService.persist(liteJobConfig);
return configService.load(false);
}
public LiteJobConfiguration load(final boolean fromCache) {
String result;
if (fromCache) {
result = jobNodeStorage.getJobNodeData(ConfigurationNode.ROOT);
if (null == result) {
result = jobNodeStorage.getJobNodeDataDirectly(ConfigurationNode.ROOT);
}
} else {
result = jobNodeStorage.getJobNodeDataDirectly(ConfigurationNode.ROOT);
}
return LiteJobConfigurationGsonFactory.fromJson(result);
}
public void persist(final LiteJobConfiguration liteJobConfig) {
checkConflictJob(liteJobConfig);
if (!jobNodeStorage.isJobNodeExisted(ConfigurationNode.ROOT) || liteJobConfig.isOverwrite()) {
jobNodeStorage.replaceJobNode(ConfigurationNode.ROOT, LiteJobConfigurationGsonFactory.toJson(liteJobConfig));
}
}
private void checkConflictJob(final LiteJobConfiguration liteJobConfig) {
Optional<LiteJobConfiguration> liteJobConfigFromZk = find();
if (liteJobConfigFromZk.isPresent() && !liteJobConfigFromZk.get().getTypeConfig().getJobClass().equals(liteJobConfig.getTypeConfig().getJobClass())) {
throw new JobConfigurationException("Job conflict with register center. The job '%s' in register center's class is '%s', your job class is '%s'",
liteJobConfig.getJobName(), liteJobConfigFromZk.get().getTypeConfig().getJobClass(), liteJobConfig.getTypeConfig().getJobClass());
}
}
private Optional<LiteJobConfiguration> find() {
if (!jobNodeStorage.isJobNodeExisted(ConfigurationNode.ROOT)) {
return Optional.absent();
}
LiteJobConfiguration result = LiteJobConfigurationGsonFactory.fromJson(jobNodeStorage.getJobNodeDataDirectly(ConfigurationNode.ROOT));
if (null == result) {
// TODO 应该删除整个job node,并非仅仅删除config node
jobNodeStorage.removeJobNodeIfExisted(ConfigurationNode.ROOT);
}
return Optional.fromNullable(result);
}
在任务注册类中注册该任务的分片数量。