分布式任务调度系统chaconne介绍，开源新作

log.info(“DemoDependentJob is running at: {}”, DateUtils.format(System.currentTimeMillis()));

return RandomUtils.randomLong(1000000L, 1000000000L);

}

@OnSuccess

public void onSuccess(JobKey jobKey, Object result, Logger log) {

log.info(“DemoDependentJob’s return value is: {}”, result);

}

@OnFailure

public void onFailure(JobKey jobKey, Throwable e, Logger log) {

log.error(“DemoDependentJob is failed by cause: {}”, e.getMessage(), e);

}

}

并行依赖示例：

有3个任务，DemoTask, DemoTaskOne, DemoTaskTwo

让DemoTaskOne, DemoTaskTwo都做完再执行DemoTask，且DemoTask可以获得DemoTaskOne, DemoTaskTwo执行后的值

DemoTaskOne:

@ChacJob

@ChacTrigger(triggerType = TriggerType.SIMPLE)

public class DemoTaskOne {

@Run

public Object execute(JobKey jobKey, Object attachment, Logger log) throws Exception {

log.info(“DemoTaskOne is running at: {}”, DateUtils.format(System.currentTimeMillis()));

return RandomUtils.randomLong(1000000L, 1000000000L);

}

@OnSuccess

public void onSuccess(JobKey jobKey, Object result, Logger log) {

log.info(“DemoTaskOne return value is: {}”, result);

}

@OnFailure

public void onFailure(JobKey jobKey, Throwable e, Logger log) {

log.error(“DemoTaskOne is failed by cause: {}”, e.getMessage(), e);

}

}

DemoTaskTwo:

@ChacJob

@ChacTrigger(triggerType = TriggerType.SIMPLE)

public class DemoTaskTwo {

@Run

public Object execute(JobKey jobKey, Object attachment, Logger log) throws Exception {

log.info(“DemoTaskTwo is running at: {}”, DateUtils.format(System.currentTimeMillis()));

return RandomUtils.randomLong(1000000L, 1000000000L);

}

@OnSuccess

public void onSuccess(JobKey jobKey, Object result, Logger log) {

log.info(“DemoTaskTwo return value is: {}”, result);

}

@OnFailure

public void onFailure(JobKey jobKey, Throwable e, Logger log) {

log.error(“DemoTaskTwo is failed by cause: {}”, e.getMessage(), e);

}

}

DemoTask:

@ChacJob

@ChacTrigger(cron = “0 0/1 * * * ?”, triggerType = TriggerType.CRON)

@ChacFork({ @ChacJobKey(className = “com.chinapex.test.chaconne.job.DemoTaskOne”, name = “demoTaskOne”),

@ChacJobKey(className = “com.chinapex.test.chaconne.job.DemoTaskTwo”, name = “demoTaskTwo”) })

public class DemoTask {

@Run

public Object execute(JobKey jobKey, Object attachment, Logger log) throws Exception {

log.info(“DemoTask is running at: {}”, DateUtils.format(System.currentTimeMillis()));

TaskJoinResult joinResult = (TaskJoinResult) attachment;

TaskForkResult[] forkResults = joinResult.getTaskForkResults();

long max = 0;

for (TaskForkResult forkResult : forkResults) {

max = Long.max(max, (Long) forkResult.getResult());

}

return max;

}

@OnSuccess

public void onSuccess(JobKey jobKey, Object result, Logger log) {

log.info(“DemoTask return max value is: {}”, result);

}

@OnFailure

public void onFailure(JobKey jobKey, Throwable e, Logger log) {

log.error(“DemoTask is failed by cause: {}”, e.getMessage(), e);

}

}

DAG任务示例

DAG任务目前只支持API创建, 后续会持续改进，增加界面方式创建DAG任务

@RequestMapping(“/dag”)

@RestController

public class DagJobController {

@Value(“${spring.application.cluster.name}”)

private String clusterName;

@Value(“${spring.application.name}”)

private String applicationName;

@Autowired

private JobManager jobManager;

@GetMapping(“/create”)

public Map<String, Object> createDagTask() throws Exception {

Dag dag = new Dag(clusterName, applicationName, “testDag”);// 创建一个Dag任务，并指定集群名，应用名，和任务名称

dag.setTrigger(new CronTrigger(“0 0/1 * * * ?”));// 设置Cron表达式

dag.setDescription(“This is only a demo of dag job”);// 任务描述

DagFlow first = dag.startWith(clusterName, applicationName, “demoDagStart”, DemoDagStart.class.getName());// 定义第一个节点

DagFlow second = first.flow(clusterName, applicationName, “demoDag”, DemoDag.class.getName());// 定义第二个节点

// 第二个节点fork两个子进程处理

second.fork(clusterName, applicationName, “demoDagOne”, DemoDagOne.class.getName());

second.fork(clusterName, applicationName, “demoDagTwo”, DemoDagTwo.class.getName());

jobManager.persistJob(dag, “123”);

return Collections.singletonMap(“ok”, 1);

}

}

上面的DAG示例说明一下，chaconne框架提供的DAG模型支持串行流入，即flow模式，也提供了fork模式进行并行处理，上例中，任务demoDag fork了两个子进程（“demoDagOne”和“demoDagTwo”），即demoDagOne和demoDagTwo同时处理完了再触发demoDag任务。

Chaconne部署说明

chaconne除了依托SpringBoot框架外，默认用MySQL存储任务信息(目前仅支持MySQL，后续会支持更多类型的数据库), 用Redis保存集群元数据和进行消息广播

所以无论使用哪种部署方式，你都需要在你的应用中设置DataSource和RedisConnectionFactory

示例代码：

@Slf4j

@Configuration

public class ResourceConfig {

@Setter

@Configuration(proxyBeanMethods = false)

@ConfigurationProperties(prefix = “spring.datasource”)

public static class DataSourceConfig {

private String jdbcUrl;

private String username;

private String password;

private String driverClassName;

private HikariConfig getDbConfig() {

if (log.isTraceEnabled()) {

log.trace("DataSourceConfig JdbcUrl: " + jdbcUrl);

log.trace("DataSourceConfig Username: " + username);

log.trace("DataSourceConfig Password: " + password);

log.trace("DataSourceConfig DriverClassName: " + driverClassName);

}

final HikariConfig config = new HikariConfig();

config.setDriverClassName(driverClassName);

config.setJdbcUrl(jdbcUrl);

config.setUsername(username);

config.setPassword(password);

config.setMinimumIdle(5);

config.setMaximumPoolSize(50);

config.setMaxLifetime(60 * 1000);

config.setIdleTimeout(60 * 1000);

config.setValidationTimeout(3000);

config.setReadOnly(false);

config.setConnectionInitSql(“SELECT UUID()”);

config.setConnectionTestQuery(“SELECT 1”);

config.setConnectionTimeout(60 * 1000);

config.setTransactionIsolation(“TRANSACTION_READ_COMMITTED”);

config.addDataSourceProperty(“cachePrepStmts”, “true”);

config.addDataSourceProperty(“prepStmtCacheSize”, “250”);

config.addDataSourceProperty(“prepStmtCacheSqlLimit”, “2048”);

return config;

}

@Primary

@Bean

public DataSource dataSource() {

return new HikariDataSource(getDbConfig());

}

}

@Setter

@Configuration(proxyBeanMethods = false)

@ConfigurationProperties(prefix = “spring.redis”)

public static class RedisConfig {

private String host;

private String password;

private int port;

private int dbIndex;

@Bean

@ConditionalOnMissingBean(RedisConnectionFactory.class)

public RedisConnectionFactory redisConnectionFactory() {

RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration();

redisStandaloneConfiguration.setHostName(host);

redisStandaloneConfiguration.setPort(port);

redisStandaloneConfiguration.setDatabase(dbIndex);

redisStandaloneConfiguration.setPassword(RedisPassword.of(password));

JedisClientConfiguration.JedisClientConfigurationBuilder jedisClientConfiguration = JedisClientConfiguration.builder();

jedisClientConfiguration.connectTimeout(Duration.ofMillis(60000)).readTimeout(Duration.ofMillis(60000)).usePooling()

.poolConfig(jedisPoolConfig());

JedisConnectionFactory factory = new JedisConnectionFactory(redisStandaloneConfiguration, jedisClientConfiguration.build());

return factory;

}

@Bean

public JedisPoolConfig jedisPoolConfig() {

JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();

jedisPoolConfig.setMinIdle(1);

jedisPoolConfig.setMaxIdle(10);

jedisPoolConfig.setMaxTotal(200);

jedisPoolConfig.setMaxWaitMillis(-1);

jedisPoolConfig.setTestWhileIdle(true);

return jedisPoolConfig;

}

}

}

Chaconne去中心化部署

在你的Spring应用程序的主函数上加上@EnableChaconneEmbeddedMode注解，然后启动

示例代码：

@EnableChaconneEmbeddedMode

@SpringBootApplication

@ComponentScan

public class YourApplicationMain {

public static void main(String[] args) {

final int port = 8088;

System.setProperty(“server.port”, String.valueOf(port));

SpringApplication.run(YourApplicationMain.class, args);

}

小编13年上海交大毕业，曾经在小公司待过，也去过华为、OPPO等大厂，18年进入阿里一直到现在。

深知大多数初中级Java工程师，想要提升技能，往往是自己摸索成长，但自己不成体系的自学效果低效又漫长，而且极易碰到天花板技术停滞不前！

因此收集整理了一份《2024年最新Java开发全套学习资料》送给大家，初衷也很简单，就是希望能够帮助到想自学提升又不知道该从何学起的朋友，同时减轻大家的负担。

由于文件比较大，这里只是将部分目录截图出来，每个节点里面都包含大厂面经、学习笔记、源码讲义、实战项目、讲解视频

如果你觉得这些内容对你有帮助，可以添加下面V无偿领取！（备注Java）

最后

直到现在。**

深知大多数初中级Java工程师，想要提升技能，往往是自己摸索成长，但自己不成体系的自学效果低效又漫长，而且极易碰到天花板技术停滞不前！

因此收集整理了一份《2024年最新Java开发全套学习资料》送给大家，初衷也很简单，就是希望能够帮助到想自学提升又不知道该从何学起的朋友，同时减轻大家的负担。
[外链图片转存中…(img-9UN1BVBf-1711171468279)]
[外链图片转存中…(img-V05GIGPL-1711171468280)]
[外链图片转存中…(img-bRwNHsWi-1711171468281)]

由于文件比较大，这里只是将部分目录截图出来，每个节点里面都包含大厂面经、学习笔记、源码讲义、实战项目、讲解视频

如果你觉得这些内容对你有帮助，可以添加下面V无偿领取！（备注Java）
[外链图片转存中…(img-1ZlHDdxM-1711171468282)]

最后

[外链图片转存中…(img-34YyoUSp-1711171468282)]

[外链图片转存中…(img-4hcOM2Hw-1711171468283)]

本文已被CODING开源项目：【一线大厂Java面试题解析+核心总结学习笔记+最新讲解视频+实战项目源码】收录