启动项目,控制台打印日志如下:
2020-03-06 13:52:52.188 INFO 18472 — [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=multiStepJob]] launched with the following parameters: [{}]
2020-03-06 13:52:52.222 INFO 18472 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1]
执行步骤一操作。。。
2020-03-06 13:52:52.251 INFO 18472 — [ main] o.s.batch.core.step.AbstractStep : Step: [step1] executed in 29ms
2020-03-06 13:52:52.292 INFO 18472 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step2]
执行步骤二操作。。。
2020-03-06 13:52:52.323 INFO 18472 — [ main] o.s.batch.core.step.AbstractStep : Step: [step2] executed in 30ms
2020-03-06 13:52:52.375 INFO 18472 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step3]
执行步骤三操作。。。
2020-03-06 13:52:52.405 INFO 18472 — [ main] o.s.batch.core.step.AbstractStep : Step: [step3] executed in 29ms
2020-03-06 13:52:52.428 INFO 18472 — [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=multiStepJob]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 231ms
三个步骤依次执行成功。
多个步骤在执行过程中也可以通过上一个步骤的执行状态来决定是否执行下一个步骤,修改上面的代码:
@Component
public class MultiStepJobDemo {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Bean
public Job multiStepJob() {
return jobBuilderFactory.get(“multiStepJob2”)
.start(step1())
.on(ExitStatus.COMPLETED.getExitCode()).to(step2())
.from(step2())
.on(ExitStatus.COMPLETED.getExitCode()).to(step3())
.from(step3()).end()
.build();
}
private Step step1() {
return stepBuilderFactory.get(“step1”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤一操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
private Step step2() {
return stepBuilderFactory.get(“step2”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤二操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
private Step step3() {
return stepBuilderFactory.get(“step3”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤三操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
}
multiStepJob()方法的含义是:multiStepJob2 任务先执行 step1,当 step1 状态为完成时,接着执行 step2,当 step2 的状态为完成时,接着执行 step3。ExitStatus.COMPLETED常量表示任务顺利执行完毕,正常退出,该类还包含以下几种退出状态:
public class ExitStatus implements Serializable, Comparable {
/**
-
Convenient constant value representing unknown state - assumed not
-
continuable.
*/
public static final ExitStatus UNKNOWN = new ExitStatus(“UNKNOWN”);
/**
-
Convenient constant value representing continuable state where processing
-
is still taking place, so no further action is required. Used for
-
asynchronous execution scenarios where the processing is happening in
-
another thread or process and the caller is not required to wait for the
-
result.
*/
public static final ExitStatus EXECUTING = new ExitStatus(“EXECUTING”);
/**
- Convenient constant value representing finished processing.
*/
public static final ExitStatus COMPLETED = new ExitStatus(“COMPLETED”);
/**
-
Convenient constant value representing job that did no processing (e.g.
-
because it was already complete).
*/
public static final ExitStatus NOOP = new ExitStatus(“NOOP”);
/**
- Convenient constant value representing finished processing with an error.
*/
public static final ExitStatus FAILED = new ExitStatus(“FAILED”);
/**
-
Convenient constant value representing finished processing with
-
interrupted status.
*/
public static final ExitStatus STOPPED = new ExitStatus(“STOPPED”);
…
}
启动项目,控制台日志打印如下:
2020-03-06 14:21:49.384 INFO 18745 — [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=multiStepJob2]] launched with the following parameters: [{}]
2020-03-06 14:21:49.427 INFO 18745 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1]
执行步骤一操作。。。
2020-03-06 14:21:49.456 INFO 18745 — [ main] o.s.batch.core.step.AbstractStep : Step: [step1] executed in 29ms
2020-03-06 14:21:49.501 INFO 18745 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step2]
执行步骤二操作。。。
2020-03-06 14:21:49.527 INFO 18745 — [ main] o.s.batch.core.step.AbstractStep : Step: [step2] executed in 26ms
2020-03-06 14:21:49.576 INFO 18745 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step3]
执行步骤三操作。。。
2020-03-06 14:21:49.604 INFO 18745 — [ main] o.s.batch.core.step.AbstractStep : Step: [step3] executed in 28ms
2020-03-06 14:21:49.629 INFO 18745 — [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=multiStepJob2]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 238ms
Flow 的用法
========
Flow 的作用就是可以将多个步骤 Step 组合在一起然后再组装到任务 Job 中。举个 Flow 的例子,在 job 包下新建FlowJobDemo类:
@Component
public class FlowJobDemo {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Bean
public Job flowJob() {
return jobBuilderFactory.get(“flowJob”)
.start(flow())
.next(step3())
.end()
.build();
}
private Step step1() {
return stepBuilderFactory.get(“step1”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤一操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
private Step step2() {
return stepBuilderFactory.get(“step2”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤二操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
private Step step3() {
return stepBuilderFactory.get(“step3”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤三操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
// 创建一个flow对象,包含若干个step
private Flow flow() {
return new FlowBuilder(“flow”)
.start(step1())
.next(step2())
.build();
}
}
上面代码中,我们通过FlowBuilder将 step1 和 step2 组合在一起,创建了一个名为 flow 的 Flow,然后再将其赋给任务 Job。使用 Flow 和 Step 构建 Job 的区别是,Job 流程中包含 Flow 类型的时候需要在build()方法前调用end()方法。
启动程序,控制台日志打印如下:
2020-03-06 14:36:42.621 INFO 18865 — [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=flowJob]] launched with the following parameters: [{}]
2020-03-06 14:36:42.667 INFO 18865 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1]
执行步骤一操作。。。
2020-03-06 14:36:42.697 INFO 18865 — [ main] o.s.batch.core.step.AbstractStep : Step: [step1] executed in 30ms
2020-03-06 14:36:42.744 INFO 18865 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step2]
执行步骤二操作。。。
2020-03-06 14:36:42.771 INFO 18865 — [ main] o.s.batch.core.step.AbstractStep : Step: [step2] executed in 27ms
2020-03-06 14:36:42.824 INFO 18865 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step3]
执行步骤三操作。。。
2020-03-06 14:36:42.850 INFO 18865 — [ main] o.s.batch.core.step.AbstractStep : Step: [step3] executed in 25ms
2020-03-06 14:36:42.874 INFO 18865 — [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=flowJob]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 245ms
并行执行
====
任务中的步骤除了可以串行执行(一个接着一个执行)外,还可以并行执行,并行执行在特定的业务需求下可以提供任务执行效率。
将任务并行化只需两个简单步骤:
-
将步骤 Step 转换为 Flow;
-
任务 Job 中指定并行 Flow。
举个例子,在 job 包下新建SplitJobDemo类:
@Component
public class SplitJobDemo {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Bean
public Job splitJob() {
return jobBuilderFactory.get(“splitJob”)
.start(flow1())
.split(new SimpleAsyncTaskExecutor()).add(flow2())
.end()
.build();
}
private Step step1() {
return stepBuilderFactory.get(“step1”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤一操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
private Step step2() {
return stepBuilderFactory.get(“step2”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤二操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
private Step step3() {
return stepBuilderFactory.get(“step3”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤三操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
private Flow flow1() {
return new FlowBuilder(“flow1”)
.start(step1())
.next(step2())
.build();
}
private Flow flow2() {
return new FlowBuilder(“flow2”)
.start(step3())
.build();
}
}
上面例子中,我们创建了两个 Flow:flow1(包含 step1 和 step2)和 flow2(包含 step3)。然后通过JobBuilderFactory的split方法,指定一个异步执行器,将 flow1 和 flow2 异步执行(也就是并行)。
启动项目,控制台日志打印如下:
2020-03-06 15:25:43.602 INFO 19449 — [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=splitJob]] launched with the following parameters: [{}]
2020-03-06 15:25:43.643 INFO 19449 — [cTaskExecutor-1] o.s.batch.core.job.SimpleStepHandler : Executing step: [step3]
2020-03-06 15:25:43.650 INFO 19449 — [cTaskExecutor-2] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1]
执行步骤三操作。。。
执行步骤一操作。。。
2020-03-06 15:25:43.673 INFO 19449 — [cTaskExecutor-2] o.s.batch.core.step.AbstractStep : Step: [step1] executed in 23ms
2020-03-06 15:25:43.674 INFO 19449 — [cTaskExecutor-1] o.s.batch.core.step.AbstractStep : Step: [step3] executed in 31ms
2020-03-06 15:25:43.714 INFO 19449 — [cTaskExecutor-2] o.s.batch.core.job.SimpleStepHandler : Executing step: [step2]
执行步骤二操作。。。
2020-03-06 15:25:43.738 INFO 19449 — [cTaskExecutor-2] o.s.batch.core.step.AbstractStep : Step: [step2] executed in 24ms
2020-03-06 15:25:43.758 INFO 19449 — [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=splitJob]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 146ms
可以看到 step3 并没有在 step2 后才执行,说明步骤已经是并行化的(开启并行化后,并行的步骤执行顺序并不能 100%确定,因为线程调度具有不确定性)。
任务决策器
=====
决策器的作用就是可以指定程序在不同的情况下运行不同的任务流程,比如今天是周末,则让任务执行 step1 和 step2,如果是工作日,则之心 step1 和 step3。
使用决策器前,我们需要自定义一个决策器的实现。在 cc.mrbird.batch 包下新建 decider 包,然后创建MyDecider类,实现JobExecutionDecider接口:
@Component
public class MyDecider implements JobExecutionDecider {
@Override
public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) {
LocalDate now = LocalDate.now();
DayOfWeek dayOfWeek = now.getDayOfWeek();
if (dayOfWeek == DayOfWeek.SATURDAY || dayOfWeek == DayOfWeek.SUNDAY) {
return new FlowExecutionStatus(“weekend”);
} else {
return new FlowExecutionStatus(“workingDay”);
}
}
}
MyDecider实现JobExecutionDecider接口的decide方法,该方法返回FlowExecutionStatus。上面的逻辑是:判断今天是否是周末,如果是,返回FlowExecutionStatus(“weekend”)状态,否则返回FlowExecutionStatus(“workingDay”)状态。
下面演示如何在任务 Job 里使用决策器。在 job 包下新建DeciderJobDemo:
@Component
public class DeciderJobDemo {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Autowired
private MyDecider myDecider;
@Bean
public Job deciderJob() {
return jobBuilderFactory.get(“deciderJob”)
.start(step1())
.next(myDecider)
.from(myDecider).on(“weekend”).to(step2())
.from(myDecider).on(“workingDay”).to(step3())
.from(step3()).on(“*”).to(step4())
.end()
.build();
}
private Step step1() {
return stepBuilderFactory.get(“step1”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤一操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
private Step step2() {
return stepBuilderFactory.get(“step2”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤二操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
private Step step3() {
return stepBuilderFactory.get(“step3”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤三操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
private Step step4() {
return stepBuilderFactory.get(“step4”)
.tasklet((stepContribution, chunkContext) -> {
System.out.println(“执行步骤四操作。。。”);
return RepeatStatus.FINISHED;
}).build();
}
上面代码中,我们注入了自定义决策器MyDecider,然后在jobDecider()方法里使用了该决策器:
@Bean
public Job deciderJob() {
return jobBuilderFactory.get(“deciderJob”)
.start(step1())
.next(myDecider)
.from(myDecider).on(“weekend”).to(step2())
.from(myDecider).on(“workingDay”).to(step3())
.from(step3()).on(“*”).to(step4())
.end()
.build();
}
这段代码的含义是:任务 deciderJob 首先执行 step1,然后指定自定义决策器,如果决策器返回 weekend,那么执行 step2,如果决策器返回 workingDay,那么执行 step3。如果执行了 step3,那么无论 step3 的结果是什么,都将执行 step4。
启动项目,控制台输出如下所示:
2020-03-06 16:09:10.541 INFO 19873 — [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=deciderJob]] launched with the following parameters: [{}]
2020-03-06 16:09:10.609 INFO 19873 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1]
执行步骤一操作。。。
2020-03-06 16:09:10.641 INFO 19873 — [ main] o.s.batch.core.step.AbstractStep : Step: [step1] executed in 32ms
2020-03-06 16:09:10.692 INFO 19873 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step3]
执行步骤三操作。。。
2020-03-06 16:09:10.723 INFO 19873 — [ main] o.s.batch.core.step.AbstractStep : Step: [step3] executed in 31ms
2020-03-06 16:09:10.769 INFO 19873 — [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step4]
执行步骤四操作。。。
2020-03-06 16:09:10.797 INFO 19873 — [ main] o.s.batch.core.step.AbstractStep : Step: [step4] executed in 27ms
2020-03-06 16:09:10.818 INFO 19873 — [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=deciderJob]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 256ms
因为今天是 2020 年 03 月 06 日星期五,是工作日,所以任务执行了 step1、step3 和 step4。
任务嵌套
====
任务 Job 除了可以由 Step 或者 Flow 构成外,我们还可以将多个任务 Job 转换为特殊的 Step,然后再赋给另一个任务 Job,这就是任务的嵌套。
举个例子,在 job 包下新建NestedJobDemo类:
@Component
public class NestedJobDemo {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Autowired
private JobLauncher jobLauncher;
@Autowired
private JobRepository jobRepository;
@Autowired
private PlatformTransactionManager platformTransactionManager;
// 父任务
@Bean
public Job parentJob() {
return jobBuilderFactory.get(“parentJob”)
.start(childJobOneStep())
.next(childJobTwoStep())
.build();
}
// 将任务转换为特殊的步骤
private Step childJobOneStep() {
return new JobStepBuilder(new StepBuilder(“childJobOneStep”))
.job(childJobOne())
.launcher(jobLauncher)
.repository(jobRepository)
.transactionManager(platformTransactionManager)
.build();
}
// 将任务转换为特殊的步骤
private Step childJobTwoStep() {
自我介绍一下,小编13年上海交大毕业,曾经在小公司待过,也去过华为、OPPO等大厂,18年进入阿里一直到现在。
深知大多数Java工程师,想要提升技能,往往是自己摸索成长或者是报班学习,但对于培训机构动则几千的学费,着实压力不小。自己不成体系的自学效果低效又漫长,而且极易碰到天花板技术停滞不前!
因此收集整理了一份《2024年Java开发全套学习资料》,初衷也很简单,就是希望能够帮助到想自学提升又不知道该从何学起的朋友,同时减轻大家的负担。
既有适合小白学习的零基础资料,也有适合3年以上经验的小伙伴深入学习提升的进阶课程,基本涵盖了95%以上Java开发知识点,真正体系化!
由于文件比较大,这里只是将部分目录截图出来,每个节点里面都包含大厂面经、学习笔记、源码讲义、实战项目、讲解视频,并且会持续更新!
如果你觉得这些内容对你有帮助,可以扫码获取!!(备注Java获取)
![img](https://i-blog.csdnimg.cn/blog_migrate/f59bc7cbb0c8feb6d6bf86dd1b48066b.jpeg)
总结:绘上一张Kakfa架构思维大纲脑图(xmind)
其实关于Kafka,能问的问题实在是太多了,扒了几天,最终筛选出44问:基础篇17问、进阶篇15问、高级篇12问,个个直戳痛点,不知道如果你不着急看答案,又能答出几个呢?
若是对Kafka的知识还回忆不起来,不妨先看我手绘的知识总结脑图(xmind不能上传,文章里用的是图片版)进行整体架构的梳理
梳理了知识,刷完了面试,如若你还想进一步的深入学习解读kafka以及源码,那么接下来的这份《手写“kafka”》将会是个不错的选择。
-
Kafka入门
-
为什么选择Kafka
-
Kafka的安装、管理和配置
-
Kafka的集群
-
第一个Kafka程序
-
Kafka的生产者
-
Kafka的消费者
-
深入理解Kafka
-
可靠的数据传递
-
Spring和Kafka的整合
-
SpringBoot和Kafka的整合
-
Kafka实战之削峰填谷
-
数据管道和流式处理(了解即可)
《一线大厂Java面试题解析+核心总结学习笔记+最新讲解视频+实战项目源码》,点击传送门即可获取!
95%以上Java开发知识点,真正体系化!**
由于文件比较大,这里只是将部分目录截图出来,每个节点里面都包含大厂面经、学习笔记、源码讲义、实战项目、讲解视频,并且会持续更新!
如果你觉得这些内容对你有帮助,可以扫码获取!!(备注Java获取)
![img](https://i-blog.csdnimg.cn/blog_migrate/f59bc7cbb0c8feb6d6bf86dd1b48066b.jpeg)
总结:绘上一张Kakfa架构思维大纲脑图(xmind)
[外链图片转存中…(img-D4Gs5xYH-1711819292397)]
其实关于Kafka,能问的问题实在是太多了,扒了几天,最终筛选出44问:基础篇17问、进阶篇15问、高级篇12问,个个直戳痛点,不知道如果你不着急看答案,又能答出几个呢?
若是对Kafka的知识还回忆不起来,不妨先看我手绘的知识总结脑图(xmind不能上传,文章里用的是图片版)进行整体架构的梳理
梳理了知识,刷完了面试,如若你还想进一步的深入学习解读kafka以及源码,那么接下来的这份《手写“kafka”》将会是个不错的选择。
-
Kafka入门
-
为什么选择Kafka
-
Kafka的安装、管理和配置
-
Kafka的集群
-
第一个Kafka程序
-
Kafka的生产者
-
Kafka的消费者
-
深入理解Kafka
-
可靠的数据传递
-
Spring和Kafka的整合
-
SpringBoot和Kafka的整合
-
Kafka实战之削峰填谷
-
数据管道和流式处理(了解即可)
[外链图片转存中…(img-C0R3yF2l-1711819292397)]
[外链图片转存中…(img-YfHTdZtX-1711819292398)]
《一线大厂Java面试题解析+核心总结学习笔记+最新讲解视频+实战项目源码》,点击传送门即可获取!