本文解析如何在Spring Batch中利用skip-limit、skippable-exception-classes和自定义skip-policy来优雅处理对账单中的错误记录，确保作业健壮性并记录异常以便后续单独处理。
设想的信用卡对账单的处理业务场景，银行每天需要处理海量的对账文件，如果对账文件中有少量的一行或者几行错误格式的记录，在真正进行业务处理的时候，不希望因为几行错误的记录而导致整个作业的失败；而是希望这几行没有处理的记录跳过去，让整个Job正确执行，对于错误的记录则通过日志的方式记录下来后续进行单独的处理。
- skippable-exception-classes：定义可以对记录跳过的异常，还可以定义跳过一组异常，如果发生了定义的异常或者子类异常都不会导致作业失败。
- skip-limmit：任务处理发生异常时，允许跳过的最大次数。默认使用的跳过策略为LimitCheckingItemSkipPolicy。
- skip-policy：当默认的按照次数跳过策略不能满足需求时，可以配置自定义跳过策略，不过需要实现接口org.springframework.batch.core.step.skip.SkipPolicy。如果添加了skip-policy属性之后，skip-limmit和skippable-exception-classes默认策略将不起作用。
<batch:job id="skipJob2">
<batch:step id="skipStep2">
<batch:tasklet transaction-manager="transactionManager">
<batch:chunk reader="autoReader" processor="autoProcessor" writer="autoWriter" commit-interval="4" skip-limit="5" skip-policy="mySkipException">
<batch:skippable-exception-classes>
<batch:include class="java.lang.RuntimeException"/>
</batch:skippable-exception-classes>
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
1.项目框架
2.代码实现
AutoProcessor.java:
package com.xj.demo14;
import org.springframework.batch.item.ItemProcessor;
import java.util.Random;
public class AutoProcessor implements ItemProcessor<String, String> {
Random r = new Random();
@Override
public String process(String item) throws Exception {
int i = r.nextInt(10);
System.out.println("Processor "+ item + "; Random i=" + i);
if(i%2 == 0){
throw new RuntimeException("make error");
}else{
return item;
}
}
}
AutoReader.java:
package com.xj.demo14;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;
public class AutoReader implements ItemReader<String> {
private int count = 0;
private int maxCount = 30;
@Override
public String read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
if(count > maxCount){
return null;
}else{
count++;
System.out.println("reader--->" + count);
return count + "";
}
}
}
AutoWriter.java:
package com.xj.demo14;
import org.springframework.batch.item.ItemWriter;
import java.util.List;
public class AutoWriter implements ItemWriter<String> {
@Override
public void write(List<? extends String> list) throws Exception {
System.out.println("Writer Begin!");
for(String item : list){
System.out.println("Writer --->" + item);
}
System.out.println("Writer End!");
}
}
Demo14BatchMain.java:
package com.xj.demo14;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
public class Demo14BatchMain {
public static void main(String[] args) {
Demo14BatchMain batchMain = new Demo14BatchMain();
//测试skip-limit和skippable-exception-classes组合
batchMain.executeJob("demo14/job/demo14-job.xml", "skipJob1", "jobLauncher", new JobParametersBuilder().toJobParameters());
//测试skip-policy
//batchMain.executeJob("demo14/job/demo14-job.xml", "skipJob2", "jobLauncher", new JobParametersBuilder().toJobParameters());
}
/**
*执行Job
* @param jobXmlPath 配置job的xml文件路径
* @param jobId job的id
* @param jobLauncherId jobLauncher的id
* @param jobParameters 参数
*/
public void executeJob(String jobXmlPath, String jobId, String jobLauncherId, JobParameters jobParameters){
ApplicationContext context = new ClassPathXmlApplicationContext(jobXmlPath);
JobLauncher jobLauncher = (JobLauncher) context.getBean(jobLauncherId);
//获取要执行的Job
Job job = (Job)context.getBean(jobId);
try{
//开始执行作业Job
JobExecution jobExecution = jobLauncher.run(job, jobParameters);
//输出执行结果
System.out.println(jobExecution.toString());
}catch (Exception e){
e.printStackTrace();
}
}
}
MySkipException.java:
package com.xj.demo14;
import org.springframework.batch.core.step.skip.SkipLimitExceededException;
import org.springframework.batch.core.step.skip.SkipPolicy;
public class MySkipException implements SkipPolicy {
@Override
public boolean shouldSkip(Throwable throwable, int i) throws SkipLimitExceededException {
//定义了RuntimeException和NullPointerException可以跳过
if(throwable instanceof RuntimeException && throwable.getClass().getName() == RuntimeException.class.getName()){
System.out.println(throwable.getClass().getName() + "跳过");
return true;
}else if(throwable instanceof NullPointerException && throwable.getClass().getName() == NullPointerException.class.getName()){
System.out.println(throwable.getClass().getName() + "跳过");
return true;
}
return false;
}
}
demo14-job.xml:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:batch="http://www.springframework.org/schema/batch"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/batch
http://www.springframework.org/schema/batch/spring-batch.xsd">
<import resource="classpath:demo14/job/demo14-jobContext.xml"/>
<!--测试skip-limit和skippable-exception-classes组合-->
<batch:job id="skipJob1">
<batch:step id="skipStep1">
<batch:tasklet transaction-manager="transactionManager">
<batch:chunk reader="autoReader" processor="autoProcessor" writer="autoWriter" commit-interval="4" skip-limit="5">
<batch:skippable-exception-classes>
<batch:include class="java.lang.RuntimeException"/>
</batch:skippable-exception-classes>
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<!--测试skip-policy-->
<batch:job id="skipJob2">
<batch:step id="skipStep2">
<batch:tasklet transaction-manager="transactionManager">
<batch:chunk reader="autoReader" processor="autoProcessor" writer="autoWriter" commit-interval="4" skip-limit="5" skip-policy="mySkipException">
<batch:skippable-exception-classes>
<batch:include class="java.lang.RuntimeException"/>
</batch:skippable-exception-classes>
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
</beans>demo14-jobContext.xml:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"/>
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository"/>
</bean>
<bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager"/>
<!--定义读方法-->
<bean id="autoReader" class="com.xj.demo14.AutoReader"/>
<!--定义处理方法-->
<bean id="autoProcessor" class="com.xj.demo14.AutoProcessor"/>
<!--定义写方法-->
<bean id="autoWriter" class="com.xj.demo14.AutoWriter"/>
<!--自定义跳过策略-->
<bean id="mySkipException" class="com.xj.demo14.MySkipException"/>
</beans>3.运行结果
(1)测试skip-limit和skippable-exception-classes组合
这里需要说明的是，不是配置skip-limit=5，就一定是只能跳过5次，而是至少能跳过5次。具体跳过次数需要结合commit-interval这个属性。比如上面的程序。我们配置的commit-interval= 4，就意味着read 4次，然后processor 4次，最后writer 1次。而read、processor和writer都是一个整体，当从一个整体进入下一个整体的时候才会去校验跳过异常的次数是否超过了skip-limit，如果超过了，就抛出异常，停止Job；否则进入下一个整体。上面的运行结果就说明了这个问题。第一次reader整体送{1,2,3,4}进入processor整体,而在processor整体中{1,2,3,4}触发了4次异常，跳过了4次，小于等于skip-limit=5，则可以进入writer整体。第二次reader整体送{5,6,7,8}进入processor整体，在processor整体中{5,8,7}一共触发了3次异常，只有{6}有机会进入writer整体。当processor送{6}进入writer整体中时，综合判断已经触发了4+3=7次异常，大于skip-limit=5，所以抛出异常，停止Job，最终{6}也没有进入writer整体中。
(2)测试skip-policy
当配置了skip-policy属性之后，即使skip-limmit和skippable-exception-classes策略也同时配置了，但是skip-limmit和skippable-exception-classes策略不会起作用，可以看到即使跳过异常次数已经超过5次了，依然可以继续执行下去，因为最终走的是自定义跳过策略MySkipException。
参考：
扫一扫
1923
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
