SpringBatch系列文章–实战例子–文件读写入库。
代码位置：
链接：https://pan.baidu.com/s/1B-Aip8_1CI3Xv87BPidE3g 提取码：1234
代码说明
参考官网例子，使用SpringBoot2集成SpringBatch。
创建数据库表
springbatch的参数表已经创建好，不再重复创建了。
SET FOREIGN_KEY_CHECKS=0;
-- ----------------------------
-- Table structure for PEOPLE
-- ----------------------------
DROP TABLE IF EXISTS `PEOPLE`;
CREATE TABLE `PEOPLE` (
`person_id` bigint NOT NULL AUTO_INCREMENT,
`first_name` varchar(20) DEFAULT NULL,
`last_name` varchar(20) DEFAULT NULL,
PRIMARY KEY (`person_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
工程创建
创建一个springboot的基础工程，选择springbatch、jdbc、mybatis的依赖（本例子暂时没用mybatis）。
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.2.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.ycxy</groupId>
<artifactId>batchstudy04</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>batchstudy04</name>
<description>job</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jdbc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
批处理代码编写
拷贝官网的实体Person，就两个属性 private String lastName;
private String firstName;
public class Person {
private String lastName;
private String firstName;
public Person() {
}
public Person(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}
public void setFirstName(String firstName) {
this.firstName = firstName;
}
public String getFirstName() {
return firstName;
}
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
@Override
public String toString() {
return "firstName: " + firstName + ", lastName: " + lastName;
}
}
编写Job配置
import com.ycxy.batchstudy04.entity.Person;
import com.ycxy.batchstudy04.process.PersonItemProcessor;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider;
import org.springframework.batch.item.database.JdbcBatchItemWriter;
import org.springframework.batch.item.database.builder.JdbcBatchItemWriterBuilder;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import javax.sql.DataSource;
@Configuration
public class BatchJoba {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Autowired
DataSource dataSource;
@Bean
public FlatFileItemReader<Person> reader() {
return new FlatFileItemReaderBuilder<Person>()
.name("personItemReader")
.resource(new ClassPathResource("sample-data.csv"))
.delimited()
.names(new String[]{"firstName", "lastName"})
.fieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {{
setTargetType(Person.class);
}})
.build();
}
@Bean
public PersonItemProcessor processor() {
return new PersonItemProcessor();
}
@Bean
public JdbcBatchItemWriter<Person> writer() {
return new JdbcBatchItemWriterBuilder<Person>()
.itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
.sql("INSERT INTO PEOPLE (first_name, last_name) VALUES (:firstName, :lastName)")
.dataSource(dataSource)
.build();
}
@Bean
public Job importUserJob() {
return jobBuilderFactory.get("importUserJob")
.incrementer(new RunIdIncrementer())
.start(loadFileStep())
.build();
}
@Bean
public Step loadFileStep() {
return stepBuilderFactory.get("loadFileStep")
.<Person, Person> chunk(10)
.reader(reader())
.processor(processor())
.writer(writer())
.build();
}
批处理需要一个读，处理，写三个环节。
Job通过JobBuilderFactory创建，step通过StepBuilderFactory创建。因为引入了springboot和springbatch的starter，这两个bean都是已经配置好的，直接注入使用就可以了。
读和写直接使用springbatch已经有的类FlatFileItemReaderBuilder、JdbcBatchItemWriterBuilder即可。
Processor需要实现ItemProcessor接口，单独编写一个java类，具体如下，主要做逻辑处理。
public class PersonItemProcessor implements ItemProcessor<Person,Person> {
private static final Logger log = LoggerFactory.getLogger(PersonItemProcessor.class);
/**
* person 传入值
* @param person
* @return
* @throws Exception
*/
@Override
public Person process(Person person) throws Exception {
String firstName= person.getFirstName().toUpperCase();
String lastName = person.getLastName().toUpperCase();
Person transformedPerson = new Person(firstName, lastName);
log.info("Converting (" + person + ") into (" + transformedPerson + ")");
return transformedPerson;
}
}
接下来，怎么启动。
@SpringBootApplication
@EnableScheduling
@EnableBatchProcessing
public class Batchstudy04Application {
public static void main(String[] args) {
SpringApplication.run(Batchstudy04Application.class, args);
}
}
主体类还是springboot默认生成的Application，关键就是在类上的一个注解@EnableBatchProcessing。
点击右键，运行就可以了。
运行日志：
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.2.RELEASE)
2021-01-19 16:33:05.169 INFO 14296 --- [ main] c.y.b.Batchstudy04Application : Starting Batchstudy04Application on DESKTOP-HLLDRH0 with PID 14296 (F:\yusys\cenbank\demo\spbootbatch\batchstudy04\target\classes started by admin in F:\yusys\cenbank\demo\spbootbatch\batchstudy04)
2021-01-19 16:33:05.171 INFO 14296 --- [ main] c.y.b.Batchstudy04Application : No active profile set, falling back to default profiles: default
2021-01-19 16:33:05.711 INFO 14296 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JDBC repositories in DEFAULT mode.
2021-01-19 16:33:05.729 INFO 14296 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 14ms. Found 0 JDBC repository interfaces.
2021-01-19 16:33:06.082 INFO 14296 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2021-01-19 16:33:06.089 INFO 14296 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2021-01-19 16:33:06.090 INFO 14296 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.37]
2021-01-19 16:33:06.216 INFO 14296 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2021-01-19 16:33:06.216 INFO 14296 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1008 ms
2021-01-19 16:33:06.289 INFO 14296 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2021-01-19 16:33:07.690 INFO 14296 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2021-01-19 16:33:07.882 INFO 14296 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
2021-01-19 16:33:08.084 INFO 14296 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: MYSQL
2021-01-19 16:33:08.100 INFO 14296 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
2021-01-19 16:33:08.209 INFO 14296 --- [ main] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService 'taskScheduler'
2021-01-19 16:33:08.248 INFO 14296 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2021-01-19 16:33:08.258 INFO 14296 --- [ main] c.y.b.Batchstudy04Application : Started Batchstudy04Application in 3.469 seconds (JVM running for 5.256)
2021-01-19 16:33:08.259 INFO 14296 --- [ main] o.s.b.a.b.JobLauncherApplicationRunner : Running default command line with: []
2021-01-19 16:33:08.399 INFO 14296 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=importUserJob]] launched with the following parameters: [{run.id=1}]
2021-01-19 16:33:08.478 INFO 14296 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [loadFileStep]
2021-01-19 16:33:08.518 INFO 14296 --- [ main] c.ycxy.batchstudy04.PersonItemProcessor : Converting (firstName: Jill, lastName: Doe) into (firstName: JILL, lastName: DOE)
2021-01-19 16:33:08.518 INFO 14296 --- [ main] c.ycxy.batchstudy04.PersonItemProcessor : Converting (firstName: Joe, lastName: Doe) into (firstName: JOE, lastName: DOE)
2021-01-19 16:33:08.518 INFO 14296 --- [ main] c.ycxy.batchstudy04.PersonItemProcessor : Converting (firstName: Justin, lastName: Doe) into (firstName: JUSTIN, lastName: DOE)
2021-01-19 16:33:08.518 INFO 14296 --- [ main] c.ycxy.batchstudy04.PersonItemProcessor : Converting (firstName: Jane, lastName: Doe) into (firstName: JANE, lastName: DOE)
2021-01-19 16:33:08.519 INFO 14296 --- [ main] c.ycxy.batchstudy04.PersonItemProcessor : Converting (firstName: John, lastName: Doe) into (firstName: JOHN, lastName: DOE)
2021-01-19 16:33:08.534 INFO 14296 --- [ main] o.s.batch.core.step.AbstractStep : Step: [loadFileStep] executed in 56ms
2021-01-19 16:33:08.555 INFO 14296 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=importUserJob]] completed with the following parameters: [{run.id=1}] and the following status: [COMPLETED] in 121ms
数据库里已经成功插入数据了。
注意
这里是mysql的例子，如果是使用Oracle数据库，引入spring-boot-starter-data-jdbc后，jdbc的操作类会找不到。怎么解决？？？引入JPA依赖包就可以了，JPA里有关于Oralce的实现，这样就可以在oracle上运行了。
6411
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
