spring batch demo 注解版

最新推荐文章于 2024-05-19 16:54:26 发布
最新推荐文章于 2024-05-19 16:54:26 发布
阅读量6.4k 收藏 6
点赞数
分类专栏： Spring Batch 实习 文章标签： spring batch
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_33366229/article/details/78711228
版权

项目结构

pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.ncsi</groupId>
  <artifactId>Demo1_SpringBatch</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>Demo1_SpringBatch</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
  <!-- https://mvnrepository.com/artifact/junit/junit -->
 <dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <scope>test</scope>
</dependency>
<!--  <dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>3.8</version>
    <scope>test</scope>
</dependency>  -->

    <!-- Apache DBCP-->  
    <dependency>  
        <groupId>commons-dbcp</groupId>  
        <artifactId>commons-dbcp</artifactId>  
        <version>1.4</version>  
    </dependency>  

   <!-- Mysql Dependency -->  
   <dependency>  
    <groupId>mysql</groupId>  
    <artifactId>mysql-connector-java</artifactId>  
    <version>5.1.25</version>  
    </dependency>  

     <dependency>   
<groupId>org.springframework.batch</groupId>   
 <artifactId>spring-batch-core</artifactId>   
 <version>3.0.6.RELEASE</version>   
 </dependency>   

  <dependency>  
        <groupId>org.springframework</groupId>  
        <artifactId>spring-jdbc</artifactId>  
        <version>4.2.4.RELEASE</version>  
    </dependency>  
   <!--  <dependency>  
        <groupId>org.springframework</groupId>  
        <artifactId>spring-context-support</artifactId>  
        <version>4.0.9.RELEASE</version>  
    </dependency>  --> 
    <!-- https://mvnrepository.com/artifact/org.springframework/spring-tx -->
<!-- <dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-tx</artifactId>
    <version>4.3.11.RELEASE</version>
</dependency> -->
<!-- https://mvnrepository.com/artifact/org.springframework/spring-test -->
<!-- <dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-test</artifactId>
    <version>4.3.11.RELEASE</version>
    <scope>test</scope>
</dependency> -->
<!-- https://mvnrepository.com/artifact/org.springframework/spring-test -->
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-test</artifactId>
    <version>4.0.5.RELEASE</version>
    <scope>test</scope>
</dependency>

<!-- https://mvnrepository.com/artifact/log4j/log4j -->
<dependency>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
    <version>1.2.17</version>
</dependency>


  </dependencies>


  <build>
     <plugins>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.1</version>
              <configuration>
                 <source>1.8</source>
                 <target>1.8</target>
         </configuration>
         </plugin>
     </plugins>
  </build>
</project>

UserEntity:

package com.Entity;

import java.util.Date;

public class UserEntity {

  private String recordType;
  private String name;
  private Date birthday;
  private Integer children;
  private int lineNumber;//文件每行的行数 如果是错误记录，就可以把错误信息准确的放到某一行

  public UserEntity(){

  }

public String getName() {
    return name;
}

public void setName(String name) {
    this.name = name;
}

public Date getBirthday() {
    return birthday;
}

public void setBirthday(Date birthday) {
    this.birthday = birthday;
}

public Integer getChildren() {
    return children;
}

public void setChildren(Integer children) {
    this.children = children;
}


public int getLineNumber() {
    return lineNumber;
}

public void setLineNumber(int lineNumber) {
    this.lineNumber = lineNumber;
}

public String getRecordType() {
return recordType;
}


public void setRecordType(String recordType) {
this.recordType = recordType;
}


public UserEntity(String name, Date birthday, Integer children, int lineNumber) {
    super();
    this.name = name;
    this.birthday = birthday;
    this.children = children;
    this.lineNumber = lineNumber;
}

@Override
public String toString() {
    return "UserEntity [name=" + name + ", birthday=" + birthday
            + ", children=" + children + ", lineNumber=" + lineNumber + "]";
}
}

CsvJoblistener:

package com.listener;

import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;
import org.springframework.stereotype.Component;

@Component
public class csvJobListener implements JobExecutionListener {


    public csvJobListener(){
        System.out.println("listener !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
    }
    @Override
    public void beforeJob(JobExecution jobExecution) {
        // TODO Auto-generated method stub
        System.out.println(" csvJobListener . beforeJob--------------------------------------");
    }

    @Override
    public void afterJob(JobExecution jobExecution) {
        // TODO Auto-generated method stub
        System.out.println("csvJobListener . afterJob");
    }

}

CsvSetMapper:

package com.Mapper;
import java.util.Map;

import org.springframework.batch.item.file.mapping.FieldSetMapper;
import org.springframework.batch.item.file.transform.FieldSet;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Component;
import org.springframework.validation.BindException;

import com.Entity.*;
import com.Service.CommonCSVService;
import com.Util.DateUtil;

@Component
public class CsvSetMapper implements FieldSetMapper<UserEntity>{

    @Autowired(required=true)
    @Qualifier("commonCSVService1")
    private CommonCSVService commonCSVService1;

    int lineNumber=1;

    Map<String, String> csvFileNameMap;

    String csvFilePathAndName;
    @Override
    public UserEntity mapFieldSet(FieldSet fieldSet) throws BindException {
        // TODO Auto-generated method stub

        UserEntity user;
        try{
            csvFileNameMap =commonCSVService1.getJobParams();
            csvFilePathAndName =csvFileNameMap.get("csvFilePathAndName");
            System.err.println(csvFilePathAndName);

            //csv的结束行为9时 就不再继续读取了  可以用来统计文件的总行数（我这里不包括字段标记行）
            if(fieldSet!=null&&!fieldSet.readString(0).equals("9")){
                user =new UserEntity();
                user.setRecordType(fieldSet.readString(0));
                user.setLineNumber(lineNumber);
                user.setName(fieldSet.readString(1));
                user.setBirthday(DateUtil.parseDate(fieldSet.readString(2)));
                user.setChildren(fieldSet.readInt(3));
                lineNumber++;
                return user;
            }
              lineNumber=1;
        }catch (Exception ex){
            System.out.println("mapFieldSet :"+ex);
        }
        return null;
    }

}

CsvItemProcessor:

package com.processor;

import org.springframework.batch.item.ItemProcessor;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Component;

import com.Entity.UserEntity;
import com.Service.CommonCSVService;
import com.Util.StringUtil;

//@Component
public class CsvItemProcessor implements ItemProcessor<UserEntity, UserEntity> {

//  @Autowired(required=true)
//  @Qualifier("commonCSVService")
//  private CommonCSVService commonCSVService;

    @Override
    public UserEntity process(UserEntity item) throws Exception {
        // TODO Auto-generated method stub
        UserEntity user;

        try{
            if(!StringUtil.isNotNullOrNotEmptyString(item.getName())){
                return null;
            }
            user=new UserEntity();
            user.setName(item.getName());
            user.setBirthday(item.getBirthday());
            user.setChildren(item.getChildren());

            return user;
        }catch(Exception  e){
            System.out.println("CsvItemProcessor :"+e);
            throw e;
        }
    }

}

CommonCSVService:

package com.Service;

import java.util.Map;

import org.springframework.batch.core.Job;

import com.Entity.UserEntity;


public interface CommonCSVService {

    Map<String, String> getJobParams();

    boolean insertToDB(UserEntity item);

    boolean callCSVJob(Job job,String csvFilePath,String csvFileName);

}

CommonCSVServiceImpl:

package com.Service.Impl;

import java.util.HashMap;
import java.util.Map;

import com.Entity.UserEntity;
import com.Service.CommonCSVService;

import org.springframework.batch.core.BatchStatus;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service("commonCSVService1")
public class CommonCSVServiceImpl implements  CommonCSVService{

    public JobParameters jobParameters;

    public String csvFilePath;

    public String csvFileName;

    JobExecution jobExecution;

    @Autowired
    JobLauncher  jobLauncher;

    @Autowired(required = false)
    private JobRepository jobRepository;

    @Override
    public Map<String, String> getJobParams() {
        // TODO Auto-generated method stub
        Map<String,String> jobParamsMap =null;
        try{
            jobParamsMap =new HashMap<String,String>();
            jobParamsMap.put("time", jobParameters.getString("time"));
            jobParamsMap.put("filePath",jobParameters.getString("filePath"));
            jobParamsMap.put("csvFileName", jobParameters.getString("csvFileName"));
            jobParamsMap.put("csvFilePathAndName", jobParameters.getString("csvFilePathAndName"));

        }catch(Exception e){
            System.out.println("CommonCSVServiceImpl.getJobParams :"+e);
        }
        return jobParamsMap;
    }

    @Override
    public boolean insertToDB(UserEntity item) {
        // TODO Auto-generated method stub
        System.out.println(item.getName()+" !!!!!!!!!!!!!!!!!!!!!!!!!");
        System.out.println("INSERT TO DB!!!!!!!!!!!!!!!!!!!!!!!!!!");
        return true;
    }

    @Override
    public boolean callCSVJob(Job job,String csvFilePath,String csvFileName) {
        // TODO Auto-generated method stub
        jobParameters=null;
        this.csvFilePath=csvFilePath;
        this.csvFileName=csvFileName;
        String  csvFilePathAndName=csvFilePath+csvFileName;
        jobParameters = new JobParametersBuilder()
        .addLong("time", System.currentTimeMillis())
        .addString("filePath", csvFilePath)
        .addString("csvFileName", csvFileName)
        .addString("csvFilePathAndName",csvFilePathAndName)
        .toJobParameters();

        try {
            jobExecution=jobLauncher.run(job, jobParameters);

            if(jobExecution.getStatus() == BatchStatus.COMPLETED){
                return  true;
            }
        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } 
        return false;
    }

}

DateUtil

package com.Util;

import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;

public class DateUtil {
    public final static String DEFAULT_DATE_FORMAT = "dd/MM/yyyy";
    public final static String _DATE = "yyyy-MM-dd";

    public static Date parseDate(String dateStr){
        if(dateStr.length()!=10){
            return null;
        }
        Date date =null;
        Date tranDate=null;

        DateFormat df=new SimpleDateFormat(DEFAULT_DATE_FORMAT);
        DateFormat tranDF = new SimpleDateFormat(_DATE);

        // defaultDF.setLenient(true);
        try {
            date = df.parse(dateStr);
            tranDate = tranDF.parse(tranDF.format(date));
        } catch (Exception e) {
            System.out.println("DateUtil.parseDate:"+e);
        }

        return tranDate;
    }
}

FileUtil:

package com.Util;

import java.io.File;
import java.util.ArrayList;
import java.util.List;

public class FileUtil {


    public static List<String> getFilePathList(String folderpath) {
     List<String> list;
     File dir;
     File[] files;
     try {
         dir = new File(folderpath);
         files = dir.listFiles(); 
         list = new ArrayList<String>();
         if (files != null) {
             for (int i = 0; i < files.length; i++) {
                 String filePathName = files[i].getPath();
                 if (!files[i].isDirectory()) { 
                     list.add(filePathName);
                 }else {
                     System.out.println("this is Directory");
                 } 
             }
             return list;
         }
     } catch (Exception e) {
        System.out.println("FileUtil .getFilePathList :"+e);
     }
     return null;
    }

     public static String[] readAllFiles(String folderpath){
        File file;
        String[] filelist=null;
          try {
                file = new File(folderpath);
                     if (!file.isDirectory()) {
                           System.out.println("FileUtil .readAllFiles: 不是目录！！");
                     } else if (file.isDirectory()) {
                             filelist= file.list();
                     }
            } catch (Exception e) {
                  System.out.println("FileUtil .readAllFiles: "+e);
            }
            return filelist;
  }


}

StringUtil:

package com.Util;

public class StringUtil {


    public static boolean isNumeric(String str){
        try{
            for(int i=0;i<str.length();i++){
                if(!Character.isDigit(str.charAt(i))){
                    return false;
                }
            }
        }catch(Exception e){
            System.out.println("StringUtil.isNumeric :"+e);
        }
        return true;
    }


    public static boolean isNotNullOrNotEmptyString(String str){
        try{

            if(str !=null &&str.length()!=0){
                return true;
            }
        }catch(Exception e){
          System.out.println("StringUtil.isNotNullOrEmptyString :"+e);  
        }
        return false;
    }
}

CsvItemWriter:

package com.writer;

import java.util.List;

import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Component;

import com.Entity.UserEntity;
import com.Service.CommonCSVService;

@Component
public class CsvItemWriter implements ItemWriter<UserEntity> {

    @Autowired(required =false)
    @Qualifier("commonCSVService1")
    private CommonCSVService commonCSVService1;



    @Override
    public void write(List<? extends UserEntity> users){
        try{
            for(UserEntity user :users){
                if(commonCSVService1.insertToDB(user)){
                    System.out.println("CsvItemWriter :"+" insert successfully!");
                }else{
                    System.out.println("CsvItemWriter :"+" insert failed !");
                }
            }
        }catch(Exception e){
            System.out.println("CsvItemWriter: "+e);
        }
    }

}

csvJobTest:

package com.batch;

import java.util.List;
import java.util.Map;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.batch.core.Job;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

import com.Service.CommonCSVService;
import com.Util.FileUtil;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("classpath:config.xml")
public class csvJobTest {
   private String filePath;

    @Autowired(required =false)
    @Qualifier("commonCSVService1")
    private CommonCSVService commonCSVService1;

    @Autowired
    Job csvJob;

    public String getFilePath(){
        return filePath;
    }

    @Value("#{prop.usercsvFolderPath}")
    public void setFilePath(String filePath){
        this.filePath=filePath;
    }
    Map<String,String> csvFileNameMap;


    @Test
    public void testJob() throws Exception{
        List<String> filePathList;
        String[] FileList;
        try{
            filePathList =FileUtil.getFilePathList(filePath);
            FileList=FileUtil.readAllFiles(filePath);
            if(filePathList.size()>0){
                for(int i=0;i<filePathList.size();i++){
                    if(commonCSVService1.callCSVJob(csvJob, filePath, FileList[i])){
                        System.out.println("csv job  successfully!!!!");
                    }
                }
            }else{
                System.out.println("empty folder!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
            }

        }catch(Exception e){
            filePathList =FileUtil.getFilePathList(filePath);
        }
    }
}

config.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:aop="http://www.springframework.org/schema/aop"
    xmlns:tx="http://www.springframework.org/schema/tx"
    xmlns:jee="http://www.springframework.org/schema/jee"
    xmlns:jdbc="http://www.springframework.org/schema/jdbc"
    xmlns:jpa="http://www.springframework.org/schema/data/jpa"
    xmlns:batch="http://www.springframework.org/schema/batch"
    xmlns:mvc="http://www.springframework.org/schema/mvc"
    xsi:schemaLocation="http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-4.0.xsd
        http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-4.0.xsd
        http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.0.xsd
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
        http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.0.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsd
        http://www.springframework.org/schema/data/jpa http://www.springframework.org/schema/data/jpa/spring-jpa.xsd
        http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd
        http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd
        http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-4.1.7.xsd">


    <import resource="classpath:database-set.xml"/>
    <import resource="classpath:loadFilepath.xml"/>

    <!-- For consider the using of annotations foe defining Spring Bean -->
    <context:annotation-config /> 

    <!-- For defining Spring Bean -->
    <context:component-scan base-package="com" />
</beans>

database-set.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:aop="http://www.springframework.org/schema/aop"
    xmlns:tx="http://www.springframework.org/schema/tx"
    xmlns:jee="http://www.springframework.org/schema/jee"
    xmlns:jdbc="http://www.springframework.org/schema/jdbc"
    xmlns:jpa="http://www.springframework.org/schema/data/jpa"
    xmlns:batch="http://www.springframework.org/schema/batch"
    xmlns:mvc="http://www.springframework.org/schema/mvc"
    xsi:schemaLocation="http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-4.0.xsd
        http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-4.0.xsd
        http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.0.xsd
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
        http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.0.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsd
        http://www.springframework.org/schema/data/jpa http://www.springframework.org/schema/data/jpa/spring-jpa.xsd
        http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd
        http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd
        http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-4.1.7.xsd">

    <!-- Imports Other resources configuration -->
    <context:property-placeholder location="classpath:database-source.properties"/>


   <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"  
        destroy-method="close">  
        <property name="driverClassName" value="${database.driverClass}"></property>  
        <property name="url" value="${database.url}"></property>  
        <property name="username" value="${database.username}"></property>  
        <property name="password" value="${database.password}"></property>
 </bean>

    <bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">  
        <property name="dataSource" ref="dataSource"></property>  
    </bean>   
</beans>

database-source.properties:

database.name=springbatch
database.host=localhost
database.port=3306
database.username=root
database.password=123456liu
database.driverClass=com.mysql.jdbc.Driver
database.url=jdbc:mysql://${database.host}:${database.port}/${database.name}?characterEncoding=UTF-8&amp;characterSetResults=UTF-8

filepath.xml:


usercsvFolderPath=D:/csvJobFolder/

loadFilepath.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:aop="http://www.springframework.org/schema/aop"
    xmlns:tx="http://www.springframework.org/schema/tx"
    xmlns:jee="http://www.springframework.org/schema/jee"
    xmlns:jdbc="http://www.springframework.org/schema/jdbc"
    xmlns:jpa="http://www.springframework.org/schema/data/jpa"
    xmlns:batch="http://www.springframework.org/schema/batch"
    xmlns:mvc="http://www.springframework.org/schema/mvc"
    xsi:schemaLocation="http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-4.0.xsd
        http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-4.0.xsd
        http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.0.xsd
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
        http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.0.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsd
        http://www.springframework.org/schema/data/jpa http://www.springframework.org/schema/data/jpa/spring-jpa.xsd
        http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd
        http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd
        http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-4.1.7.xsd">

 <!-- config file path -->  
    <bean id="prop" class="org.springframework.beans.factory.config.PropertiesFactoryBean">  
        <property name="locations">
            <array>  
                <value>classpath:filepath.properties</value>  
            </array>  
        </property>  
    </bean>

    </beans>

result:

[INFO ] 2017-12-07 09:48:45,511 method:org.springframework.test.context.TestContextManager.retrieveTestExecutionListeners(TestContextManager.java:242)
Could not instantiate TestExecutionListener [org.springframework.test.context.web.ServletTestExecutionListener]. Specify custom listener classes or make the default listener classes (and their required dependencies) available. Offending class: [javax/servlet/ServletContext]
[INFO ] 2017-12-07 09:48:45,667 method:org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:316)
Loading XML bean definitions from class path resource [config.xml]
[INFO ] 2017-12-07 09:48:45,810 method:org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:316)
Loading XML bean definitions from class path resource [database-set.xml]
[INFO ] 2017-12-07 09:48:45,888 method:org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:316)
Loading XML bean definitions from class path resource [loadFilepath.xml]
[INFO ] 2017-12-07 09:48:46,389 method:org.springframework.context.support.AbstractApplicationContext.prepareRefresh(AbstractApplicationContext.java:513)
Refreshing org.springframework.context.support.GenericApplicationContext@15975490: startup date [Thu Dec 07 09:48:46 CST 2017]; root of context hierarchy
[INFO ] 2017-12-07 09:48:46,546 method:org.springframework.beans.factory.support.DefaultListableBeanFactory.registerBeanDefinition(DefaultListableBeanFactory.java:749)
Overriding bean definition for bean 'jobRepository': replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=simpleBatchConfiguration; factoryMethodName=jobRepository; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/batch/core/configuration/annotation/SimpleBatchConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=modularBatchConfiguration; factoryMethodName=jobRepository; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/batch/core/configuration/annotation/ModularBatchConfiguration.class]]
[INFO ] 2017-12-07 09:48:46,546 method:org.springframework.beans.factory.support.DefaultListableBeanFactory.registerBeanDefinition(DefaultListableBeanFactory.java:749)
Overriding bean definition for bean 'jobLauncher': replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=simpleBatchConfiguration; factoryMethodName=jobLauncher; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/batch/core/configuration/annotation/SimpleBatchConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=modularBatchConfiguration; factoryMethodName=jobLauncher; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/batch/core/configuration/annotation/ModularBatchConfiguration.class]]
[INFO ] 2017-12-07 09:48:46,546 method:org.springframework.beans.factory.support.DefaultListableBeanFactory.registerBeanDefinition(DefaultListableBeanFactory.java:749)
Overriding bean definition for bean 'transactionManager': replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=simpleBatchConfiguration; factoryMethodName=transactionManager; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/batch/core/configuration/annotation/SimpleBatchConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=modularBatchConfiguration; factoryMethodName=transactionManager; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/batch/core/configuration/annotation/ModularBatchConfiguration.class]]
[INFO ] 2017-12-07 09:48:46,546 method:org.springframework.beans.factory.support.DefaultListableBeanFactory.registerBeanDefinition(DefaultListableBeanFactory.java:749)
Overriding bean definition for bean 'jobExplorer': replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=simpleBatchConfiguration; factoryMethodName=jobExplorer; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/batch/core/configuration/annotation/SimpleBatchConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=modularBatchConfiguration; factoryMethodName=jobExplorer; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/batch/core/configuration/annotation/ModularBatchConfiguration.class]]
[INFO ] 2017-12-07 09:48:46,561 method:org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.isOverriddenByExistingDefinition(ConfigurationClassBeanDefinitionReader.java:279)
Skipping bean definition for [BeanMethod:name=csvSetMapper,declaringClass=com.configuration.csvJobConfiguration]: a definition for bean 'csvSetMapper' already exists. This top-level bean definition is considered as an override.
[INFO ] 2017-12-07 09:48:46,561 method:org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.isOverriddenByExistingDefinition(ConfigurationClassBeanDefinitionReader.java:279)
Skipping bean definition for [BeanMethod:name=csvJobListener,declaringClass=com.configuration.csvJobConfiguration]: a definition for bean 'csvJobListener' already exists. This top-level bean definition is considered as an override.
[INFO ] 2017-12-07 09:48:46,733 method:org.springframework.core.io.support.PropertiesLoaderSupport.loadProperties(PropertiesLoaderSupport.java:172)
Loading properties file from class path resource [database-source.properties]
[WARN ] 2017-12-07 09:48:46,748 method:org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:304)
@Bean method ScopeConfiguration.stepScope is non-static and returns an object assignable to Spring's BeanFactoryPostProcessor interface. This will result in a failure to process annotations such as @Autowired, @Resource and @PostConstruct within the method's declaring @Configuration class. Add the 'static' modifier to this method to avoid these container lifecycle issues; see @Bean Javadoc for complete details
[WARN ] 2017-12-07 09:48:46,764 method:org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:304)
@Bean method ScopeConfiguration.jobScope is non-static and returns an object assignable to Spring's BeanFactoryPostProcessor interface. This will result in a failure to process annotations such as @Autowired, @Resource and @PostConstruct within the method's declaring @Configuration class. Add the 'static' modifier to this method to avoid these container lifecycle issues; see @Bean Javadoc for complete details
[INFO ] 2017-12-07 09:48:46,873 method:org.springframework.core.io.support.PropertiesLoaderSupport.loadProperties(PropertiesLoaderSupport.java:172)
Loading properties file from class path resource [filepath.properties]
[INFO ] 2017-12-07 09:48:47,675 method:org.springframework.batch.core.repository.support.JobRepositoryFactoryBean.afterPropertiesSet(JobRepositoryFactoryBean.java:183)
No database type set, using meta data indicating: MYSQL
[INFO ] 2017-12-07 09:48:47,801 method:org.springframework.batch.core.launch.support.SimpleJobLauncher.afterPropertiesSet(SimpleJobLauncher.java:195)
No TaskExecutor has been set, defaulting to synchronous executor.
listener !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[WARN ] 2017-12-07 09:48:47,932 method:org.springframework.batch.core.listener.AbstractListenerFactoryBean.isListener(AbstractListenerFactoryBean.java:222)
org.springframework.batch.item.ItemReader is an interface.  The implementing class will not be queried for annotation based listener configurations.  If using @StepScope on a @Bean method, be sure to return the implementing class so listner annotations can be used.
[WARN ] 2017-12-07 09:48:47,932 method:org.springframework.batch.core.listener.AbstractListenerFactoryBean.isListener(AbstractListenerFactoryBean.java:222)
org.springframework.batch.item.ItemWriter is an interface.  The implementing class will not be queried for annotation based listener configurations.  If using @StepScope on a @Bean method, be sure to return the implementing class so listner annotations can be used.
[WARN ] 2017-12-07 09:48:47,932 method:org.springframework.batch.core.listener.AbstractListenerFactoryBean.isListener(AbstractListenerFactoryBean.java:222)
org.springframework.batch.item.ItemProcessor is an interface.  The implementing class will not be queried for annotation based listener configurations.  If using @StepScope on a @Bean method, be sure to return the implementing class so listner annotations can be used.
[WARN ] 2017-12-07 09:48:47,932 method:org.springframework.batch.core.listener.AbstractListenerFactoryBean.isListener(AbstractListenerFactoryBean.java:222)
org.springframework.batch.item.ItemReader is an interface.  The implementing class will not be queried for annotation based listener configurations.  If using @StepScope on a @Bean method, be sure to return the implementing class so listner annotations can be used.
[WARN ] 2017-12-07 09:48:47,948 method:org.springframework.batch.core.listener.AbstractListenerFactoryBean.isListener(AbstractListenerFactoryBean.java:222)
org.springframework.batch.item.ItemWriter is an interface.  The implementing class will not be queried for annotation based listener configurations.  If using @StepScope on a @Bean method, be sure to return the implementing class so listner annotations can be used.
[WARN ] 2017-12-07 09:48:47,948 method:org.springframework.batch.core.listener.AbstractListenerFactoryBean.isListener(AbstractListenerFactoryBean.java:222)
org.springframework.batch.item.ItemProcessor is an interface.  The implementing class will not be queried for annotation based listener configurations.  If using @StepScope on a @Bean method, be sure to return the implementing class so listner annotations can be used.
[INFO ] 2017-12-07 09:48:48,451 method:org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:133)
Job: [FlowJob: [name=csvJob]] launched with the following parameters: [{time=1512611328042, filePath=D:/csvJobFolder/, csvFileName=csvJobTest.csv, csvFilePathAndName=D:/csvJobFolder/csvJobTest.csv}]
 csvJobListener . beforeJob--------------------------------------
[INFO ] 2017-12-07 09:48:48,845 method:org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:146)
Executing step: [csvStep]
ItemReader .filePath:D:/csvJobFolder/csvJobTest.csv
csvLineMapper ....
user1 ---------!!!!!!!!!!!!!!!!!!!!!!!!!
INSERT TO DB!!!!!!!!!!!!!!!!!!!!!!!!!!
CsvItemWriter : insert successfully!
user2 ---------!!!!!!!!!!!!!!!!!!!!!!!!!
INSERT TO DB!!!!!!!!!!!!!!!!!!!!!!!!!!
CsvItemWriter : insert successfully!
user3 ---------!!!!!!!!!!!!!!!!!!!!!!!!!
INSERT TO DB!!!!!!!!!!!!!!!!!!!!!!!!!!
CsvItemWriter : insert successfully!
csvJobTest.csv-------------------------
csvJobListener . afterJob
[INFO ] 2017-12-07 09:48:49,373 method:org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:136)
Job: [FlowJob: [name=csvJob]] completed with the following parameters: [{time=1512611328042, filePath=D:/csvJobFolder/, csvFileName=csvJobTest.csv, csvFilePathAndName=D:/csvJobFolder/csvJobTest.csv}] and the following status: [COMPLETED]
csv job  successfully!!!!
[INFO ] 2017-12-07 09:48:49,373 method:org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:873)
Closing org.springframework.context.support.GenericApplicationContext@15975490: startup date [Thu Dec 07 09:48:46 CST 2017]; root of context hierarchy
[INFO ] 2017-12-07 09:48:49,373 method:org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.stop(DefaultLifecycleProcessor.java:356)
Stopping beans in phase 0

上面的Configuration类 在csvWriter 方法上没有注解@Bean @Stepscope 致使在csvWriter的实现类上 就不能注入 CommonCSVService bean 所以commonCSVService 会报nullpoint 错误

小白鼠丶
关注
  • 0
    点赞
  • 踩
  • 6
    收藏
    觉得还不错? 一键收藏
  • 3
    评论
专栏目录
Spring Batch入门实例
weixin_44728363的博客
10-14 708
利用Spring Batch写了一个简单的读取、处理、导出文件的demo，加深理解
【SpringBoot系列】最详细demo-- Spring Batch批处理
wufaqidong1的博客
07-18 3496
先用一个图让你有一个大概印象，这个东西是什么JobRepository是用户注册Job的容器，就是存储数据的地方，可以看做是一个数据库的接口，在任务执行的时候需要通过它来记录任务状态等等信息。JobLauncher是任务启动器，通过它来启动任务，可以看做是程序的入口。Job代表着一个具体的任务。Step代表着一个具体的步骤，一个Job可以包含多个Step（想象把大象放进冰箱这个任务需要多少个步骤你就明白了）。Item就是输出->处理->输出，一个完整Step流程。...
spring-batch批处理框架介绍及demo
lewenson的博客
09-03 2330
最近在项目中，有批处理的相关需求。最终选定技术框架为spring-batch批处理框架，在此专栏中为大家分享spring-batch的基础知识与项目遇到的一些实际问题的解决方案。
spring 启动之解析配置类
weixin_45849726的博客
05-19 806
invokeBeanFactoryPostProcessors会调用实现了BeanFactoryPostProcessors接口的postProcessBeanFactory方法invokeBeanFactoryPostProcessors会调用实现BeanDefinitionRegistryPostProcessor接口的postProcessBeanDefinitionRegistry方法。
springbatch 批量读写demo
qq_40250122的博客
05-14 833
这两天做到一个需求，需要一次性将一张数据表里的所有资源数据中的位置信息录入到redis中，表中的数据有几百万起步，采用的是java定时任务，循环每次查出一万条数据存入redis，测试时发现插入效率非常慢，经过排查发现是通过mybatis查询数据的耗时超过了总耗时的99%，我这台机器上是每查询一万条数据返回耗时87s左右。 解决方法： 在mapper.xml中的中加入fetchSize参数，设置的大一些，如下： pageSize 每次分页大小配置 /* * Copyright yangjunxiong 2
Consider defining a bean of type问题解决
热门推荐
Andrew_Chenwq的博客
09-05 4万+
会有一些包你明明@Autowired 但是还是找不到什么bean 导致你项目启动不了 解决方法一: 这个问题主要是因为项目拆包后，在原有的引用项目中无法找到对应的包的路径。在项目的Application启动文件里，使用@MapperScan注解，把当前的路径引用进去就可以了 解决方法二: 在相应的 @Autowired(required = false) 如下。所以，解决方法是让原来引用的项目可以找到资源的路径就可以了。明白真正靠得住的只有自己。
springbatch DEMO
09-14
springbatch 文件写入DB， 需要自己创建一个users表 /* SQLyog Professional v10.51 MySQL - 5.5.28-log ********************************************************************* */ /*!40101 SET NAMES utf8 *...
spring batch demo
06-14
这个"spring batch demo"提供了使用Spring Batch实现批处理操作的实例，帮助我们了解如何有效地利用该框架。 Spring Batch 提供了以下核心功能： 1. **读取数据**：Spring Batch 提供多种`ItemReader`接口的实现，...
springBoot+springBatch批量处理数据demo
02-29
在这个“springBoot+springBatch批量处理数据demo”中，我们将探讨如何将这两个强大的工具结合在一起，实现高效的数据处理。 首先，SpringBoot的核心特性在于其自动配置，它通过`@SpringBootApplication`注解自动...
Spring Batch学习demo项目源码
10-30
这个"Spring Batch学习demo项目源码"是针对Spring Batch进行学习实践的一个实例，旨在帮助开发者了解并掌握如何在实际应用中使用Spring Batch。 在源码中，`batch-xml`文件可能是配置文件，通常在Spring Batch项目...
spring batch demo[极简版]
Steve.Tao的博客
03-07 957
1. maven&lt;dependency&gt; &lt;groupId&gt;org.springframework.batch&lt;/groupId&gt; &lt;artifactId&gt;spring-batch-core&lt;/artifactId&gt; &lt;version&gt;3.0.8.RELEASE&lt;/version&gt; &lt;/depe
Spring Batch批处理框架
11-16
Spring Batch批处理框架Spring Batch批处理框架Spring Batch批处理框架
10分钟搞定 Spring 批处理组件 —— spring-batch
m0_71777195的博客
08-13 6317
Spring Batch 是一个轻量级、全面的批处理框架，旨在支持开发对企业系统的日常运营至关重要的健壮。Spring Batch 建立在人们所期望的 Spring Framework 的特性（生产力、基于 POJO 的开发方法和一般易用性）之上，同时使开发人员可以在必要时轻松访问和利用更先进的企业服务。Spring Batch 不是调度框架。在商业和开源空间中都有许多优秀的企业调度程序（例如 Quartz、Tivoli、Control-M 等）可用。它旨在与调度程序一起工作，而不是取代调度程序。...
Spring Batch学习记录及示例项目代码
专注Java后端技术干货、项目源码总结分享，期待您的关注。
08-15 558
Spring Batch学习记录总结 关于Spring Batch的相关知识点参考：https://mp.weixin.qq.com/s/OUMwyo2EopXkSHGn2OlghQ Spring Batch的一些基本概念： ItemReader：对资源的读处理，如从数据库查询、文件读取、变量读取等。 ItemProcessor：对读取的数据进行处理，可以实现自己的业务逻辑操作来对数据处理，如对数据进行计算、逻辑处理、格式转换等。 ItemWriter：对资源的写处理，如写入数据库、写入文件.
SpringBatch批处理实战教程
04-24
  课程介绍：        本系列视频教程结合具体案例详细讲解了SpringBatch实现批处理的方式，包括Job，Flow，决策器，监听器，ItemReader，ItemWriter，ItemProcessor，JobLauncher，JobOperator。
beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type
qq_45311792的博客
12-20 691
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type ‘com.heima.model.mappers.app.ApShowBehaviorMapper’ available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@o
SpringBoot整合SpringBatch实用简例
justlpf的专栏
03-03 444
faultTolerant().retryLimit(3).retry(Exception.class).skipLimit(100).skip(Exception.class). //捕捉到异常就重试,重试100次还是异常,JOB就停止并标志失败。writer(getDataWriter()). //指定ItemWriter build();start(handleDataStep()). //start是JOB执行的第一个step。
Spring Batch(4): Job详解
Spring , Hadoop, Spark , BI , ML
03-26 1万+
第四章 配置作业Job4.1 基本配置Job的配置有3个必须的属性，name，jobRepository，steps。一个简单的Job配置如下：<job id="footballJob"> <step id="playerload" parent="s1" next="gameLoad"/> <step id="gameLoad" parent
大数据批处理框架Spring Batch
java、c++、机器学习方向King
11-18 1759
如今微服务架构讨论的如火如荼。但在企业架构里除了大量的OLTP交易外，还存在海量的批处理交易。在诸如银行的金融机构中，每天有3-4万笔的批处理作业需要处理。针对OLTP，业界有大量的开源框架、优秀的架构设计给予支撑；但批处理领域的框架确凤毛麟角。是时候和我们一起来了解下批处理的世界哪些优秀的框架和设计了，今天我将以Spring Batch为例，和大家一起探秘批处理的世界。 初识批处理
springbatch读bat文件demo
最新发布
06-05
下面是一个 Spring Batch 读取 bat 文件的 demo： 1. 首先需要在 pom.xml 文件中添加 Spring Batch 的依赖： ``` <groupId>org.springframework.batch <artifactId>spring-batch-core <version>4.3.1 ``` 2....

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 3
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值