reader: 读是每次读一条数据的,直到读取到chunk()里面任务大小就会进行写入的操作。
ItemWriter: 这个是写,chunk大小来解决的,如果你的chunk()设置的提交间隔太小会执行很多次。
通过xxl-job进行调度
在任务管理器中进行配置好那个JobHandler以后(当然你最先配置好的是你的执行器管理),然后进行调度的时候根据框架的执行流程来:
JobLauncher------->
Job---------->Flow
----->Step------------(ItemReader,ItemProcess,ItemWriter)
problem:
实现IteamReader接口的类读取到也数据,那个ItemWriter也获取到了它的List数据:
但是在实现了ItemWriter的这个类中的重载方法writer中进行日志打印输出的时候输出了很多次。
截图如下:
2022-04-18 10:48:31.408 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryAllDayCaseBySysdocCheck : 159 ] - ==> Preparing: select lcaa.clmnum from clm_baseinfo_lcaa lcaa LEFT JOIN clm_worktask_swflwti t on lcaa.clmnum = t.clmnum and t.status = 1 where lcaa.status = 1 and lcaa.procecode='C' and lcaa.inspectflag is null and t.submitdte BETWEEN CAST(date_format(date_add(now(), interval - 1 day),'%Y%m%d') AS SIGNED integer) AND CAST(date_format(date_add(now(), interval - 1 day),'%Y%m%d') AS SIGNED integer) -- perid 是有效的督查规则那边传递过来的外包商id
2022-04-18 10:48:31.420 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryAllDayCaseBySysdocCheck : 159 ] - ==> Parameters:
2022-04-18 10:48:31.422 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryAllDayCaseBySysdocCheck : 159 ] - <== Total: 4
2022-04-18 10:48:31.422 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskReader : 49 ] - 捞取每日理赔的的案件数目:4条
2022-04-18 10:48:31.434 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为:[700190151, 700189900, 700191218]
2022-04-18 10:48:31.434 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.435 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.436 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.447 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为:[700190151, 700189900, 700191218]
2022-04-18 10:48:31.448 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.448 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.449 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.453 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为:[700190151, 700189900, 700191218]
2022-04-18 10:48:31.454 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.454 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.455 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.466 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为:[700190151]
2022-04-18 10:48:31.466 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.466 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.467 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.471 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为:[700189900]
2022-04-18 10:48:31.472 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.472 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.473 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.478 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为:[700191218]
2022-04-18 10:48:31.478 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.478 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.479 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.494 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为:[700191213]
2022-04-18 10:48:31.494 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.495 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.496 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.506 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为:[700191213]
2022-04-18 10:48:31.506 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.507 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.508 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.512 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为:[700191213]
2022-04-18 10:48:31.512 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.512 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.513 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.517 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为:[700191213]
2022-04-18 10:48:31.517 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.517 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.518 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.549 [ Thread-58 ] [] - [ INFO ] [ com.xxl.job.clm.listener.CommonJobListener : 23 ] - job:inspectorTaskJobInit结束,参数为:{date=1650250111302}
2022-04-18 10:48:31.550 [ Thread-58 ] [] - [ ERROR ] [ com.xxl.job.clm.listener.CommonJobListener : 25 ] - job:inspectorTaskJobInit运行失败,参数为:{date=1650250111302}
2022-04-18 10:48:31.559 [ Thread-58 ] [] - [ INFO ] [ o.s.batch.core.launch.support.SimpleJobLauncher : 142 ] - Job: [FlowJob: [name=inspectorTaskJobInit]] completed with the following parameters: [{date=1650250111302}] and the following status: [FAILED]
2022-04-18 10:48:31.560 [ Thread-57 ] [] - [ DEBUG ] [ com.xxl.job.core.thread.TriggerCallbackThread : 38 ] - >>>>>>>>>>> xxl-job, push callback request, logId:1911245
2022-04-18 10:48:34.842 [ Thread-59 ] [] - [ INFO ] [ o.s.batch.core.launch.support.SimpleJobLauncher : 139 ] - Job: [FlowJob: [name=inspectorTaskJobInit]] launched with the following parameters: [{date=1650250114810}]
2022-04-18 10:48:34.862 [ Thread-59 ] [] - [ INFO ] [ com.xxl.job.clm.listener.CommonJobListener : 18 ] - job:inspectorTaskJobInit开始运行,参数为:{date=1650250114810}
2022-04-18 10:48:34.896 [ SimpleAsyncTaskExecutor-7 ] [] - [ INFO ] [ o.springframework.batch.core.job.SimpleStepHandler : 146 ] - Executing step: [findQuaterInspectorStep]
2022-04-18 10:48:34.903 [ SimpleAsyncTaskExecutor-8 ] [] - [ INFO ] [ o.springframework.batch.core.job.SimpleStepHandler : 146 ] - Executing step: [findDayInspectorStep]
2022-04-18 10:48:34.906 [ SimpleAsyncTaskExecutor-7 ] [] - [ ERROR ] [ org.springframework.batch.core.step.AbstractStep : 229 ] - Encountered an error executing step findQuaterInspectorStep in job inspectorTaskJobInit
org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
查看 chunk()方法里面的大小数是3,也就是读到3个的时候进行提交一次,大家可以看到它的下面的查询语句也是这样进行查询的,会进行查询多次的。