spring-batch

Spring batch scope issue while using spring boot

北战南征 提交于 2019-11-27 20:16:41
I'm having Standalone spring batch job. This works perfectly fine when in JUNIT @RunWith(SpringJUnit4ClassRunner.class) //@SpringApplicationConfiguration(classes = KPBootApplication.class) @ContextConfiguration(locations={"classpath:kp-sb.xml"}) public class KPBootApplicationTests { private final static Logger LOG=LoggerFactory.getLogger(KPBootApplicationTests.class); @Autowired ApplicationContext context; @Autowired private JobLauncher jobLauncher; @Autowired private Job job; @Test public void testJob() { final JobParameters jobParameters = new JobParametersBuilder() .toJobParameters();

Could not open JPA EntityManager for transaction; nested exception is java.lang.IllegalStateException

不问归期 提交于 2019-11-27 17:38:12
问题 I am quite new to Spring and Spring-Batch in particular. Still I somehow managed to install the Spring Batch-Admin . I added custom jobs and Hibernate/JPA for persistence. Everything is working as expected, up to the point where the first chunk should be persisted. Then I receive the following error-message: org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is java.lang.IllegalStateException: Already value [org

Spring batch-Delete the flatfile from directory after processed

那年仲夏 提交于 2019-11-27 15:15:03
问题 In spring batch , I am using MultiResourceItemReader to read multiple files from the directory. Then I am using a FlatFileItemReader as a delegate to process individual files. My usecase is to delete the file once it is processed completely(READ-WRITE is done) and then multiResourceitemReader has to pick another file and it has to go on. I tried FileDeletingTasklet to delete file in a directory, but as per Spring docs , the execute method will be called only once. How can I achieve delete on

Spring Batch FlatFileItemWriter - How to use stepExecution.jobId to generate file name

我的未来我决定 提交于 2019-11-27 15:12:19
问题 I have this FileWriter where I'm trying to append the current Job Id to the filename that is generated. <bean id="csvFileWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step"> <property name="resource"> <bean class="org.springframework.core.io.FileSystemResource"> <constructor-arg type="java.lang.String"> <value>${csv.file}_#{stepExecution.jobExecution.jobId}</value> </constructor-arg> </bean> </property> <property name="lineAggregator"> <bean class="org

How to read all files in a folder with spring-batch and MultiResourceItemReader?

主宰稳场 提交于 2019-11-27 14:59:52
问题 I want to configure spring-batch to read all csv files inside a specific folder sequentially. The following does not work because the delegate will try to open a file named *.csv , which of course is invalid. What do I have to change here? @Bean public ItemReader<String> reader() { MultiResourceItemReader<String> reader = new MultiResourceItemReader<>(); reader.setResources(new Resource[] {new FileSystemResource("/myfolder/*.csv")}); reader.setDelegate(new FlatFileItemReader<>(..)); return

Spring Batch - Looping a reader/processor/writer step

你说的曾经没有我的故事 提交于 2019-11-27 14:54:49
ANSWER Based on the accepted answer code the following adjustment to that code worked for me: // helper method to create a split flow out of a List of steps private static Flow createParallelFlow(List<Step> steps) { SimpleAsyncTaskExecutor taskExecutor = new SimpleAsyncTaskExecutor(); taskExecutor.setConcurrencyLimit(steps.size()); Flow[] flows = new Flow[steps.size()]; for (int i = 0; i < steps.size(); i++) { flows[i] = new FlowBuilder<SimpleFlow>(steps.get(i).getName()).start(steps.get(i)).build(); } return new FlowBuilder<SimpleFlow>("parallelStepsFlow") .split(taskExecutor) .add(flows)

Spring Batch skip exception for ItemWriter

人走茶凉 提交于 2019-11-27 14:48:25
I'm trying to use Spring Batch 2.2.5 with Java config. Here is the config that I have: @Configuration @EnableBatchProcessing public class JobConfiguration { @Autowired private JobBuilderFactory jobBuilder; @Autowired private StepBuilderFactory stepBuilder; @Bean @Autowired public Job processDocumentsJob() { return jobBuilder.get("processDocumentsJob") .start(procesingStep()) .build(); } @Bean @Autowired public Step procesingStep() { CompositeItemProcessor<File, DocumentPackageFileMetadata> compositeProcessor = new CompositeItemProcessor<File, DocumentPackageFileMetadata>(); compositeProcessor

Spring Batch - Using an ItemWriter with List of Lists

痞子三分冷 提交于 2019-11-27 14:33:17
Our processor returns a List<?> (effectively passing a List<List<?>> ) to our ItemWriter . Now, we observed that the JdbcBatchItemWriter is not programmed to handle item instanceof List . We also observed to process item instanceof List ; we need to write a custom ItemSqlParameterSourceProvider . But the sad part is that it returns SqlParameterSource which can handle only one item and again not capable of handling a List . So, can someone help us understand how to handle list of lists in the JdbcBatchItemWriter ? Typically, the design pattern is: Reader -> reads something, returns ReadItem

Routing data to multiple files in item writer based on item's property as criteria

℡╲_俬逩灬. 提交于 2019-11-27 14:10:08
问题 I am getting a list of items in my reader. There is a property called Code in each item object having several possible values not known to me before hand. 1) Based on the value of Code in each item, I want to write that particular item in a output file pertaining to that Code . For e.g. if my current item's Code is "abc", the item should be written in to abc.txt in the writer. 2) If there is a Code "xyz" in current item, for which the file is not present, a new file should get created and the

Spring batch to upload a CSV file and insert into database accordingly

徘徊边缘 提交于 2019-11-27 13:22:00
问题 My project has this requirement where user uploads a CSV file which has to be pushed to mysql database. I know we can use Spring batch to process large number of records. But I'm not able to find any tutorial/sample code for this requirement of mine. All the tutorials which I came across just hardcoded the CSV file name in it like below: https://spring.io/guides/gs/batch-processing/ I'll need to use the file uploaded by user and process it accordingly. Any help here would be appreciated.. If