spring-batch

How can we share data between the different steps of a Job in Spring Batch?

由 a 夏天 提交于 2019-11-26 17:25:57
Digging into Spring Batch, I'd like to know as to How can we share data between the different steps of a Job? Can we use JobRepository for this? If yes, how can we do that? Is there any other way of doing/achieving the same? the job repository is used indirectly for passing data between steps (Jean-Philippe is right that the best way to do that is to put data into the StepExecutionContext and then use the verbosely named ExecutionContextPromotionListener to promote the step execution context keys to the JobExecutionContext . It's helpful to note that there is a listener for promoting

Spring Batch - Looping a reader/processor/writer step

由 孤者浪人 提交于 2019-11-26 16:48:19
问题 ANSWER Based on the accepted answer code the following adjustment to that code worked for me: // helper method to create a split flow out of a List of steps private static Flow createParallelFlow(List<Step> steps) { SimpleAsyncTaskExecutor taskExecutor = new SimpleAsyncTaskExecutor(); taskExecutor.setConcurrencyLimit(steps.size()); Flow[] flows = new Flow[steps.size()]; for (int i = 0; i < steps.size(); i++) { flows[i] = new FlowBuilder<SimpleFlow>(steps.get(i).getName()).start(steps.get(i))

How to set up multi-threading in Spring Batch?

由 懵懂的女人 提交于 2019-11-26 16:17:26
问题 I've successfully set up a tutorial Spring Batch project. I'd really like to know if it's possible to make it multi-threaded at the "Spring level". The basic idea of what I want is to make a list of tasks or task steps and let them be picked up and worked on by independent threads, ideally out of a pool limited to 'n' number of threads. Is this possible? If so, how? Could someone show guide me to that point from where I'm currently at? The simple project I have is from this tutorial here. It

Spring-batch @BeforeStep does not work with @StepScope

由 [亡魂溺海] 提交于 2019-11-26 15:48:19
问题 I'm using Spring Batch version 2.2.4.RELEASE I tried to write a simple example with stateful ItemReader, ItemProcessor and ItemWriter beans. public class StatefulItemReader implements ItemReader<String> { private List<String> list; @BeforeStep public void initializeState(StepExecution stepExecution) { this.list = new ArrayList<>(); } @AfterStep public ExitStatus exploitState(StepExecution stepExecution) { System.out.println("******************************"); System.out.println(" READING

Define an in-memory JobRepository

由 爷，独闯天下 提交于 2019-11-26 14:39:15
问题 I'm testing Spring Batch using Spring boot. My need is to define jobs working on an Oracle Database but I don't want to save jobs and steps states inside this DB. I've read in the documentation I can use a in-memory repository with the MapJobRepositoryFactoryBean. Then, I've implemented this bean: @Bean public JobRepository jobRepository() { MapJobRepositoryFactoryBean factoryBean = new MapJobRepositoryFactoryBean(new ResourcelessTransactionManager()); try { JobRepository jobRepository =

Spring Batch - Using an ItemWriter with List of Lists

由 倖福魔咒の 提交于 2019-11-26 14:20:05
问题 Our processor returns a List<?> (effectively passing a List<List<?>> ) to our ItemWriter . Now, we observed that the JdbcBatchItemWriter is not programmed to handle item instanceof List . We also observed to process item instanceof List ; we need to write a custom ItemSqlParameterSourceProvider . But the sad part is that it returns SqlParameterSource which can handle only one item and again not capable of handling a List . So, can someone help us understand how to handle list of lists in the

Get current resource name using MultiResourceItemReader Spring batch

由 孤者浪人 提交于 2019-11-26 12:48:44
问题 I am using MultiResourceItemReader in Spring Batch for reading multiple XML files and I want to get current resource.Here is my configuration: public class MultiFileResourcePartitioner extends MultiResourceItemReader<MyObject> { @Override public void update(final ExecutionContext pExecutionContext) throws ItemStreamException { super.update(pExecutionContext); if (getCurrentResource() != null && getCurrentResource().getFilename() != null) { System.out.println(\"update:\" + getCurrentResource()

How to get access to job parameters from ItemReader, in Spring Batch?

由 橙三吉。 提交于 2019-11-26 12:04:46
问题 This is part of my job.xml : <job id=\"foo\" job-repository=\"job-repository\"> <step id=\"bar\"> <tasklet transaction-manager=\"transaction-manager\"> <chunk commit-interval=\"1\" reader=\"foo-reader\" writer=\"foo-writer\" /> </tasklet> </step> </job> This is the item reader: import org.springframework.batch.item.ItemReader; import org.springframework.beans.factory.annotation.Value; import org.springframework.stereotype.Component; @Component(\"foo-reader\") public final class MyReader

Difference between spring batch remote chunking and remote partitioning

由 心不动则不痛 提交于 2019-11-26 11:07:50
问题 What is the difference between spring batch remote chunking and remote partitioning? I can not understand the difference between remote chunking and remote partitioning in spring batch. Could anybody please explain? 回答1: Remote Partitioning Partitioning is a master/slave step configuration that allows for partitions of data to be processed in parallel. Each partition is described via some metadata. For example, if you were processing a database table, partition 1 may be ids 0-100, partition 2

How to java-configure separate datasources for spring batch data and business data? Should I even do it?

由 落花浮王杯 提交于 2019-11-26 10:34:51
问题 My main job does only read operations and the other one does some writing but on MyISAM engine which ignores transactions, so I wouldn\'t require necessarily transaction support. How can I configure Spring Batch to have its own datasource for the JobRepository , separate from the one holding the business data? The initial one datasource-configurations is done like the following: @Configuration public class StandaloneInfrastructureConfiguration { @Autowired Environment env; @Bean public
订阅 spring-batch