spring-batch

Spring MongoItemReader not reading all records on single execution

十年热恋 提交于 2020-08-26 04:52:59
问题 @Bean public Job orderJob() throws Exception { return jobBuilderFactory.get("orderJob").incrementer(new RunIdIncrementer()).listener(listener()) .flow(orderStep()).end().build(); } @Bean public Step orderStep() throws Exception { return stepBuilderFactory.get("orderStep").<OrderCollection, Order>chunk(1000) .reader(orderReader()).processor(orderProcessor()).writer(orderWriter()) .allowStartIfComplete(true).build(); } @Bean @StepScope public MongoItemReader<OrderCollection> orderReader()

TransactionRequiredException: no transaction is in progress while using JPAItemWriter

∥☆過路亽.° 提交于 2020-08-22 12:03:53
问题 I am facing weird issue with an application which is a spring boot application. Details Here: The app has a spring batch job which uses JpaItemWriter for performing the writes to the database. Also, the application is configured to use hibernate ItemWriter configuration as follows: @Bean(name = "itemWriter") @StepScope public ItemWriter<Record> itemWriter() { JpaItemWriter<Record> itemWriter = new JpaItemWriter<>(); itemWriter.setEntityManagerFactory(emf); return itemWriter; } The batch job

What is the difference between spring scheduled tasks and spring batch jobs

不羁岁月 提交于 2020-08-22 11:42:14
问题 I dont understand the difference between scheduled tasks and batch jobs in spring. By scheduled tasks I mean the ones which are configured like these: @EnableScheduling public class AppConfig{ .. and used like @Scheduled(fixedRate=550) public void doSomething(){ .. By batch jobs I mean these: @EnableBatchProcessing public class AppConfig{ .. and lots of implementations like: Jobs , Job launcher , Steps , ItemReader , ItemWriter ... etc I would like to know the main difference between them

SPRING-BATCH ERROR: No context holder available for step scope

孤者浪人 提交于 2020-08-10 19:45:26
问题 This is my simple job configuration: @Configuration public class Testjob { @Autowired private StepBuilderFactory stepBuilderFactory; @Autowired private JobBuilderFactory jobBuilderFactory; @Bean @org.springframework.batch.core.configuration.annotation.StepScope public Step step1() { return stepBuilderFactory.get("step1") .tasklet((stepContribution, chunkContext) -> { System.out.println("Hello World !"); return RepeatStatus.FINISHED; }).build(); } @Bean @DependsOn("step1") public Job job1() {

Spring Batch - Kafka: KafkaItemReader reads the data ALWAYS from beginning

最后都变了- 提交于 2020-08-10 06:12:19
问题 I'm willing to use Spring Batch for Kafka data consumption. This spring-tips link has a basic example for the same. Here's my reader : @Bean KafkaItemReader<String, String> kafkaItemReader() { var props = new Properties(); props.putAll(this.properties.buildConsumerProperties()); return new KafkaItemReaderBuilder<String, String>() .partitions(0) .consumerProperties(props) .name("customers-reader") .saveState(true) .topic("test-consumer") .build(); } My application.properties file: spring:

Transaction manager not rolling back on Spring Batch job

送分小仙女□ 提交于 2020-08-08 05:26:51
问题 I have a challenge where I need to read "unprocessed" data from an SQL Server database, process the data, then selectively update two to six tables in a DB2 database and then mark that data as processed in the original database on SQL Server. At any point, should anything fail, I want all the updates to rollback. If I have 10 unprocessed items and 9 are good but one fails I still want the 9 good ones to complete and the tenth one to return to it's original state until we can research the

Difference between Spring Cloud Task and Spring Batch?

こ雲淡風輕ζ 提交于 2020-08-06 06:33:04
问题 I went through the Introducing Spring Cloud Task, but things are not clear for the following questions. I'm using Spring Batch 1) What's the use of Spring Cloud Task when we already have the metadata provided by Spring Batch ? 2) We're planning to use Spring Cloud Data Flow to monitor the Spring Batch. All the batch jobs can be imported into the SCDF as task and can be sceduled there, but don't see support for MongoDB. Hope MySQL works well. What is the difference between Spring Cloud Task

Spring batch DB to JSON files [duplicate]

杀马特。学长 韩版系。学妹 提交于 2020-08-05 04:13:29
问题 This question already has answers here : Spring Batch: Writing data to multiple files with dynamic File Name (2 answers) Closed 11 days ago . This question might seem to be a duplicate of this but it is not My requirement is to read data from db using JdbcPagingItemReader and process individual records for some additional processing and in writer create individual json files for each processed item with file name id_of_record_json_fie.txt For example if reader reads 100 records then 100 JSON

Passing info between steps in Spring? [duplicate]

南笙酒味 提交于 2020-08-01 06:38:06
问题 This question already has answers here : How can we share data between the different steps of a Job in Spring Batch? (11 answers) Closed 12 months ago . I'm trying to make a Spring Batch and I have no experience with it. Is it possible to pass information from each batch step or must they be completely independent? For example if I have <batch:step id="getSQLs" next="runSQLs"> <batch:tasklet transaction-manager="TransactionManager" ref="runGetSQLs" /> </batch:step> <batch:step id="runSQLs">

Calling stored procedure with an IN and OUT Parameter from Spring Batch

耗尽温柔 提交于 2020-07-10 08:27:26
问题 I am attempting to execute a stored procedure from Spring Batch, the stored procedure has two parameters, an IN parameter and OUT parameter. What I want is to get the result set and the out parameter when the stored procedure is called. I referred to StoredProcedureItemReader and StoredProcedureItemReaderBuilder I can use this to call a stored procedure that has only IN parameter, however, I can't call after registering OUT parameter. If we refer raw JDBC template we could use, it is possible