spring-batch

问题 In spring batch reference, there is a pattern called driving query pattern https://docs.spring.io/spring-batch/4.1.x/reference/html/common-patterns.html#drivingQueryBasedItemReaders I don't know if I misunderstand this pattern right, doesn't driving query pattern cause N+1 problem? It looks like a classic N+1 problem to me, especially they are bad practice in ORM world. 回答1: Yes it does. For each item, the processor will issue another query to grab additional information about the current

问题 I have started using spring from last few months and I have a question on transactions. I have a java method inside my spring batch job which first does a select operation to get first 100 rows with status as 'NOT COMPLETED' and does a update on the selected rows to change the status to 'IN PROGRESS'. Since I'm processing around 10 million records, I want to run multiple instances of my batch job and each instance has multiple threads. For a single instance, to make sure two threads are not

问题 I am new to spring batch and there is a requirement for me to read the data from DB and write in to JSON format. whats the best way to do this ? Any api's are there? or we need to write custom writer ? or i need to user JSON libraries such as GSON or JACKSON ? Please guide me... 回答1: To read data from a relational database, you can use one of the database readers. You can find an example in the spring-batch-samples repository. To write JSON data, Spring Batch 4.1.0.RC1 provides the

问题 I need to deliver some messages with high guarantee. This messages should be delivered as limited collections (100 at least as example). To resolve my task I think I should use spring batch. I need to configure exponentially increasing time for each next failed attempt. Each data object should have 5 delivery attempts as max. I expect to have a chance monitor all attempts and next retry time via db. Spring batch provides prepared tables for that. It seems my flow have only one job, but

问题 this is my first Batch-Project where I want to avoid writing XML-Files and use for configurations Java-Config. So I have few Batch-Jobs and application works as expected. For documentation reasons or presentation I would like to use the possibility of Spring-IDE to show batch-graph what works very good when using spring-config in XML. But I am not available to find how I can use Batch-Graph when Batch-Job is configured in Java. I already scanned all spring-configuation within Eclipse, so in

问题 this is my first Batch-Project where I want to avoid writing XML-Files and use for configurations Java-Config. So I have few Batch-Jobs and application works as expected. For documentation reasons or presentation I would like to use the possibility of Spring-IDE to show batch-graph what works very good when using spring-config in XML. But I am not available to find how I can use Batch-Graph when Batch-Job is configured in Java. I already scanned all spring-configuation within Eclipse, so in

问题 I have an existing spring batch project that have multiple steps. I want to modify a step so I can stop the job : jobExecution.getStatus() == STOPPED . My step : @Autowired public StepBuilderFactory stepBuilderFactory; @Autowired private StepReader reader; @Autowired private StepProcessor processor; @Autowired private StepWriter writer; @Autowired public GenericListener listener; @Bean @JobScope @Qualifier("mystep") public Step MyStep() throws ReaderException { return stepBuilderFactory.get(

问题 I am using Spring batch with spring boot for process my Csv file. When i run the application i am getting below error trace. 2018-08-27 16:23:35.694 INFO 12016 --- [nio-9004-exec-1] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: ORACLE 2018-08-27 16:23:36.025 INFO 12016 --- [nio-9004-exec-1] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor. 2018-08-27 16:23:36.898 INFO 12016 --- [nio-9004-exec-1] o

问题 In my Spring Batch application I have written a CustomItemWriter which internally writes item to DynamoDB using DynamoDBAsyncClient, this client returns Future object. I have a input file with millions of record. Since CustomItemWriter returns future object immediately my batch job exiting within 5 sec with status as COMPLETED, but in actual it is taking 3-4 minutes to write all item to the DB, I want that batch job finishes only after all item written to DataBase. How can i do that? job is

问题 What are the ways to log exact item which failed during writing a chunk of say 10 items size ? Is there a way to catch it in onWriteError method of ItemWriteListener ? Is it possible to know this by extending ItemWriter interface ? Thanks and Regards, Nik 回答1: you could configure skip logic with Integer.MAX_VALUE = 2.147.483.647 <step id="step1"> <tasklet> <chunk reader="flatFileItemReader" writer="itemWriter" commit-interval="10" skip-limit="2147483647"> <skippable-exception-classes>