spring-batch

JobLauncherTestUtils throws NoUniqueBeanDefinitionException while trying to test spring batch steps

﹥>﹥吖頭↗ 提交于 2021-01-27 14:51:44
问题 I am using Spring boot and Spring batch. I have defined more than one job. I am trying to build junit to test specific task within a job. Therefor I am using the JobLauncherTestUtils library. When I run my test case I always get NoUniqueBeanDefinitionException. This is my test class: @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = {BatchConfiguration.class}) public class ProcessFileJobTest { @Configuration @EnableBatchProcessing static class TestConfig {

How to implement and limit API calls per second in Spring Rest

眉间皱痕 提交于 2021-01-27 12:36:32
问题 I have spring batch application with spring MVC. In this application, I have to call Google API. There is a restriction of max 4 req per sec for API. Now I have to call google API from inside the spring batch. So I have two questions. q1: How can I implement rest call to Google API. I know about Rest Template but I want that there is any better approach like feign client that we use in microservices. q2: how can I restrict 4 calls per second. In case you have any question. Please let me know

Enum to String conversion during Spring batch import fails

∥☆過路亽.° 提交于 2021-01-27 09:02:35
问题 I'm creating a Spring batch job using Spring Boot and Hiberna but I'm experiencing a problem during the insert. This is the relevant part of the code: @Bean public JdbcBatchItemWriter<OphthalmicLens> writer(DataSource dataSource) throws SQLException { return new JdbcBatchItemWriterBuilder<OphthalmicLens>() .itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>()) .sql("INSERT INTO OphthalmicLens (`createdBy`,`createdDate`,`lastModifiedBy`,`lastModifiedDate`,`sid`,

How can we take the result of `MethodInvokingTaskletAdapter` as a reader in the Spring Batch Step?

限于喜欢 提交于 2021-01-25 07:21:21
问题 How can we take the result of MethodInvokingTaskletAdapter as a reader in the Spring Batch Step? Reference - https://docs.spring.io/spring-batch/docs/current/reference/html/index-single.html#taskletStep and https://github.com/spring-projects/spring-batch/pull/567 Here is the code that I developed JobConfiguration.java @Configuration public class JobConfiguration { @Autowired private JobBuilderFactory jobBuilderFactory; @Autowired private StepBuilderFactory stepBuilderFactory; @Bean public

How can we take the result of `MethodInvokingTaskletAdapter` as a reader in the Spring Batch Step?

人走茶凉 提交于 2021-01-25 07:21:11
问题 How can we take the result of MethodInvokingTaskletAdapter as a reader in the Spring Batch Step? Reference - https://docs.spring.io/spring-batch/docs/current/reference/html/index-single.html#taskletStep and https://github.com/spring-projects/spring-batch/pull/567 Here is the code that I developed JobConfiguration.java @Configuration public class JobConfiguration { @Autowired private JobBuilderFactory jobBuilderFactory; @Autowired private StepBuilderFactory stepBuilderFactory; @Bean public

Invalid column name exception - JdbcPagingItemReader query with alias

痴心易碎 提交于 2021-01-24 07:48:09
问题 Spring batch step fails when JdbcPagingItemReader query has a join and alias. It works fine when I remove the join and just do a simple query from employee table. Below is the code snippet that fails. Did anyone encounter such an issue ? Any help would be appreciated. spring-batch-core-4.0.1.RELEASE spring-boot-2.0.0.RELEASE @Autowired @Bean(destroyMethod = "") @StepScope public JdbcPagingItemReader<String> dbItemReader(final DataSource dataSource, final PreparedStatementSetter paramSetter) {

Invalid column name exception - JdbcPagingItemReader query with alias

我们两清 提交于 2021-01-24 07:47:45
问题 Spring batch step fails when JdbcPagingItemReader query has a join and alias. It works fine when I remove the join and just do a simple query from employee table. Below is the code snippet that fails. Did anyone encounter such an issue ? Any help would be appreciated. spring-batch-core-4.0.1.RELEASE spring-boot-2.0.0.RELEASE @Autowired @Bean(destroyMethod = "") @StepScope public JdbcPagingItemReader<String> dbItemReader(final DataSource dataSource, final PreparedStatementSetter paramSetter) {

Spring batch rollback all chunks & write one file at a time

拥有回忆 提交于 2021-01-21 11:38:22
问题 I am a newbie in Spring batch and I have a couple of questions. Question 1: I am using a MultiResourceItemReader for reading a bunch of CSV files and a JDBC Item writer to update the DB in batches. The commit interval is set to 1000. If there is a file with a 10k records and I encounter a DB error at the 7th batch is there any way I can roll back all the previously committed chunks? Question 2: If there are two files each having 100 records and the commit interval is set to 1000 then the

spring Batch flow job Vs spring composed task

江枫思渺然 提交于 2021-01-21 08:57:25
问题 I want to execute my apps using spring-complex-task and I have already build complex spring-batch Flow Jobs which executes perfectly fine. could you please explain what is difference between spring Batch flow job Vs spring composed task? and which is best among them? 回答1: A composed task within Spring Cloud Data Flow is actually built on Spring Batch in that the transition from task to task is managed by a dynamically generated Spring Batch job. This model allows the decomposition of a batch

Spring Batch and S3 Integration - how to remove null characters first from S3 before start reading?

冷暖自知 提交于 2021-01-19 09:52:40
问题 In my case, we get the FlatFile from the source system and keep it on server and then we push this file to Amazon S3 Bucket due to some automated process. The Source system is Mainframe which somehow puts null characters into FlatFile and its unavoidable for them. Now before we start reading FlatFile we must need to remove null characters (like we do using linux command - tr \'\\000\' \' \' < \"%s\" > \"%s\" ) from the file present in Amazon S3 bucket. So far I don't see a way (not unable to