spring-batch

spring batch not processing all records

混江龙づ霸主 提交于 2021-02-04 22:00:58
问题 I am using spring batch to read records from postgresql DB using RepositoryItemReader and then write it to a topic. I see that there were around 1 million records which had to be processed but it didn't process all the records. I have set pageSize for reader as 10,000 and same as commit interval (chunk size) @Bean public TaskletStep broadcastProductsStep(){ return stepBuilderFactory.get("broadcastProducts") .<Product, Product> chunk(10000) .reader(productsReader.repositoryItemReader())

spring batch not processing all records

只愿长相守 提交于 2021-02-04 21:57:18
问题 I am using spring batch to read records from postgresql DB using RepositoryItemReader and then write it to a topic. I see that there were around 1 million records which had to be processed but it didn't process all the records. I have set pageSize for reader as 10,000 and same as commit interval (chunk size) @Bean public TaskletStep broadcastProductsStep(){ return stepBuilderFactory.get("broadcastProducts") .<Product, Product> chunk(10000) .reader(productsReader.repositoryItemReader())

Spring Batch disable Spring Boot AutoConfiguration for specific jobs

坚强是说给别人听的谎言 提交于 2021-02-04 21:52:49
问题 I have multiple jobs for my Spring Batch application, but only a single job uses some specific Spring Boot auto configuration features: a job that uses spring-data-jpa auto configuration, to configure a database for business transactions (not for Spring Batch management) a job that does not use the database at all I have packaged both jobs in the same unit because it makes sense from business perspective. Both jobs will work together and the output of one job will be the input of the other

Spring Batch disable Spring Boot AutoConfiguration for specific jobs

跟風遠走 提交于 2021-02-04 21:52:19
问题 I have multiple jobs for my Spring Batch application, but only a single job uses some specific Spring Boot auto configuration features: a job that uses spring-data-jpa auto configuration, to configure a database for business transactions (not for Spring Batch management) a job that does not use the database at all I have packaged both jobs in the same unit because it makes sense from business perspective. Both jobs will work together and the output of one job will be the input of the other

javax.persistence.TransactionRequiredException: no transaction is in progress - Spring Batch

末鹿安然 提交于 2021-02-04 21:00:21
问题 I am using Spring Boot & Batch example. In this example, I am reading data from XML and writing to MySQL database. Here is the source code: https://www.linkedin.com/pulse/spring-batch-read-xml-prateek-ashtikar/?published=t. while running this example, I am getting below error - javax.persistence.TransactionRequiredException: no transaction is in progress at org.hibernate.internal.AbstractSharedSessionContract.checkTransactionNeededForUpdateOperation(AbstractSharedSessionContract.java:409) ~

DatabaseType not found for product name: [Impala]

一个人想着一个人 提交于 2021-02-04 08:31:06
问题 While trying to setup a Spring boot - spring batch project, I am running into an error stating: DatabaseType not found for product name: [Impala] Full stack trace is as follows: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'configurer': Invocation of init method failed; nested exception is org.springframework.batch.core.configuration.BatchConfigurationException: java.lang.IllegalArgumentException: DatabaseType not found for product name: [Impala] at

How to apply something like @Lazy to Spring Batch?

早过忘川 提交于 2021-02-04 06:53:42
问题 I have a Spring Batch project with multiple jobs (job A, job B, job C,...). When I run a particular job A, I got the log of the job A shows that all of the beans of job B, C,... are created too. Is there any way to avoid the creation of the other beans when job A is launched. I have tried to use @Lazy annotation but it 's seem not working. @Configuration @EnableBatchProcessing public class BatchConfiguration { @Autowired public JobBuilderFactory jobBuilderFactory; @Autowired public

How to apply something like @Lazy to Spring Batch?

不打扰是莪最后的温柔 提交于 2021-02-04 06:53:00
问题 I have a Spring Batch project with multiple jobs (job A, job B, job C,...). When I run a particular job A, I got the log of the job A shows that all of the beans of job B, C,... are created too. Is there any way to avoid the creation of the other beans when job A is launched. I have tried to use @Lazy annotation but it 's seem not working. @Configuration @EnableBatchProcessing public class BatchConfiguration { @Autowired public JobBuilderFactory jobBuilderFactory; @Autowired public

To read Excel can we use Spring batch?

旧巷老猫 提交于 2021-02-02 09:58:18
问题 I want to know if it is possible to use Spring Batch , in order to read from an file Excel and save it in Database. remark : the content of file Excel chang every 2 hours. And if it is not possible with Spring Batch, what other solution can i use 回答1: Go take a look on spring-batch-extensions for Excel. You will find some examples of ExcelItemReader and ExcelItemWriter. Here is the introduction of the spring-batch-extensions project for Excel : Spring Batch extension which contains ItemReader

To read Excel can we use Spring batch?

时光毁灭记忆、已成空白 提交于 2021-02-02 09:56:08
问题 I want to know if it is possible to use Spring Batch , in order to read from an file Excel and save it in Database. remark : the content of file Excel chang every 2 hours. And if it is not possible with Spring Batch, what other solution can i use 回答1: Go take a look on spring-batch-extensions for Excel. You will find some examples of ExcelItemReader and ExcelItemWriter. Here is the introduction of the spring-batch-extensions project for Excel : Spring Batch extension which contains ItemReader