spring-batch

Using FlatFileItemReader with a TaskExecutor (Thread Safety)

匆匆过客 提交于 2019-12-04 04:20:13
There are a lot of examples which use FlatFileItemReader along with TaskExecutor . I provide samples below (both with XML and Java Config): Using Oracle Coherence with Spring Batch Spring Batch Multithreading Example I have used it my self with XML configuration for large CSVs (GB size) writing to database with the out-of-the-box JpaItemWriter . There seem to be no issues even without setting save-state = false or taking any kind of special handling. Now, FlatFileItemReader is documented as not thread-safe . My guess was that JpaItemWriter was "covering" the issue by persisting Sets i.e.

Spring Batch - skippable exception in a TaskletStep

拥有回忆 提交于 2019-12-04 03:52:16
问题 I am trying to cause a job not to have BatchStatus.FAILED if a certain exception occurs. The docs talk about using skippable-exception-classes within <chunk> , but how can I do the same within a TaskletStep ? The below code does not work: <batch:step id="sendEmailStep"> <batch:tasklet> <bean class="com.myproject.SendEmail" scope="step" autowire="byType"> <batch:skippable-exception-classes> <batch:include class="org.springframework.mail.MailException" /> </batch:skippable-exception-classes> <

I am getting DataSource Not Supported when using DataSouceBuilder

▼魔方 西西 提交于 2019-12-04 03:09:50
问题 I am new to Spring-Batch (and Spring in general), and have been following on line documentation to teach myself what I need to do this task. I am trying to connect to a DB2 database. If I declare the DB2 connection with XML like this: <bean id="wcs_dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource"> <property name="driverClassName" value="com.ibm.db2.jcc.DB2Driver" /> <property name="url" value="jdbc:db2://127.0.0.1/DEV" /> <property name="username" value="user" /

Attempt to update step execution id=1 with wrong version (2), where current version is 1

廉价感情. 提交于 2019-12-04 02:55:28
Im using SpringBatch 2.1.7 release core and Infrastructure jars to read a CSV file and save it to DB. Integrated my code with Spring quartz scheduler to run for every minute, The Batch is working fine with reading and writing but its failing with the error "org.springframework.dao.OptimisticLockingFailureException: Attempt to update step execution id=1 with wrong version (2), where current version is 1" due to Tx conflicts. Please suggest how can i resolve this issue. I had this same exception. org.springframework.dao.OptimisticLockingFailureException: Attempt to update step execution id=0

Caused by: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '10' for key 'PRIMARY' - in Spring Batch

时间秒杀一切 提交于 2019-12-04 02:33:40
问题 I am developing Spring Boot + Batch example and I am contineously getting the below error Error: 2018-12-08 17:31:40.333 ERROR 19576 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step step1 in job job org.springframework.dao.DuplicateKeyException: PreparedStatementCallback; SQL [INSERT INTO CUSTOMER VALUES (?, ?, ?, ?)]; Duplicate entry '10' for key 'PRIMARY'; nested exception is java.sql.BatchUpdateException: Duplicate entry '10' for key 'PRIMARY' at org

spring batch file writer to write directly to amazon s3 storage without PutObjectRequest

坚强是说给别人听的谎言 提交于 2019-12-04 02:33:18
问题 I'm trying to upload a file to amazon s3. Instead of uploading, I want to read the data from database using spring batch and write the file directly into the s3 storage. Is there anyway we can do that ? 回答1: Spring Cloud AWS adds support for the Amazon S3 service to load and write resources with the resource loader and the s3 protocol. Once you have configured the AWS resource loader, you can write a custom Spring Batch writer like: import java.io.OutputStream; import java.util.List; import

how can I batchUpdate with a query that requires 2 parameters and only one of them is stored in a list

家住魔仙堡 提交于 2019-12-04 02:24:12
问题 I use Spring-JDBC to insert the list of facebook friends for a user in my MySQL database. I have a final Long that contains the user uid and a List that contains the list of his friends. my query is: final String sqlInsert="insert into fb_user_friends(fb_uid,friend_uid) values(?,?)"; I create batch parameters using SqlParameterSourceUtils SqlParameterSource[] batch = SqlParameterSourceUtils.createBatch(friendsList.toArray()); and I execute the insert using: int[] insertCounts = this.

Spring batch retry mechanism for reader failure

三世轮回 提交于 2019-12-04 02:20:17
问题 I have one spring batch job which will run daily once. I had implemented reader, processor and writer. Suppose if any exception happens while running reader task, then entire job will fail. I want to rerun same day for that failure jobs after 5 mins or immediately. Kindly let me know how i can implement in spring batch or provide me any sample code or website which have information. 回答1: Take a look at Spring Retry. It started as part of Spring Batch but as of version 2.2.0 it has spined-off

Need to configure my JPA layer to use a TransactionManager (Spring Cloud Task + Batch register a PlatformTransactionManager unexpectedly)

蹲街弑〆低调 提交于 2019-12-03 23:33:09
问题 I am using Spring Cloud Task + Batch in a project. I plan to use different datasources for business data and Spring audit data on the task. So I configured something like: @Bean public TaskConfigurer taskConfigurer() { return new DefaultTaskConfigurer(this.singletonNotExposedSpringDatasource()); } @Bean public BatchConfigurer batchConfigurer() { return new DefaultBatchConfigurer(this.singletonNotExposedSpringDatasource()); } whereas main datasource is autoconfigured through

load external config file in spring batch admin

女生的网名这么多〃 提交于 2019-12-03 22:32:48
问题 I downloaded the spring batch admin application in the github repo and I imported it in eclipse. it works perfectly. Then, I asked myself how to import an external config file into the application that I can use in my job definition class. I tried this : VM Arguments -Dspring.config.location=C:/path/to/config/file/application.properties Job configuration src/main/java org.springframework.batch.admin.sample.job ------------------------------------------- @Configuration public class