spring-batch

Spring Batch - skippable exception in a TaskletStep

▼魔方 西西 提交于 2019-12-01 21:07:54
I am trying to cause a job not to have BatchStatus.FAILED if a certain exception occurs. The docs talk about using skippable-exception-classes within <chunk> , but how can I do the same within a TaskletStep ? The below code does not work: <batch:step id="sendEmailStep"> <batch:tasklet> <bean class="com.myproject.SendEmail" scope="step" autowire="byType"> <batch:skippable-exception-classes> <batch:include class="org.springframework.mail.MailException" /> </batch:skippable-exception-classes> </bean> </batch:tasklet> </batch:step> I implemented this functionality in the Tasklet as Michael Minella

Issue with configuring Atomikos on a Spring Boot / Spring Batch application

*爱你&永不变心* 提交于 2019-12-01 20:58:21
I am trying to get Atomikos to work with my Spring Boot/Spring Batch application. Here is are the relevant portions of my code: Datasource config: @Configuration public class DatasourceConfiguration extends AbstractCloudConfig { @Bean @Qualifier("batch_database") public DataSource batchDatasource() { return connectionFactory().dataSource("batch_database"); } @Bean public PlatformTransactionManager transactionManager(){ return new JtaTransactionManager(); } @Bean public TaskConfigurer configurer(){ return new DefaultTaskConfigurer(batchDatasource()); } } Atomikos auto-config dependency: compile

Spring Batch correctly restart uncompleted jobs in clustered environment

偶尔善良 提交于 2019-12-01 19:45:42
I used the following logic to restart the uncompleted jobs on single-node Spring Batch application: public void restartUncompletedJobs() { try { jobRegistry.register(new ReferenceJobFactory(documetPipelineJob)); List<String> jobs = jobExplorer.getJobNames(); for (String job : jobs) { Set<JobExecution> runningJobs = jobExplorer.findRunningJobExecutions(job); for (JobExecution runningJob : runningJobs) { runningJob.setStatus(BatchStatus.FAILED); runningJob.setEndTime(new Date()); jobRepository.update(runningJob); jobOperator.restart(runningJob.getId()); } } } catch (Exception e) { LOGGER.error(e

I am getting DataSource Not Supported when using DataSouceBuilder

…衆ロ難τιáo~ 提交于 2019-12-01 16:38:06
I am new to Spring-Batch (and Spring in general), and have been following on line documentation to teach myself what I need to do this task. I am trying to connect to a DB2 database. If I declare the DB2 connection with XML like this: <bean id="wcs_dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource"> <property name="driverClassName" value="com.ibm.db2.jcc.DB2Driver" /> <property name="url" value="jdbc:db2://127.0.0.1/DEV" /> <property name="username" value="user" /> <property name="password" value="pass5" /> </bean> Then load it in my code like so: @Bean public

Caused by: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '10' for key 'PRIMARY' - in Spring Batch

心不动则不痛 提交于 2019-12-01 14:31:55
I am developing Spring Boot + Batch example and I am contineously getting the below error Error: 2018-12-08 17:31:40.333 ERROR 19576 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step step1 in job job org.springframework.dao.DuplicateKeyException: PreparedStatementCallback; SQL [INSERT INTO CUSTOMER VALUES (?, ?, ?, ?)]; Duplicate entry '10' for key 'PRIMARY'; nested exception is java.sql.BatchUpdateException: Duplicate entry '10' for key 'PRIMARY' at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate

how can I batchUpdate with a query that requires 2 parameters and only one of them is stored in a list

℡╲_俬逩灬. 提交于 2019-12-01 14:18:05
I use Spring-JDBC to insert the list of facebook friends for a user in my MySQL database. I have a final Long that contains the user uid and a List that contains the list of his friends. my query is: final String sqlInsert="insert into fb_user_friends(fb_uid,friend_uid) values(?,?)"; I create batch parameters using SqlParameterSourceUtils SqlParameterSource[] batch = SqlParameterSourceUtils.createBatch(friendsList.toArray()); and I execute the insert using: int[] insertCounts = this._jdbcTemplate.batchUpdate(sqlInsert,batch); the problem here that the list contains only the 2nd parameter that

Performance optimization for processing of 115 million records for inserting into Oracle

旧时模样 提交于 2019-12-01 13:11:05
I have a requirement where I am reading a text file placed in Unix of size 19 GB and having records around 115 million. My Spring Batch (Launcher) is getting triggered by Autosys and Shell script once the file is placed in the location. Initially on execution of this process it took around 72hrs to read, process(Null checks and date parsing) and write the data into the Oracle database. But after certain configuration changes like using Throttle Limit, Task Executor etc, I was able to reduce execution time to 28hrs currently. I need this process to be complete in 4hrs,also, using SQL loader

Spring batch retry mechanism for reader failure

感情迁移 提交于 2019-12-01 13:01:09
I have one spring batch job which will run daily once. I had implemented reader, processor and writer. Suppose if any exception happens while running reader task, then entire job will fail. I want to rerun same day for that failure jobs after 5 mins or immediately. Kindly let me know how i can implement in spring batch or provide me any sample code or website which have information. Take a look at Spring Retry . It started as part of Spring Batch but as of version 2.2.0 it has spined-off to a standalone project (and dependency). Being able to use it declaratively is particularly nice: import

Unexpected token (START_OBJECT), expected VALUE_STRING: need JSON String that contains type id (for subtype of java.lang.Object)

风格不统一 提交于 2019-12-01 12:57:32
I already went through the below few links, but it did not solved my problem: Java Jackson: deserialize complex polymorphic object model: JsonMappingException: Unexpected token (START_OBJECT), expected VALUE_STRING Java Jackson - Unexpected token (START_ARRAY), expected VALUE_STRING In my project, we've 15 batch jobs. I am converting 15 batch jobs into 15 different micro-services. For that I have created one Spring Boot module, which has Entity classes, JPA repository layer and service layer. That jar I will import into each micro-services (batch jobs), so that batch job should work fine. I am

Grouping/summarizing spring-batch records

淺唱寂寞╮ 提交于 2019-12-01 12:55:14
I'm new to spring-batch and I want to use it to implement the following proces: read: a list of items from webservice process: groups these items write: these groups back to webservice Example: Input: transactionID, store, amount 1, FooStore, 2.50 2, BarStore, 19.99 3, FooStore, 12,49 Output: totalsID, store, amount 1, FooStore, 14.99 2, BarStore, 19.99 Is this even possible using spring-batch, am I missing an important step, or does this conflict with the concept of batch processing to such an extent that I should look for a different solution? I have found a possible solution, that is by