spring-batch

Spring batch with column alias as sort key - malformed “where” statement

喜夏-厌秋 提交于 2019-12-19 23:17:10
问题 I'm using Spring-batch version 3.0.6.RELEASE to query a MySQL DB for some data and then process it. Details about usage: query provider is MySqlPagingQueryProvider when setting up the query provider I'm specifying an alias of one of the columns in the query as the sort key (please check the query below for more details) Query: SELECT target.tx_timestamp AS event_datetime, .... FROM some_table AS target .... WHERE target.tx_timestamp > :startTime AND target.tx_timestamp <= :endTime; The code:

Spring Batch correctly restart uncompleted jobs in clustered environment

余生长醉 提交于 2019-12-19 21:24:42
问题 I used the following logic to restart the uncompleted jobs on single-node Spring Batch application: public void restartUncompletedJobs() { try { jobRegistry.register(new ReferenceJobFactory(documetPipelineJob)); List<String> jobs = jobExplorer.getJobNames(); for (String job : jobs) { Set<JobExecution> runningJobs = jobExplorer.findRunningJobExecutions(job); for (JobExecution runningJob : runningJobs) { runningJob.setStatus(BatchStatus.FAILED); runningJob.setEndTime(new Date()); jobRepository

How to skip lines with ItemReader in Spring-Batch?

删除回忆录丶 提交于 2019-12-19 11:25:51
问题 I have a custom item reader that transforms lines from a textfile to my entity: public class EntityItemReader extends AbstractItemStreamItemReader<MyEntity> { @Override public MyEntity read() { String line = delegate.read(); //analyze line and skip by condition //line.split //create entity with line values } } This is similar to the FlatFileItemReader . The read MyEntity will then be persisted to a DB by a JdbcItemReader . Problem: sometimes I have lines that contain values that should be

Spring Batch - Reading multiple line log message

情到浓时终转凉″ 提交于 2019-12-19 11:24:50
问题 I am facing a problem to read multi-line log message as a single message in our spring batch application configured with spring integration, this application has to read multiline log message (example exception stack trace) as a single message, later it has to process and classify the message for further indexing. Each line is identified by its timestamp (pattern mentioned above i.e. DATE_PATTERN) and it may continue mutltiple lines, I am trying to continue reading a message until I see

Spring Batch - Reading multiple line log message

一个人想着一个人 提交于 2019-12-19 11:22:28
问题 I am facing a problem to read multi-line log message as a single message in our spring batch application configured with spring integration, this application has to read multiline log message (example exception stack trace) as a single message, later it has to process and classify the message for further indexing. Each line is identified by its timestamp (pattern mentioned above i.e. DATE_PATTERN) and it may continue mutltiple lines, I am trying to continue reading a message until I see

Grouping/summarizing spring-batch records

廉价感情. 提交于 2019-12-19 11:17:49
问题 I'm new to spring-batch and I want to use it to implement the following proces: read: a list of items from webservice process: groups these items write: these groups back to webservice Example: Input: transactionID, store, amount 1, FooStore, 2.50 2, BarStore, 19.99 3, FooStore, 12,49 Output: totalsID, store, amount 1, FooStore, 14.99 2, BarStore, 19.99 Is this even possible using spring-batch, am I missing an important step, or does this conflict with the concept of batch processing to such

I would like to create a spring batch project where batch does not use my datasource

浪子不回头ぞ 提交于 2019-12-19 10:42:47
问题 I have seen a lot of examples of Spring Batch projects where either (a) a dataSource is defined, or (b) no dataSource is defined. However, in my project, I would like my business logic to have access to a dataSource, but I want Spring Batch to NOT use the dataSource. Is this possible? This guy has a similar problem: Spring boot + spring batch without DataSource 回答1: Generally, using spring-batch without a database is not a good idea, since there could be concurrency issues depending on the

Spring Batch doesn't use custom datasource to create tables

☆樱花仙子☆ 提交于 2019-12-19 10:20:15
问题 I am working on REST-service (with Spring boot), which runs batch job. I want Batch to work only with embedded datasource (to store metadata), while default datasource (Postgres in my case) will be used to store buisness entities. The problem is that Batch tries to create metadata tables (like batch_job_execution , batch_job_instance , etc.) in default datasource at startup. Here is the sample configuration, which reproduces the problem: BatchConfiguration @Configuration

Spring Batch and Pivotal Cloud Foundry [closed]

佐手、 提交于 2019-12-19 09:00:03
问题 Closed . This question needs to be more focused. It is not currently accepting answers. Want to improve this question? Update the question so it focuses on one problem only by editing this post. Closed 3 years ago . We are evaluating Spring Batch framework to replace our home grown batch framework in our organization and we should be able to deploy the batch in Pivotal Cloud Foundry (PCF). In this regard, can you let us know your thoughts on the issue below: Let us say if we use Remote

Is it a bad idea to change the Spring Batch Meta-Data tables manually?

扶醉桌前 提交于 2019-12-18 17:04:11
问题 Background I'm using Spring Batch 2.1.8, and run jobs by CommandLineJobRunner . Such as: java org.springframework.batch.core.launch.support.CommandLineJobRunner classpath:launchContext.xml theJobId Problem At some condition such as a server crash, running job could be interrupted. But the interrupted job left a STARTED status in the Spring Batch Meta-Data tables, and can't be run again. org.springframework.batch.core.repository.JobExecutionAlreadyRunningException: A job execution for this job