spring-batch

问题 We have a spring batch job where we trying to process around 10 million records. Now doing this in single thread will be very slow since we have to match SLA. To improve performance, we have developed a POC where master step is creating partitions where each partition represent one unique prod id. This can range from anywhere between 500 to 4500. In POC we have 500 such unique prod id. Now each partition is being given a prod id and step work on it. All this end to end works fine. What we

问题 I have an issue with integrating the spring batch admin console into my application. The good news is that when i go to http://mymachine.com:8080/ReportingManager/batch/configuration I can get to the admin console. But when I go to http://mymachine.com:8080/ReportingManager/batch/job-configuration i get a 404 and an error saying that /batch/jobs can't be found. I've just gone through the exercise of comparing the difference in output of org.springframework.web logs between my application logs

问题 running latest Spring 4.1.0 and spring batch 3.0.1 Uisng <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"> <property name="transactionManager" ref="transactionManager" /> </bean> <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager"/> I have a job that executes every few seconds, it's a very basic ETL job, check if Db has some data transforms and pushes to another system.

问题 I am trying to create a batch using spring batch, spring mvc and spring boot. And I am using log4j2 for logging. My goal is to load an external log4j2.xml configuration file thanks to JVM arguments like that -Dlog4j.configurationFile=file://C:\{path}\Workspace\demo-indexeur\config\log4j2.xml Spring boot detect my file (I have no error in the console) and load it but doesn't apply the configuration I put inside the file. Here is the debug of the load of my configuration file : 2015-05-06 15:36

问题 I am using Spring Batch to implement a batch job, but for some reasons my job is allways null, when I am trying to start my job. It seems to me that my job isn't injected when needed by the Spring framework. Can somebody tell me what I am doing wrong please?! I am a little bit tired of try and error... This is my spring config (batch config): <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:batch="http://www.springframework.org/schema

问题 spring Batch for my project, and I'm simply trying to read from a csv file and load data to a database, using JdbcBatchItemWriter as writer. I'm looking for a way to tell the writer to insert a new row, but, on duplicate key (or duplicate unique identifier) update row instead of failing. I know I can do that directly in the sql statement, but that would be specific to Mysql, though I want my code to be DBMS-independent. here is my writer declaration in the java config @Bean @StepScope public

问题 I'd like to use JpaItemWriter to batch persist entities. But when I use the following code to persist, I'm told: Hibernate: select nextval ('hibernate_sequence') [] 2014-03-19 15:46:02,237 ERROR : TransactionRequiredException: no transaction is in progress How can I enable transactions on the following: @Bean public ItemWriter<T> writer() { JpaItemWriter<T> itemWriter = new JpaItemWriter<>(); itemWriter.setEntityManagerFactory(emf); } @Configuration @EnableTransactionManagement

问题 I'd like to use JpaItemWriter to batch persist entities. But when I use the following code to persist, I'm told: Hibernate: select nextval ('hibernate_sequence') [] 2014-03-19 15:46:02,237 ERROR : TransactionRequiredException: no transaction is in progress How can I enable transactions on the following: @Bean public ItemWriter<T> writer() { JpaItemWriter<T> itemWriter = new JpaItemWriter<>(); itemWriter.setEntityManagerFactory(emf); } @Configuration @EnableTransactionManagement

问题 Use case: A one-time read of data set X (from database) into a Collection C. [Collection size could be say 5000] Use Collection C to process/enrich items in a Spring Batch Step (say enrichStep) If C is much greater than what can be passed via ExecutionContext, how can we make it available in the ItemProcessor of the enrichStep? 回答1: In your enrichStep add a StepExecutionListener.beforeStep and load your huge collection in a HugeCollectionBeanHolder bean. In this way you will load collection

问题 I'm trying to read some records from a database using loops then do some calculations on the records (updating a field called total). But i'm new to spring batch so please can anyone provide me with some tips. 回答1: this sounds like something the chunk pattern would address. you can use re-use existing Spring Batch components to read from the database, compose your own processor, then pass back to a Spring Batch component to store. say the use case is like this; - read a record - record