spring-batch

there is one csv file having 100 columns, but we need only 3-5 columns which needs to be loaded into database. I dont want to specify all the 100 columns in linetokenizer in job xml. Please suggest how we can proceed in this case arkabhi Try using a custom fieldSetMapper. You can use it similar to a ResultSet with indexes. You have to list all the column names only if you want automatic mapping. Specify only the delimiter, in your case "," <bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step"> <property name="resource" value="YOURFILE" />

I have a simple Spring Batch application that's pulling records from a database and just printing the rows to the screen. Just a simple POC application. The application works just fine with Spring Boot 1.2.1.RELEASE but when I updated to 1.2.3.RELEASE I get an error message about "No qualifying bean of type [javax.sql.DataSource] is defined" I'm not sure if this is a Spring Boot issue or Spring Batch issue. Is there way to define the datasource explicitly for the Spring Batch repository? Full stack trace. org.springframework.beans.factory.BeanCreationException: Error creating bean with name

I have a Spring Boot Batch application that I'm writing integration tests against. When I execute a test, the entire batch application runs. How can I execute just the application code under test? Here's my test code. When it executes, the entire batch job step runs (reader, processor, and writer). Then, the test runs. @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = BatchApplication.class)) @TestExecutionListeners({ DependencyInjectionTestExecutionListener.class, StepScopeTestExecutionListener.class }) public class

I'm having trouble getting a conditional spring batch flow to work using java config. The samples I've seen in spring batch samples, or spring batch's test code, or on stack overflow tend to show a conditional where a single step needs to be executed on condition, or it's the final step, or both. That's not the case I need to solve. In procedural pseudo code, I want it to behave like initStep() if decision1() subflow1() middleStep() if decision2() subflow2() lastStep() So, subflow1 and 2 are conditional, but init, middle and last always execute. Here's my stripped down test case. In the

I am fairly new to Spring and Spring Batch, so feel free to ask any clarifying questions if you have any. I am seeing an issue with Spring Batch that I cannot recreate in our test or local environments. We have a daily job that connects to Websphere MQ via JMS and retrieves a set of records. This job uses the out-of-the-box JMS ItemReader. We implement our own ItemProcessor, but it doesn't do anything special other than logging. There are no filters or processing that should affect incoming records. The problem is that out of the 10,000+ daily records on MQ, only about 700 or so (the exact

I am trying to create a Spring Batch job that will process an xml file that will be served up through a REST call. I am trying to use an XML file that's hosted on the internet to test this. The file is located at: http://www.w3schools.com/xml/plant_catalog.xml I downloaded this file locally and am able to convert it to an object and write it, but I don't know how I can do the same without downloading the file locally. This works locally, but how can I specify a URL as the resource for the xml file to read? Thanks :) launch-context.xml <batch:job id="job1"> <batch:step id="step1"> <batch

I'm trying to create a Spring Batch job using a ListItemReader<String> , ItemProcessor<String, String> and ItemWriter<String> . The XML looks like the following, <job id="sourceJob" xmlns="http://www.springframework.org/schema/batch"> <step id="step1" next="step2"> <tasklet> <chunk reader="svnSourceItemReader" processor="metadataItemProcessor" writer="metadataItemWriter" commit-interval="1" /> </tasklet> </step> <step id="step2"> <tasklet ref="lastRevisionLoggerTasklet"></tasklet> </step> </job> <bean id="svnSourceItemReader" class="com.example.repository.batch.SvnSourceItemReader" scope=

Problem: Read file of size > 10 MB and load it in staging table using Spring Batch. How can we maintain state while reading a file, in order to restart the job if it fails? As per the documentation the FileItemReader is not thread safe and if we try to make it thread safe, we end up loosing restartability. So basic questions are: Is there a way to read the file in blocks and each thread knows which block it needs to read? If we make the read synchronous, what changes are required to make the job restartable in this scenario? If anyone has faced similar issues or have any analysis of how it