DataLoader Using Spring Batch - Partitioning

问题

I am working on a data loader that reads the flat file , some processing and write to the database. The dataloader.properties files contains the value

LOAD=MM1,MM2,MM3,MM4,MM5...

I have to read this properties file and using spring batch's partition step and partitioner , i want all the files inside the folder(MM1), do the reading and writing to db parallel, any db error for a particular file should rollback the content of that file, similarly for the other folders(MM2)As i am new to spring batch , i want to know how to do late binding for the folder and the file names in the XML config file , with appropriate rollback necessary. I also want to share the data within the folder partition step

Following is the rough configuration of the job that does this. is it a correct approach according to the above requirement ?

<!--  The Data Loading Configuration goes here  -->      

<job id="partition${jobname}Job" name="${jobname}" xmlns="http://www.springframework.org/schema/batch">
    <step id="exchangestep" next="filestep">
        <partition step="step1" partitioner="exchangepartitioner">
            <handler grid-size="999" task-executor="taskExecutor" />
        </partition>
    </step>
    <step id="filestep">
        <partition step="step1" partitioner="filepartitioner">
            <handler grid-size="99" task-executor="taskExecutor" />
        </partition>
    </step>
</job>

<bean id="folderpartitioner" class="org.springframework.batch.core.partition.support.MultiResourcePartitioner">
    <property name="resources" value="#${folder}" />
</bean>

<bean id="filepartitioner" class="org.springframework.batch.core.partition.support.MultiResourcePartitioner">
    <property name="resources" value="#${file}" />
</bean>

<bean id="taskExecutor" class="org.springframework.core.task.SyncTaskExecutor" />

<step id="step1" xmlns="http://www.springframework.org/schema/batch">
    <tasklet transaction-manager="transactionManager">
        <chunk writer="itemWriter" reader="itemReader" processor="itemProcessor" commit-interval="5000" />
    </tasklet>
</step>

<bean id="itemReader" scope="step" autowire-candidate="false" parent="itemReaderParent">
    <property name="resource" value="#{stepExecutionContext[fileName]}" />
</bean>

<bean id="multifileReader" class="org.springframework.batch.item.file.MultiResourceItemReader">
    <property name="resources" value="#${filename}" />
    <property name="delegate" ref="fileItemReader" />
</bean>

<bean id="fileItemReader" parent="itemReaderParent" />


<!--  CONFIGURE THE FLAT FILE ITEM READER TO READ INDIVIDUAL BATCH -->
<bean id="itemReaderParent" class="org.springframework.batch.item.file.FlatFileItemReader" abstract="true">
    <property name="lineMapper">
        <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
            <property name="lineTokenizer">
                <bean class="org.springframework.batch.item.file.transform.FixedLengthTokenizer">
                    <property name="names" value="${columns}" />
                    <property name="columns" value="${range}" />
                </bean>
            </property>
            <property name="fieldSetMapper">
                <bean class="org.springframework.batch.item.file.mapping.FieldSetMapper">
                    <property name="targetType" value="DataLoaderMapper" />
                </bean>
            </property>
        </bean>
    </property>
</bean>

来源：https://stackoverflow.com/questions/7878122/dataloader-using-spring-batch-partitioning

标签

java

transactions

spring-batch