How to set up multi-threading in Spring Batch?

Question

I've successfully set up a tutorial Spring Batch project. I'd really like to know if it's possible to make it multi-threaded at the "Spring level".

The basic idea of what I want is to make a list of tasks or task steps and let them be picked up and worked on by independent threads, ideally out of a pool limited to 'n' number of threads.

Is this possible? If so, how? Could someone show guide me to that point from where I'm currently at?

The simple project I have is from this tutorial here. It basically has different tasks which print out a message to the screen.

Here's my current simpleJob.xml file, which contains the job details:

<import resource="applicationContext.xml"/>

    <bean id="hello" class="helloworld.PrintTasklet">
        <property name="message" value="Hello"/>
    </bean>

    <bean id="space" class="helloworld.PrintTasklet">
        <property name="message" value=" "/>
    </bean>

    <bean id="world" class="helloworld.PrintTasklet">
        <property name="message" value="World!\n"/>
    </bean>

    <bean id="taskletStep" class="org.springframework.batch.core.step.tasklet.TaskletStep" >
        <property name="jobRepository" ref="jobRepository"/>
        <property name="transactionManager" ref="transactionManager"/>
    </bean>

    <bean id="simpleJob" class="org.springframework.batch.core.job.SimpleJob">
        <property name="name" value="simpleJob" />
        <property name="steps">
            <list>
                <bean parent="taskletStep">
                    <property name="tasklet" ref="hello"/>
                </bean>
                <bean parent="taskletStep">
                    <property name="tasklet" ref="space"/>
                </bean>
                <bean parent="taskletStep">
                    <property name="tasklet" ref="world"/>
                </bean>
            </list>
        </property>
        <property name="jobRepository" ref="jobRepository"/>
    </bean>

My appContext contains the job repository bean (SimpleJobRepository), transaction manager (ResourceLessTransactionManager) and job launcher (SimpleJobLauncher). I can provide this code if desired as well, I just didn't want to bog down this post with tons of XML.

Thanks very very much for any help!

Answer 1

Create a Split and you will be able to use multithreading between the different branches. Use a TaskExecutor to define your parallelism policy.

See Multi-threaded Step

Be sure to use the latest version of Spring Batch if you want to use multithreading and the MapJobRepository (prior to latest version, this JobRepository was not thread safe).

Answer 2

Take a look at Jean answers'. An easy way is create a bean of SimpleAsyncTaskExecutor and associate a tasklet to use this bean, like that.

<bean id="simpleTaskExecutor"
    class="org.springframework.core.task.SimpleAsyncTaskExecutor">
    <property name="concurrencyLimit" value="10"/>
</bean>

<batch:job id="jobTest">
    <batch:step id="step1">
    <!-- throttle-limit default is 4. Increase this to ensure that a thread pool is fully utilized -->
        <batch:tasklet task-executor="simpleTaskExecutor" throttle-limit="20">
            <batch:chunk />
        </batch:tasklet>
    </batch:step>
</batch:job>