Using 2 different datasources : Spring batch

Question

I have 2 different datasources, one to read and another one to write results like below:

• ItemReader should get data from dataSource_1.
• ItemWriter should write data to dataSource_2.

knowing that reader and writer are in the same tasklet.

As per the documentation, we can configure a single transaction manager at tasklet

In this scenario, how do i use the transaction manager here?

I cannot rely on the container and i'm not using ORM layer (JPA..), i use direct JDBC driver to read in database 1 and write into database2.

current conf :

<bean id="dataSource1" class="org.apache.commons.dbcp.BasicDataSource">
    <property name="driverClassName" value="${batch.or.jdbc.driver}" />
    <property name="url" value="${batch.or.jdbc.url}" />
    <property name="username" value="${batch.or.jdbc.user}" />
    <property name="password" value="${batch.or.jdbc.password}" />
</bean>

<bean id="dataSource2" class="org.apache.commons.dbcp.BasicDataSource">
    <property name="driverClassName" value="${batch.caux.jdbc.driver}" />
    <property name="url" value="${batch.caux.jdbc.url}" />
    <property name="username" value="${batch.caux.jdbc.user}" />
    <property name="password" value="${batch.caux.jdbc.password}" />
</bean>

<bean id="baseReader" class="org.springframework.batch.item.database.JdbcCursorItemReader">
        <property name="dataSource" ref="dataSource1" />
</bean>

<bean id="baseWriter" class="org.springframework.batch.item.database.JdbcBatchItemWriter">
        <property name="dataSource2" ref="dataSource2" />
        <property name="sql" value="${batch.param.insert}" />
</bean>

How could i configure the JTA/XA transaction ( Atomikos ) with Spring Batch?

Answer 1

Forget about Spring Batch.

Assume you need to write a application service, which read from one datasource1, and write to datasource2, what will you need to do?

Distributed transaction is the answer for you. That of course involve extra configurations. If you are in a J2EE container (Websphere, Weblogic, JBoss etc), then the transaction manager they provide should be able to handle distributed transaction. You should be able to find the corresponding TransactionManager implementation in Spring for each of these platforms. What you need is only configure the datasource (which should be also under the container) to use XA-aware drivers, and in your app, use the corresponding transaction manager, and lookup the XA-aware datasources by JNDI

However if you cannot rely on the container (e.g. you are writing a standalone app, and etc), you will need to find an transaction manager that is capable on that. Atomikos is one the the most famous free JTA/XA library.

Of course, if you are hundred percent sure that all transaction-aware actions will be in datasource2, you can consider using datasource transaction manager only for datasource2, but honestly it is not a preferred approach I will suggest.

Answer 2

You will need to use a XA compatible driver for your 2 data-sources with a JTA transaction Manager.

see this article and this one if you are not familiar with distributed transactions

regards

Answer 3

If the reader can be outside of the transaction, you can use the writer's trx manager only. If you need the reader and the writer in the same transaction, probably you need a XA compatible transaction manager.