how to load data faster with talend and sql server

Question

I use Talend to load data into a sql-server database.

It appears that the weakest point of my job is not the dataprocessing, but the effective load in my database, w

Answer 1

I had the same problem and solved it by defining an index on target table.

Usually, the target table has an id field which is its primary key and hence indexed. So, all sort of joins with it would work just fine. But the update from a flat file is done by some data fields, so each update statement have to make full table scan.

The above also explains why it works fast with INSERT and becomes slow with INSERT OR UPDATE