how to load data faster with talend and sql server

Question

I use Talend to load data into a sql-server database.

It appears that the weakest point of my job is not the dataprocessing, but the effective load in my database, w

Answer 1

Recommend a couple simple things:

1. Where possible / reasonable, change from ETL to ELT.
2. Set up a CDC process and only handle the changes. Depending on the database and needs, this can be handled (a) directly on the database, (b) through automated Talend functionality (need a subscription), (c) manually via SQL (full outer join) and a custom Java function that generates an MD5 hash, or (d) manually via SQL (full outer join) and the tAddCRCRow component.
3. Where possible, load multiple tables concurrently.
4. Where possible, use bulk loading for tables.
5. Sometimes, a clear and load is acceptable as an approach and faster than checking for updates.