Do I need a ETL?

Question

We currently use Datastage ETL to - Export a CSV/text file with data from 15 tables(3 different schemas) on a daily basis.

I am wondering If there is a simpler way t

Answer 1

My 2 cents.

Datastage is an awful tool, and expensive to license.

SSIS is much simpler, or cloverETL is good.

ETL tool vs code is a good question.

ETL tools often have better performance as can queue data up ready to be used where programming is is going to do this one at a time, and datastage can do this in parallel (but again i think it blows). PLus ETL tools can get data from multiple heterogeneous sources, where as you cant do this (easily) with code.

However if any data transformations etc are all to be done with data on the same server, I generally end up doing as much in SQL/TSQL(or PL/SQL) as possible, as it is just tonnes easier to debug/maintain. Primary Keys/Foreign Keys are your friend, and any missed lookups can be checked through checking counts later on to ensure data integrity is in order.