.Net vs SSIS: What should SSIS be used for?

Question

If I have the option of using .Net and can do data transformations just fine in .Net, when would I need SSIS? Is there a certain task

Answer 1

Bit late to answer this question but I hope it worth,

SSIS is often misunderstood when compared to programming language. SSIS is a framework whereas C# is a language on .NET Framework. I have extensive experience in handling & developing large data warehousing solution using (MSBI suite) and also had developed large websites (ASP.NET) - so I can't be biased.

SSIS if not used properly can decrease performance by par. SSIS packages have three kind of transformation:

1. Blocking transformation - which can only pass the data when above transformation is completed fetching all the rows and completed required calculation on it.
2. Semi-Blocking transformation - Which can pass partial data
3. Non-Blocking - which process the row as soon as it is ready

SSIS works exceptionally good with non blocking transformation with proper setting on control flow and data flow. I have used it on larger (over 2 TB of data warehouse) and I can guarantee that it was the fastest load experience. You can check Microsoft blog about We Loaded 1TB in 30 Minutes with SSIS, and So Can You

I agree that SSIS degraded performance when dealing with blocking transformation and they should be carried by T-SQL whenever required.

Coming to C#, I accept that SSIS uses .NET framework and data provider to accomplish task. But C#, as a language is bit more logical and must be treated to deal with business logic. For example, If we have to run exe with different parameter based on condition, you can write a package which will consider parameters and then logically decide what parameter needs to be passed to run an exe file. It would be lengthy process to do that in SSIS while I can do that easily in C# because logical thing can be easily done in language instead of a framework.

Now the point here is what is more convenient approach to solve your problem statement. SSIS is a sure winner loading large amount of records loading data from source to destination while C# is perfect for writing logic. Even if you like C#, I won't recommend you to choose for doing ETL (Extract Transform Load) operation on large data warehouse systems.