How can I load a large flat file into a database table using SSIS?

Question

I\'m not sure how it works so I\'m looking for the right solution. I think SSIS is the right way to go but I have never used it before

Scenario:

Every mornin

Answer 1

I just want to give my idea for the next guy who may pass by this question. So I'm going to suggest my idea for each scenario's.
1. Getfile from FTP or local.
I would suggest you to use Drop box, Google Drive or any other file syncing cloud services of your choice see this link for detail.
2. I would suggest loading all flat file data to staging table as you suggested Then comparing the data would be easily done by using MERGE between your staging table and Target table on your unique column (ID). You can see this link for how to use merge script. The 2nd & 3rd scenarios will be solved if you are using MERGE Script.
For the last two scenarios i suggest you use SQL JOB to automatically run the package and schedule it at off hours or on time where the server is not busy.Please take a look at the link for detail on how to Run a Package Using a SQL Server Agent Job just type it on your favorite search engine and you will find tons of blogs that shows how its done.