问题
I have Multiple CSV files in Folder
Example :
Member.CSv
Leader.CSv
I need to load them in to Data base tables .
I have worked on it using ForEachLoop Container ,Data FlowTask, Excel Source and OLEDB Destination
we can do if by using Expressions and Precedence Constraints but how can I do using Script task if I have more than 10 files ..I got Stuck with this one
回答1:
We have a similar issue, our solution is a mixture of the suggestions above.
- We have a number of files types sent from our client on a daily basis.
- These have a specific filename pattern (e.g. SalesTransaction20160218.csv, Product20160218.csv)
- Each of these file types have a staging "landing" table of the structure you expect
- We then have a .net script task that takes the filename pattern and loads that data into a landing table.
- There are also various checks that are done within the csv parser - matching number of columns, some basic data validation, before loading into the landing table
We are not good enough .net programmers to be able to dynamically parse an unknown file structure, create SQL table and then load the data in. I expect it is feasible, after all, that is what the SSIS Import/Export Wizard does (with some manual intervention)
As an alternative to this (the process is quite delicate), we are experimenting with a HDFS data landing area, then it allows us to use analytic tools like R to parse the data within HDFS. After that utilising PIG to load the data into SQL.
来源：https://stackoverflow.com/questions/35152924/how-to-load-multiple-csv-files-into-multiple-tables