Bulk Insert Sql Server millions of record

Question

I have a Windows Service application that receives a stream of data with the following format

IDX|20120512|075659|00000002|3|AALI                 |Astra Agro         


        

Answer 1

First, download free LumenWorks.Framework.IO.Csv library.

Second, use the code like this

StreamReader sr = new TextReader(yourStream);
var sbc = new SqlBulkCopy(connectionString);
sbc.WriteToServer(new LumenWorks.Framework.IO.Csv.CsvReader(sr));

Yeah, it is really that easy.

Answer 2

If you are writing this in C# you might want to look at the SqlBulkCopy class.

Lets you efficiently bulk load a SQL Server table with data from another source.

Answer 3

You can use SSIS "Sql Server Integration Service" for converting data from source data flow to destination data flow. The source can be a text file and destination can be a SQL Server table. Your conversion executes in bulk insert mode.