bulkinsert

I'm writing my graduate work about methods of importing data from file to SQL Server table. I have created my own programm and now I'm comparing it with some standart methods as bcp, BULK INSERT, INSERT ... SELECT * FROM OPENROWSET(BULK...). My program read in lines from file, parse them and import them one by one using ordinary INSERTs. I have generated file with 1 million lines each with 4 columns for tests. And now I have situation that my program works 160 seconds while standart methods 5-10 seconds. So qestion is why BULK operations are faster than 1 million INSERTs? Do they use special

In my application I need to massively improve insert performance. Example: A file with about 21K records takes over 100 min to insert. There are reasons it can takes some time, like 20 min or so but over 100 min is just too long. Data is inserted into 3 tables (many-to-many). Id's are generated from a sequence but I have already googled and set hibernate.id.new_generator_mappings = true and allocationSize + sequence increment to 1000. Also the amount of data is not anything extraordinary at all, the file is 90 mb. I have verified with visual vm that most of the time is spent in jdbc driver

I have this csv named test.csv with the content below 1,"test user",,,4075619900,example@example.com,"Aldelo for Restaurants","this is my deal",,"location4" 2,"joe johnson",,"32 bit",445555519,antle@gmail.com,"Restaurant Pro Express","smoe one is watching u",,"some location" Here is my SQL FILE to do the BULK insert USE somedb GO CREATE TABLE CSVTemp (id INT, name VARCHAR(255), department VARCHAR(255), architecture VARCHAR(255), phone VARCHAR(255), email VARCHAR(255), download VARCHAR(255), comments TEXT, company VARCHAR(255), location VARCHAR(255)) GO BULK INSERT CSVTemp FROM 'c:\test\test

I am trying to find out how I can improve my insert performance in a temporary table in SQL Server using c#. Some people are saying that I should use SQLBulkCopy however I must be doing something wrong as it seems to work much slower than simply building an SQL insert string instead. My code to create table using SQLBulkCopy is below: public void MakeTable(string tableName, List<string> ids, SqlConnection connection) { SqlCommand cmd = new SqlCommand("CREATE TABLE ##" + tableName + " (ID int)", connection); cmd.ExecuteNonQuery(); DataTable localTempTable = new DataTable(tableName); DataColumn

I am trying to run the following command to bulk insert data from a CSV file-- BULK INSERT TestDB.dbo.patent FROM 'C:\1patents.csv' WITH (FIRSTROW = 1, FIELDTERMINATOR = '^', ROWTERMINATOR='\n'); The error I am getting is this-- Msg 4866, Level 16, State 1, Line 1 The bulk load failed. The column is too long in the data file for row 1, column 6. Verify that the field terminator and row terminator are specified correctly. Msg 7399, Level 16, State 1, Line 1 The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error. Msg

Hi I am using 16 collections to insert around 3-4 million json objects ranging from 5-10k per object.I am using stored procedure to insert these documents.I have 22 Capacity Unit. function bulkImport(docs) { var collection = getContext().getCollection(); var collectionLink = collection.getSelfLink(); // The count of imported docs, also used as current doc index. var count = 0; // Validate input. if (!docs) throw new Error("The array is undefined or null."); var docsLength = docs.length; if (docsLength == 0) { getContext().getResponse().setBody(0); } // Call the CRUD API to create a document.