bulkinsert

I have about 6500 files for a sum of about 17 GB of data, and this is the first time that I've had to move what I would call a large amount of data. The data is on a network drive, but the individual files are relatively small (max 7 MB). I'm writing a program in C#, and I was wondering if I would notice a significant difference in performance if I used BULK INSERT instead of SQLBulkCopy. The table on the server also has an extra column, so if I use BULK INSERT I'll have to use a format file and then run an UPDATE for each row. I'm new to forums, so if there was a better way to ask this

I have to execute ~6k update queries on a table through sql (No Hibernate/JDBC). Query is something like update A set some_id='value1' where id='value2' It takes too long to execute all these queries. Is there a way to improve the performance? Create a temp table (containing just the value1 and value2 values) and populate it in bulk (ie, you can potentially do this with a single insert statement). Then do an update using a join between your existing table and the temp table. Something like INSERT INTO SomeTempTable(id, some_id) VALUES (1,2), (3,4), (5,6), ....... UPDATE A INNER JOIN

Could someone please explain how does BULK INSERT internally work and why is it much faster than the normal INSERT operations ? Regards, Shishir. BULK INSERT runs in-process with the database engine of SQL Server and thus avoids passing data through the network layer of the Client API - this makes it faster than BCP and DTS / SSIS. Also, with BULK INSERT, you can specify the ORDER BY of the data, and if this is the same as the PK of the table, then the locking occurs at a PAGE level. Writes to the transaction logs happen at a page level rather than a row level as well. In the case of regular

What is the fastest way to insert 237 million records into a table that has rules (for distributing data across child tables)? I have tried or considered: Insert statements. Transactional inserts ( BEGIN and COMMIT ). The COPY FROM command. http://pgbulkload.projects.postgresql.org/ Inserts are too slow (four days) and COPY FROM ignores rules (and has other issues). Example data: station_id,taken,amount,category_id,flag 1,'1984-07-1',0,4, 1,'1984-07-2',0,4, 1,'1984-07-3',0,4, 1,'1984-07-4',0,4,T Table structure (with one rule included): CREATE TABLE climate.measurement ( id bigserial NOT NULL,