SQL Server insert performance

Question

I have an insert query that gets generated like this

INSERT INTO InvoiceDetail (LegacyId,InvoiceId,DetailTypeId,Fee,FeeTax,Investigatorid,SalespersonId,Creat         


        

Answer 1

Most likely this is commit flush wait. If you don't wrap sets of INSERTs into explicitly managed transaction then each INSERT is its own auto-committed transaction. Meaning each INSERT issues automatically a commit, and a commit has to wait until the log is durable (ie. written to disk). Flushing the log after each insert is extremely slow.

For instance, trying to insert 100k rows like yours on a single row commit style:

set nocount on; 
declare @start datetime = getutcdate();  

declare @i int = 0;
while @i < 100000
begin
INSERT INTO InvoiceDetail (
  LegacyId,InvoiceId,DetailTypeId,Fee,
  FeeTax,Investigatorid,SalespersonId,
  CreateDate,CreatedById,IsChargeBack,
  Expense,RepoAgentId,PayeeName,ExpensePaymentId,
  AdjustDetailId) 
  VALUES(1,1,2,1500.0000,0.0000,163,1002,
    '11/30/2001 12:00:00 AM',
    1116,0,550.0000,850,NULL,1,NULL); 
  set @i = @i+1;
end

select datediff(ms, @start, getutcdate());

This runs in about 12seconds on my server. But adding transaction management and committing every 1000 rows the insert of 100k rows lasts only about 4s:

set nocount on;  
declare @start datetime = getutcdate();  

declare @i int = 0;
begin transaction
while @i < 100000
begin
INSERT INTO InvoiceDetail (
  LegacyId,InvoiceId,DetailTypeId,
  Fee,FeeTax,Investigatorid,
  SalespersonId,CreateDate,CreatedById,
  IsChargeBack,Expense,RepoAgentId,
  PayeeName,ExpensePaymentId,AdjustDetailId) 
  VALUES(1,1,2,1500.0000,0.0000,163,1002,
    '11/30/2001 12:00:00 AM',
    1116,0,550.0000,850,NULL,1,NULL); 
  set @i = @i+1;
  if (@i%1000 = 0)
  begin
    commit
    begin transaction
  end  
end
commit;
select datediff(ms, @start, getutcdate());

Also given that I can insert 100k rows in 12 seconds even w/o the batch commit, while you need 30 minutes, its worth investigating 1) the speed of your IO subsystem (eg. what Avg. Sec per Transaction you see on the drives) and 2) what else is the client code doing between retrieving the @@identity from one call and invoking the next insert. It could be that the bulk of time is in the client side of the stack. One simple solution would be to launch multiple inserts in parallel (BeginExecuteNonQuery) so you feed the SQL Server inserts constantly.