Bulk insert is not working properly in Azure SQL Server

Question

I\'m not able to insert the bulk amount of data into Azure SQL server DB using C# webapi

Consider

I want to insert 60K> data in SQL. In my local sql server

Answer 1

When using SqlBulkCopy, depend on the Route-Trip-Time (distance) between your web server and the Azure SQL server, there are two parameters you need to be aware of:

1. BatchSize (for me I set batch size to 500)

2. BulkCopyTimeOut: default is 30 seconds (I set to 60 secs)

   using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connection)) {
           bulkCopy.BatchSize = array.Length;
           bulkCopy.DestinationTableName = tempTableName;
           bulkCopy.BulkCopyTimeout = 60;...}
   

With this configuration, my whole process of getting 20K+ events from EventStore to my Projection MicroService, then the microService Update/Write those 20K events to a Database took approximately 15 minutes (depends on the RTT- route trip time between the Event Store and the MicroService, and between Microservice and the database)