bulk insert with linq-to-sql

Question

I have a query that looks like this:

using (MyDC TheDC = new MyDC())
{
   foreach (MyObject TheObject in TheListOfMyObjects)
   {
      DBTable TheTable = ne         


        

Answer 1

It will generate a single insert statement for every record, but will send them all to the server in a single batch and run in a single transaction.

That is what the SubmitChanges() outside the loop does.

If you moved it inside, then every iteration through the loop would go off to the server for the INSERT and run in it's own transaction.

I don't believe there is any way to fire off a SQL BULK INSERT.

Answer 2

LINQ Single Insert from List:

                int i = 0;
                foreach (IPAPM_SRVC_NTTN_NODE_MAP item in ipapmList)
                {
                    ++i;
                    if (i % 50 == 0)
                    {
                        ipdb.Dispose();
                        ipdb = null;
                        ipdb = new IPDB();
                        // .NET CORE
                        //ipdb.ChangeTracker.AutoDetectChangesEnabled = false; 
                        ipdb.Configuration.AutoDetectChangesEnabled = false;
                    }

                    ipdb.IPAPM_SRVC_NTTN_NODE_MAP.Add(item);
                    ipdb.SaveChanges();
                }                 

Answer 3

I would suggest you take a look at N.EntityFramework.Extension. It is a basic bulk extension framework for EF 6 that is available on Nuget and the source code is available on Github under MIT license.

Install-Package N.EntityFramework.Extensions

https://www.nuget.org/packages/N.EntityFramework.Extensions

Once you install it you can simply use BulkInsert() method directly on the DbContext instance. It support BulkDelete, BulkInsert, BulkMerge and more.

BulkInsert()

var dbcontext = new MyDbContext();  
var orders = new List<Order>();  
for(int i=0; i<10000; i++)  
{  
   orders.Add(new Order { OrderDate = DateTime.UtcNow, TotalPrice = 2.99 });  
}  
dbcontext.BulkInsert(orders);  

Answer 4

I modified the code from the following link to be more efficient and used it in my application. It is quite convenient because you can just put it in a partial class on top of your current autogenerated class. Instead of InsertOnSubmit add entities to a list, and instead of SubmitChanges call YourDataContext.BulkInsertAll(list).

http://www.codeproject.com/Tips/297582/Using-bulk-insert-with-your-linq-to-sql-datacontex

partial void OnCreated()
{
    CommandTimeout = 5 * 60;
}

public void BulkInsertAll<T>(IEnumerable<T> entities)
{                        
    using( var conn = new SqlConnection(Connection.ConnectionString))
    {
        conn.Open();

        Type t = typeof(T);

        var tableAttribute = (TableAttribute)t.GetCustomAttributes(
            typeof(TableAttribute), false).Single();
        var bulkCopy = new SqlBulkCopy(conn)
        {
            DestinationTableName = tableAttribute.Name
        };

        var properties = t.GetProperties().Where(EventTypeFilter).ToArray();
        var table = new DataTable();

        foreach (var property in properties)
        {
            Type propertyType = property.PropertyType;
            if (propertyType.IsGenericType &&
                propertyType.GetGenericTypeDefinition() == typeof(Nullable<>))
            {
                propertyType = Nullable.GetUnderlyingType(propertyType);
            }

            table.Columns.Add(new DataColumn(property.Name, propertyType));
        }

        foreach (var entity in entities)
        {
            table.Rows.Add(
                properties.Select(
                property => property.GetValue(entity, null) ?? DBNull.Value
                ).ToArray());
        }

        bulkCopy.WriteToServer(table);
    }                                               
}

private bool EventTypeFilter(System.Reflection.PropertyInfo p)
{
    var attribute = Attribute.GetCustomAttribute(p,
        typeof(AssociationAttribute)) as AssociationAttribute;

    if (attribute == null) return true;
    if (attribute.IsForeignKey == false) return true;

    return false;
}

Answer 5

The term Bulk Insert usually refers to the SQL Server specific ultra fast bcp based SqlBulkCopy implementation. It is built on top of IRowsetFastLoad.

Linq-2-SQL does not implement insert using this mechanism, under any conditions.

If you need to bulk load data into SQL Server and need it to be fast, I would recommend hand coding using SqlBulkCopy.

Linq-2-SQL will attempt to perform some optimisations to speed up multiple inserts, however it still will fall short of many micro ORMs (even though no micro ORMs I know of implement SqlBulkCopy)