Large Scale Additions and Deletions Many-to-Many Relationship in Entity Framework

问题

I am currently working on a project that has recently acquired many database records. I now am trying to optimize for this. Here are my models:

public class QueueEntity
{
    public int Id { get; set; }
    public string Name { get; set; }
    public virtual List<PhraseEntity> Phrases { get; set; }
}

public class PhraseEntity
{
    public int Id { get; set; }
    public string Name { get; set; }
    public virtual SourceEntity Source { get; set; }
    public virtual List<QueueEntity> Queues { get; set; }
}

public class SourceEntity 
{
    public int Id { get; set; }
    public string Name { get; set; }
    public virtual List<PhraseEntity> Phrases { get; set; }
}

I am now testing with about 100,000 phrase records, and say each 20,000 increment of these phrases belongs to a different source.

Users have the ability to add and delete 'Sources' from a Queue; essentially, I just add or delete all the phrases from the source to the queue. This structure was not my choice, and it is too late for it to be changed.

These saves and deletions are now causing my test system to crash, as I can't seem to access the Phrases on a Queue without pulling them into memory. Even when attempting a batch delete (e.g. context.Queues.Single(m => m.Id == queueId).Phrases.Take(1000); or getting an IQueryable of 1000 phrases stored as variable 'phrases' and performing context.Queues.Single(m => m.Id == queueId).Phrases.RemoveAll(m => phrases.Contains(m));) it seems that I am attempting to pull all phrase records on that queue into memory.

Is there a simpler way to remove the relationship between these records while still using a LINQ expression?

回答1:

In case anyone else stumbles across this question with the same issue, we decided against using the ORM for this situation. We just wrote two SQL stored procedures (one for additions and one for deletions), and this simplified the problem and increased efficiency immensely. When testing with 100,000+ records, it would take a ridiculous amount of time to complete the actions if it didn't throw an out of memory exception. With the stored procedures, our operations were completed in a couple seconds or less. The only downfalls to this approach come from a testability standpoint.

来源：https://stackoverflow.com/questions/39235943/large-scale-additions-and-deletions-many-to-many-relationship-in-entity-framewor

标签

mysql

entity-framework

linq

many-to-many