Entity Framework Core count does not have optimal performance

Question

I need to get the amount of records with a certain filter.

Theoretically this instruction:

_dbContext.People.Count (w => w.Type == 1);


        

Answer 1

I am using EFCore 1.1 here.

This can occur if EFCore cannot translate the entire Where clause to SQL. This can be something as simple as DateTime.Now that might not even think about.

The following statement results in a SQL query that will surprisingly run a SELECT * and then C# .Count() once it has loaded the entire table!

   int sentCount = ctx.ScheduledEmail.Where(x => x.template == template &&
                   x.SendConfirmedDate > DateTime.Now.AddDays(-7)).Count();

But this query will run an SQL SELECT COUNT(*) as you would expect / hope for:

   DateTime earliestDate = DateTime.Now.AddDays(-7);
   int sentCount = ctx.ScheduledEmail.Where(x => x.template == template 
                   && x.SendConfirmedDate > earliestDate).Count();

Crazy but true. Fortunately this also works:

   DateTime now = DateTime.Now;
   int sentCount = ctx.ScheduledEmail.Where(x => x.template == template &&
                   x.SendConfirmedDate > now.AddDays(-7)).Count();

Answer 2

Does this get what you want:

_dbContext.People.Where(w => w.Type == 1).Count();

Answer 3

sorry for the bump, but...

probably the reason the query with the where clause is slow is because you didnt provide your database a fast way to execute it.

in case of the select count(*) from People query we dont need to know the actual data for each field and we can just use a small index that doesnt have all these fields in them so we havent got to spend our slow I/O on. The database software would be clever enough to see that the primary key index requires the least I/O to do the count on. The pk id's require less space than the full row so you get more back to count per I/O block so you can complete faster.

Now in the case of the query with the Type it needs to read the Type to determine it's value. You should create an index on Type if you want your query to be fast or else it will have to do a very slow full table scan, reading all rows. It helps when your values are more discriminating. A column Gender (usually) only has two values and isnt very discriminating, a primary key column where every value is unique is highly dscriminating. Higher discriminating values will result in a shorter index range scan and a faster result to the count.

Answer 4

If you want to optimize performance and the current EF provider is not not (yet) capable of producing the desired query, you can always rely on raw SQL.

Obviously, this is a trade-off as you are using EF to avoid writing SQL directly, but using raw SQL can be useful if the query you want to perform can't be expressed using LINQ, or if using a LINQ query is resulting in inefficient SQL being sent to the database.

A sample raw SQL query would look like this:

var results = _context.People.FromSql("SELECT Id, Name, Type, " +
                                      "FROM People " +
                                      "WHERE Type = @p0",                                                     
                                      1);

As far as I know, raw SQL queries passed to the FromSql extension method currently require that you return a model type, i.e. returning a scalar result may not yet be supported.

You can however always go back to plain ADO.NET queries:

using (var connection = _context.Database.GetDbConnection())
{
    connection.Open();

    using (var command = connection.CreateCommand())
    {
        command.CommandText = "SELECT COUNT(*) FROM People WHERE Type = 1";
        var result = command.ExecuteScalar().ToString();
    }
}

Answer 5

Try this

(from x in _dbContext.People where x.Type == 1 select x).Count();

or you could do the async version of it like:

await (from x in _dbContext.People where x.Type == 1 select x).CountAsync();

and if those don't work out for you, then you could at least make the query more efficient by doing:

(from x in _dbContext.People where x.Type == 1 select x.Id).Count();

or

await (from x in _dbContext.People where x.Type == 1 select x.Id).CountAsync();

Answer 6

It seems that there has been some problem with one of the early releases of Entity Framework Core. Unfortunately you have not specified exact version so I am not able to dig into EF source code to tell what exactly has gone wrong.

To test this scenario, I have installed the latest EF Core package and managed to get correct result.

Here is my test program:

And here is SQL what gets generated captured by SQL Server Profiler:

As you can see it matches all the expectations.

Here is the excerpt from packages.config file:

...
<package id="Microsoft.EntityFrameworkCore" version="1.1.0" targetFramework="net452" />
...

So, in your situation the only solution is to update to the latest package which is 1.1.0 at the time of writing this.