Update Asp.net app to Npgsql 3 and removing Preload Reader

Question

I have updated my ASP.NET app from NpgSQL 2.2.5 to 3.0.1. In the breaking changes it's specified that they have removed the Preload Reader support. So I remove it from the string connection. Testing my web app, I got the error "An operation is already in progress." specially in the linq query like this:

 var plugins =
            from p in _pluginRepository.GetPlugins() // this method return this: GetAll().OrderBy(p => p.Created)
            join e in _userPluginRepository.GetByUserId(user.Id).ToList() on p.Id equals e.Plugin.Id into pe
            from e in pe.DefaultIfEmpty()
            select new PluginViewModel
            {
                Active = e != null,
                Name = p.Translations.ToUserLanguage(loggedInUser),
                Key = p.Key,
                PluginId = p.Id,
                SettingId = e == null ? 0 : e.Id,
                ExpireDate = e != null && e.ExpireDate.HasValue ? e.ExpireDate.Value : (DateTime?) null,
                Grants = e == null ? UserPluginGrants.None.GetHashCode().ToString() : e.Grants.GetHashCode().ToString()
            };

To solve this error, I have to append a ToList after the GetPlugins method. Is this the correct behavior to use without Preload Reader? Why?

Answer 1

In Npgsql 2.x, using the Preload Reader made Npgsql pull the entire result set of the query from the database into the memory of your application. This freed the connection and allowed another command to be executed while still traversing the resultset of the first query. In other words, it allowed you to program as if you could execute multiple queries concurrently (known sometimes as MARS), although behind the scenes this was implemented inefficiently.

Adding a ToList() does exactly the same thing - pull everything into client memory, only it happens in your application code instead of in the database driver. So it's definitely an acceptable way to port your application from Npgsql 2.x to 3.x.

Now, if the result set being pulled (in this case GetPlugins) is small, this is a perfectly valid approach. If it's big, however you should look into alternatives. In your example, the join could be sent to the database, making your Linq expression translate into a single SQL query and eliminating the need for multiple queries (ORMs such as Entity Framework can usually do this for you). A more extreme solution would be to use multiple database connections, but that is heavier and also problematic if you're using transactions.

Note that there's an issue open for implementing true MARS in Npgsql (although it isn't likely to be implemented very soon): https://github.com/npgsql/npgsql/issues/462