Implement IQueryable wrapper to translate result objects

Question

Update 2013-08-22:

After having a look at the 'Building an IQueryable provider series' (thanks for the link!) I got a bit further. I updated the code accordingly. It is still not fully working though. If I understand the tutorial correctly, the GetEnumerator is invoked in case multiple elements are requested (e.g. by a ToList() call on the queryable, or any aggregation function). So all the GetEnumerator implementation of the wrapper has to do is call an Execute on the provider and pass the queryable's expression. In the other case, if only a single element is requested, Execute is called directly. The queryable's expression also reflects whether it is for a single or multiple elements. Is this correct?

Unfortunately now I get an InvalidOperationException saying 'Sequence contains more than one element' when calling Execute on the source query provider. What does this mean? I just pass the expression without any translation since the same types are involved as mentioned above. The translatation bit with IEnumerable in the code is probably incomplete, but for now I don't even get to that point.

---

I'm trying to implement a simple IQueryable wrapper using a single underlying IQueryable as the data source which calls a translation function for each result object.

I thought this would be relatively trivial since the only thing the wrapper has to do is translating. However I couldn't get my implementation to work.

See below for what I got so far. For some queries it is working but I receive a StackOverflowException InvalidOperationException at some point. I guess this happens due to the cyclic association between my queryable and my query provider. But I do not understand how to implement this correctly.

Here my questions and thoughts on it:

1. Why does the IQueryable have a Provider which in turn returns an IQueryable again? Doesn't this call for endless recursion?

2. Why is it not enough to implement IEnumerator? Why does FirstOrDefault for instance not use the enumerator to get the element? When I debugged the application GetEnumerator() was not called by FirstOrDefault() on my queryable.

3. Since the enumerator is not used in every case, where is the correct point to call the translation function? The Execute-methods of the QueryProvider seemed to be the right place. But do I still need the translation call in the Enumerator for some cases? Update: I know realized that I need to provide my own IEnumerable implementation providing the TranslatingEnumerator and return this enumerable from my Execute method. In order to get the enumerator GetEnumerator calls Execute (see below). The LINQ code requesting the enumerator seems to make sure that the expression actually returns an IEnumerable.

Some side notes on the code:

• The translation source type is named TDatabaseEntity, the translation target type is named TBusinessEntity.

• I'm essentially providing an IQueryable which takes the result objects retrieved from an underlying IQueryable and translates them to the TBusinessEntity type objects.

• I'm aware that the Expression also needs to be translated. However I set this aside since in my actual application I'm using the same types for TBusinessEntity and TDatabaseEntity, so the Expression can be passed straight through.

• The result objects still need to be translated to other instances though, despite being of the same type. Update: My translation layer is working already within my application and also takes care of related entities. It's just the 'implementing an IQueryable wrapper' thing I'm stuck with.

• I'm afraid the whole implementation is incorrect -- the TODOs in the code are just my own notes.

Background: I'm kind of implementing my own detaching of entities received from DbContext within my data access layer to prevent my business layer from getting in touch with the actual entities -- due to some bugs with EF and other requirements I can't directly use EF detached entities.

Thanks for your help!

IQueryable implementation

internal class TranslatingQueryable<TDatabaseEntity, TBusinessEntity> : IQueryable<TBusinessEntity>
{
    private readonly IQueryProvider _provider;
    private readonly IQueryable<TDatabaseEntity> _source;

    internal TranslatingQueryable(TranslatingQueryProvider provider, IQueryable<TDatabaseEntity> source)
    {
        Guard.ThrowIfArgumentNull(provider, "provider");
        Guard.ThrowIfArgumentNull(source, "source");

        _provider = provider;
        _source = source;
    }

    internal TranslatingQueryable(Func<object, object> translateFunc, IQueryable<TDatabaseEntity> databaseQueryable)
        : this(new TranslatingQueryProvider(translateFunc, databaseQueryable.Provider), databaseQueryable)
    {
    }

    public IEnumerator<TBusinessEntity> GetEnumerator()
    {
        return ((IEnumerable<TBusinessEntity>)Provider.Execute(Expression)).GetEnumerator();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return ((IEnumerable)Provider.Execute(Expression)).GetEnumerator();
    }

    public Expression Expression
    {
        get
        {
            return _source.Expression;
        }
    }

    public Type ElementType
    {
        get
        {
            return typeof(TBusinessEntity);
        }
    }

    public IQueryProvider Provider
    {
        get
        {
            return _provider;
        }
    }
}

IQueryProvider implementation

public class TranslatingQueryProvider : IQueryProvider
{
    private readonly Func<object, object> _translateFunc;
    private readonly IQueryProvider _databaseQueryProvider;

    public TranslatingQueryProvider(Func<object, object> translateFunc, IQueryProvider databaseQueryProvider)
    {
        _translateFunc = translateFunc;
        _databaseQueryProvider = databaseQueryProvider;
    }

    public IQueryable CreateQuery(Expression expression)
    {
        var databaseQueryable = _databaseQueryProvider.CreateQuery<object>(expression);

        return new TranslatingQueryable<object, object>(this, databaseQueryable);
    }

    public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
    {
        var databaseQueryable = _databaseQueryProvider.CreateQuery<object>(expression);

        return new TranslatingQueryable<object, TElement>(this, databaseQueryable);
    }

    public object Execute(Expression expression)
    {
        return Execute<object>(expression);
    }

    public TResult Execute<TResult>(Expression expression)
    {
        // TODO This call throws an InvalidOperationException if an enumeration is requested
        var databaseResult = _databaseQueryProvider.Execute<TResult>(expression);

        var databaseEnumerable = databaseResult as IEnumerable;
        if (databaseEnumerable != null)
        {
            if (typeof(TResult).IsAssignableFrom(typeof(IEnumerable)))
            {
                throw new InvalidOperationException();
            }

            return (TResult)(object)new TranslatingEnumerable(databaseEnumerable, _translateFunc);
        }
        else
        {
            return (TResult)_translateFunc(databaseResult);
        }
    }

    private class TranslatingEnumerable : IEnumerable
    {
        private readonly TranslatingEnumerator _enumerator;

        public TranslatingEnumerable(IEnumerable databaseEnumerable, Func<object, object> translateFunc)
        {
            _enumerator = new TranslatingEnumerator(translateFunc, databaseEnumerable.GetEnumerator());
        }

        public IEnumerator GetEnumerator()
        {
            return _enumerator;
        }
    }
}

IEnumerator implementation

internal class TranslatingEnumerator : IEnumerator
{
    private readonly Func<object, object> _translateFunc;
    private readonly IEnumerator _databaseEnumerator;

    internal TranslatingEnumerator(Func<object, object> translateFunc, IEnumerator databaseEnumerator)
    {
        _translateFunc = translateFunc;
        _databaseEnumerator = databaseEnumerator;
    }

    public bool MoveNext()
    {
        return _databaseEnumerator.MoveNext();
    }

    public void Reset()
    {
        _databaseEnumerator.Reset();
    }

    public object Current
    {
        get
        {
            return _translateFunc(_databaseEnumerator.Current);
        }
    }

    object IEnumerator.Current
    {
        get
        {
            return Current;
        }
    }
}

Answer 1

Alright here is my best to answer this

Why does the IQueryable have a Provider which in turn returns an IQueryable again? Doesn't this call for endless recursion? You want to return an IQueryable for this instance

SomeEnumerable.Where(x=>x.Field == something).Select(x=>x.SomeOtherField) Think JQuery if you are familiar with chaining

Why is it not enough to implement IEnumerator? Why does FirstOrDefault for instance not use the enumerator to get the element? When I debugged the application GetEnumerator() was not called by FirstOrDefault() on my queryable.

Since IQueryable has 2 special properties query provider and query expression:

What is the difference between IQueryable<T> and IEnumerable<T>?

Since the enumerator is not used in every case, where is the correct point to call the translation function? The Execute-methods of the QueryProvider seemed to be the right place. But do I still need the translation call in the Enumerator for some cases?

Execute is the correct place, you will have to parse the expression tree because the provider has no idea what to do when you execute.

I have also added this link that helped me enormously when I implemented my own query provider http://blogs.msdn.com/b/mattwar/archive/2008/11/18/linq-links.aspx

What you might be able to get away with is using the same way the author in this post takes the dbReader and converts it to actual objects but instead of the reader take your DBEntity and convert it to your BusinessEntity, but I'm not sure if this is possible. Because everytime you add a linq clause(Select,Where...) it creates a new query of that return type, so if you had an entity of DBEntity "entities" and you did entities.Select(x=>x.someField) and some field is say of type int it's now IQueryable, now your model doesn't work because its expecting int and it's getting DBEntitity

Answer 2

By now I found out why I received an exception every time the query has been enumerated: The IQueryable infrastructure of the Entity Framework is implemented very differently from the pattern described in Building an IQueryable provider series, pt. 1.

• The blog post suggests to implement GetEnumerator() by calling Execute() on the provider.

• In contrast, in the EF infrastructure, ObjectQueryProvider's Execute() method only accepts expressions which return a single result object -- but not an enumerable collection of result objects (this is even documented in the source code). Accordingly, ObjectQuery's GetEnumerator() method does not call Execute() but another method getting the result right from the database.

Thus, any translating IQueryable implementation which uses an underlying database query to get the objects must use the same pattern -- the translating GetEnumerator() method just calls GetEnumerator() on the underlying database query and injects this into a new TranslatingEnumerator.