What is the most optimal method of querying a reliable dictionary collection

Question

I would like to use service fabric reliable dictionaries to store data that we intend to query via a rest interface. I was wondering what was the most optimal approach to querying data held in those collections.

The documentation doesn't seem to provide any methods to support querying a IReliableDictionary interface apart from simply enumerating over the collection.

Is the only option to fetch every object out of the reliable dictionary and pop into a List<> collection for querying?

Thanks

Answer 1

As Vaclav mentions in his answer, you can always enumerate the entire dictionary and filter as you wish. Additionally, take a look at the docs for IReliableDictionary, specifically CreateEnumerableAsync(ITransaction, Func<TKey, Boolean>, EnumerationMode). This allows you to filter the keys you wish to include in the enumeration up-front, which may improve perf (e.g. by avoiding paging unnecessary values back in from disk).

Simple Example (exception handling/retry/etc. omitted for brevity):

var myDictionary = await this.StateManager.GetOrAddAsync<IReliableDictionary<int, string>>(new Uri("fabric:/myDictionary"));

using (var tx = this.StateManager.CreateTransaction())
{
    // Fetch all key-value pairs where the key an integer less than 10
    var enumerable = await myDictionary.CreateEnumerableAsync(tx, key => key < 10, EnumerationMode.Ordered); 
    var asyncEnumerator = enumerable.GetAsyncEnumerator();

    while (await asyncEnumerator.MoveNextAsync(cancellationToken))
    {
        // Process asyncEnumerator.Current.Key and asyncEnumerator.Current.Value as you wish
    }
}

You could create canned or dynamic key filters corresponding to your REST queries which search based on key - for value-based queries you will need to fall back to full enumeration (e.g. by using the same pattern as above, but omitting the key filter) or use notifications in order to build your own secondary index.

Answer 2

IReliableDictionary implements IEnumerable, which means you can query it with LINQ expressions, same way you would with an IDictionary.

Note that an enumeration over IReliableDictionary uses snapshot isolation, so it is lock free. Basically this means when you start the enumeration you are enumerating over the dictionary as it is at the time you started. If items are added or removed while you're enumerating, those changes won't appear in that enumeration. More info about this and Reliable Collections in general here: https://azure.microsoft.com/en-us/documentation/articles/service-fabric-reliable-services-reliable-collections/

Answer 3

Look here What would be the best way to search an IReliableDictionary?

There is two way to do it

1. Eli Arbel has async extension methods on Gist that provide a bridge to Ix-Async and also async implementations of Select, SelectMany, and Where.
2. You can wrap IAsyncEnumerable in a regular IEnumerable that simply wraps the asynchronous