Generic class for performing mass-parallel queries. Feedback?

Question

I don\'t understand why, but there appears to be no mechanism in the client library for performing many queries in parallel for Windows Azure Table Storage. I\'ve created a

Answer 1

Have you considered using the Task Parallel Library?

http://msdn.microsoft.com/en-us/library/dd537609.aspx

Answer 2

Guess I'm late to the party. I would add two things:

1. ManualResetEvent is IDisposable. So you need to make sure it gets disposed somewhere.
2. Error handling - if one of the queries fails it'll probably fail the whole thing. You should probably retry failed requests. Alternatively you could return the values you did get back with some indication of which queries failed, so that the caller could retry the queries.
3. Client side timeouts - there are none. This isn't a problem if the server side times out for you, but if that ever fails (eg, network issues) the client will hang forever.

Also, I think this is actually a better approach that the Task Parallel Library. I tried the Task-per-query approach before this. The code was actually more awkward, and it tended to result in having a lot of active threads. I still haven't tested extensively with your code, but it seems to work better on first blush.

Update

I've put some work into a more-or-less rewrite of the code above. My rewrite removes all locking, supports client-side timeouts of hung transactions (rare, but it does happen, and can really ruin your day), and some exception handling logic. There is a full solution with tests up on Bitbucket. The most relevant code lives in one file, though it does require some helpers that are in other parts of the project.