Have multiple calls wait on the same internal async task

Question

(Note: this is an over-simplified scenario to demonstrate my coding issue.)

I have the following class interface:

public class CustomerService
{
    Task<IEnumerable<Customer>> FindCustomersInArea(String areaName);
    Task<Customer> GetCustomerByName(String name);
    :
}

This is the client-side of a RESTful API which loads a list of Customer objects from the server then exposes methods that allows client code to consume and work against that list.

Both of these methods work against the internal list of Customers retrieved from the server as follows:

private Task<IEnumerable<Customer>> LoadCustomersAsync()
{
    var tcs = new TaskCompletionSource<IEnumerable<Customer>>();

    try
    {
        // GetAsync returns Task<HttpResponseMessage>
        Client.GetAsync(uri).ContinueWith(task =>
        {
            if (task.IsCanceled)
            {
                tcs.SetCanceled();
            }
            else if (task.IsFaulted)
            {
                tcs.SetException(task.Exception);
            }
            else
            {
                // Convert HttpResponseMessage to desired return type
                var response = task.Result;

                var list = response.Content.ReadAs<IEnumerable<Customer>>();

                tcs.SetResult(list);
            }
        });
    }
    catch (Exception ex)
    {
        tcs.SetException(ex);
    }
}

The Client class is a custom version of the HttpClient class from the WCF Web API (now ASP.NET Web API) because I am working in Silverlight and they don't have an SL version of their client assemblies.

After all that background, here's my problem:

All of the methods in the CustomerService class use the list returned by the asynchronous LoadCustomersAsync method; therefore, any calls to these methods should wait (asynchronously) until the LoadCustomers method has returned and the appopriate logic executed on the returned list.

I also only want one call made from the client (in LoadCustomers) at a time. So, I need all of the calls to the public methods to wait on the same internal task.

To review, here's what I need to figure out how to accomplish:

1. Any call to FindCustomersInArea and GetCustomerByName should return a Task that waits for the LoadCustomersAsync method to complete. If LoadCustomersAsync has already returned (and the cached list still valid), then the method may continue immediately.

2. After LoadCustomersAsync returns, each method has additional logic required to convert the list into the desired return value for the method.

3. There must only ever be one active call to LoadCustomersAsync (of the GetAsync method within).

4. If the cached list expires, then subsequent calls will trigger a reload (via LoadCustomersAsync).

Let me know if you need further clarification, but I'm hoping this is a common enough use case that someone can help me work out the logic to get the client working as desired.

Answer 1

Disclaimer: I'm going to assume you're using a singleton instance of your HttpClient subclass. If that's not the case we need only modify slightly what I'm about to tell you.

---

Yes, this is totally doable. The mechanism we're going to rely on for subsequent calls to LoadCustomersAsync is that if you attach a continuation to a Task, even if that Task completed eons ago, you're continuation will be signaled "immediately" with the task's final state.

Instead of creating/returning a new TaskCompletionSource<T> (TCS) every time from the LoadCustomerAsync method, you would instead have a field on the class that represents the TCS. This will allow your instance to remember the TCS that last represented the call that represented a cache-miss. This TCS's state will be signaled exactly the same as your existing code. You'll add the knowledge of whether or not the data has expired as another field which, combined with whether the TCS is currently null or not, will be the trigger for whether or not you actually go out and load the data again.

Ok, enough talk, it'll probably make a lot more sense if you see it.

The Code

public class CustomerService 
{ 
    // Your cache timeout (using 15mins as example, can load from config or wherever)
    private static readonly TimeSpan CustomersCacheTimeout = new TimeSpan(0, 15, 0);

    // A lock object used to provide thread safety
    private object loadCustomersLock = new object();
    private TaskCompletionSource<IEnumerable<Customer>> loadCustomersTaskCompletionSource;
    private DateTime loadCustomersLastCacheTime = DateTime.MinValue;

    private Task<IEnumerable<Customer>> LoadCustomersAsync()
    {
        lock(this.loadCustomersLock)
        {
            bool needToLoadCustomers = this.loadCustomersTaskCompletionSource == null
                                             ||
                                       (this.loadCustomersTaskCompletionSource.Task.IsFaulted || this.loadCustomersTaskCompletionSource.Task.IsCanceled)
                                             ||
                                       DateTime.Now - this.loadCustomersLastCacheTime.Value > CustomersService.CustomersCacheTimeout;

            if(needToLoadCustomers)
            {
                this.loadCustomersTaskCompletionSource = new TaskCompletionSource<IEnumerable<Customer>>();

                try
                {
                     // GetAsync returns Task<HttpResponseMessage>
                     Client.GetAsync(uri).ContinueWith(antecedent =>
                     {
                        if(antecedent.IsCanceled)
                        {
                            this.loadCustomersTaskCompletionSource.SetCanceled();
                        }
                        else if(antecedent.IsFaulted)
                        {
                            this.loadCustomersTaskCompletionSource.SetException(antecedent.Exception);
                        }
                        else
                        {
                            // Convert HttpResponseMessage to desired return type
                            var response = antecedent.Result;

                            var list = response.Content.ReadAs<IEnumerable<Customer>>();

                            this.loadCustomersTaskCompletionSource.SetResult(list);

                            // Record the last cache time
                            this.loadCustomersLastCacheTime = DateTime.Now;
                        }
                    });
                }
                catch(Exception ex)
                {
                    this.loadCustomersTaskCompletionSource.SetException(ex);
                }
            }
        }
    }

    return this.loadCustomersTaskCompletionSource.Task; 
}

Scenarios where the customers aren't loaded:

1. If it's the first call, the TCS will be null so the TCS will be created and customers fetched.
2. If the previous call faulted or was canceled, a new TCS will be created and the customers fetched.
3. If the cache timeout has expired, a new TCS will be created and the customers fetched.

Scenarios where the customers are loading/loaded:

1. If the customers are in the process of loading, the existing TCS's Task will be returned and any continuations added to the task using ContinueWith will be executed once the TCS has been signaled.
2. If the customers are already loaded, the existing TCS's Task will be returned and any continuations added to the task using ContinueWith will be executed as soon as the scheduler sees fit.

NOTE: I used a coarse grained locking approach here and you could theoretically improve performance with a reader/writer implementation, but it would probably be a micro-optimization in your case.

Answer 2

I think you should change the way you call Client.GetAsync(uri). Do it roughly like this:

Lazy<Task> getAsyncLazy = new Lazy<Task>(() => Client.GetAsync(uri));

And in your LoadCustomersAsync method you write:

getAsyncLazy.Value.ContinueWith(task => ...

This will ensure that GetAsync only gets called once and that everyone interested in its result will receive the same task.