Retry policy within ITargetBlock

Question

I need to introduce a retry policy to the workflow. Let\'s say there are 3 blocks that are connected in such a way:

var executionOptions = new ExecutionDataf         


        

Answer 1

I think you pretty much have to do that, you have to track the remaining number of retries for a message and you have to schedule the retried attempt somehow.

But you could make this better by encapsulating it in a separate method. Something like:

// it's a private class, so public fields are okay
private class RetryingMessage
{
    public T Data;
    public int RetriesRemaining;
    public readonly List Exceptions = new List();
}

public static IPropagatorBlock
    CreateRetryingBlock(
    Func> transform, int numberOfRetries,
    TimeSpan retryDelay, Action> failureHandler)
{
    var source = new TransformBlock>(
        input => new RetryingMessage
        { Data = input, RetriesRemaining = numberOfRetries });

    // TransformManyBlock, so that we can propagate zero results on failure
    TransformManyBlock, TOutput> target = null;
    target = new TransformManyBlock, TOutput>(
        async message =>
        {
            try
            {
                return new[] { await transform(message.Data) };
            }
            catch (Exception ex)
            {
                message.Exceptions.Add(ex);
                if (message.RetriesRemaining == 0)
                {
                    failureHandler(message.Exceptions);
                }
                else
                {
                    message.RetriesRemaining--;
                    Task.Delay(retryDelay)
                        .ContinueWith(_ => target.Post(message));
                }
                return null;
            }
        });

    source.LinkTo(
        target, new DataflowLinkOptions { PropagateCompletion = true });

    return DataflowBlock.Encapsulate(source, target);
}

I have added code to track the exceptions, because I think that failures should not be ignored, they should be at the very least logged.

Also, this code doesn't work very well with completion: if there are retries waiting for their delay and you Complete() the block, it will immediately complete and the retries will be lost. If that's a problem for you, you will have to track outstanding reties and complete target when source completes and no retries are waiting.