RX - Multi consumers by key subscription complexity

问题

I got an observable of KeyValuePair<int,double>:

--(7|45.2)--(3|11.1)--(5|13.2)--(6|36.2)--(3|57.4)

I got a list of consumers defined at runtime. They are only interested in values produced for a single key (myKey).

For example:

the consumer 7, is only interested in the value 45.2.
the consumer 3, is only interested in the values 11.1 and 57.4
the consumer 1, is only interested in values with myKey = 1, so none here.

Here is my consumer subscription code (one per consumer):

myObservable.Where(t => t.Key == myKey).Subscribe(t => /* DoSomething*/);

Let's take:

N = number of messages being produced by myObservable
M = number of consumers

Let's call Comparison the code t.Key == myKey

For every new message being published, Comparison will be executed M times (once per consumer). In the case of N messages, Comparison will be executed N * M

Is RX Extension offering another way to do to avoid executing that many comparisons?

Do I need to make it myself? (using the dictionary of (mykey, consumers) for example, and forwarding the messages to the right consumer(s))

Thanks

回答1:

You could create a class that stores internally the grouped sequence, and is able to provide a subsequence before the associated key of the subsequence has been emitted. This would allow you to subscribe to all subsequences you are interested to observe, before even connecting to the source. One way to do it is to use a dictionary of Subject<T> as propagators, like in the implementation below:

public class LookupObservable<TSource, TKey>
{
    private readonly ConcurrentDictionary<TKey, Subject<TSource>> _subjects;

    public LookupObservable(IObservable<TSource> source,
        Func<TSource, TKey> keySelector,
        IEqualityComparer<TKey> keyComparer = null)
    {
        keyComparer ??= EqualityComparer<TKey>.Default;
        _subjects = new ConcurrentDictionary<TKey, Subject<TSource>>(keyComparer);

        source
            .GroupBy(keySelector, keyComparer)
            .Subscribe
            (
                group => group.Subscribe(GetOrAddSubject(group.Key)),
                ex => Array.ForEach(_subjects.Values.ToArray(), s => s.OnError(ex)),
                () => Array.ForEach(_subjects.Values.ToArray(), s => s.OnCompleted())
            );
    }

    public IObservable<TSource> this[TKey key] => GetOrAddSubject(key);

    private Subject<TSource> GetOrAddSubject(TKey key)
        => _subjects.GetOrAdd(key, _ => new Subject<TSource>());
}

You could use the class LookupObservable like this:

IObservable<SomeType> source = GetSourceStream();

var published = source.Publish(); // Make sure not to warm it too early

var lookup = new LookupObservable<SomeType, int>(published, x => x.Key);

var consumer7 = lookup[7]; consumer7.Subscribe(x => Console.WriteLine(x));
var consumer3 = lookup[3]; consumer3.Subscribe(x => Console.WriteLine(x));
var consumer1 = lookup[1]; consumer1.Subscribe(x => Console.WriteLine(x));

published.Connect(); // Now that all subscriptions are in place, it's time to warm it

await published; // Wait for the completion of the source sequence

The internal dictionary should ensure that subscribing a consumer and propagating a message will be performed with O(1) complexity. I haven't tested it, but this implementation should be able to handle thousands of consumers and millions of messages, with negligible overhead.

回答2:

You could give this a go:

var subject = new Subject<KeyValuePair<int, double>>();

var interests = new Dictionary<int, Func<double, string>>()
{
    { 7, v => $"Seven:{v}" },
    { 3, v => $"Three:{v}" },
    { 1, v => $"One:{v}" },
};

IDisposable subscription =
    subject
        .GroupBy(x => x.Key, x => x.Value)
        .Select(gx =>
            interests.ContainsKey(gx.Key)
            ? gx.Select(x => interests[gx.Key](x))
            : Observable.Empty<string>())
        .Merge()
        .Subscribe(x => Console.WriteLine(x));

When I test with this code:

subject.OnNext(new KeyValuePair<int, double>(7, 45.2));
subject.OnNext(new KeyValuePair<int, double>(3, 11.1));
subject.OnNext(new KeyValuePair<int, double>(5, 13.2));
subject.OnNext(new KeyValuePair<int, double>(6, 36.2));
subject.OnNext(new KeyValuePair<int, double>(3, 57.4));

I get this output:

Seven:45.2
Three:11.1
Three:57.4

The comparison is done once per key.

来源：https://stackoverflow.com/questions/52958608/rx-multi-consumers-by-key-subscription-complexity

标签

system.reactive