In c# , how to iterate IEnumerable in multithreading environment

问题

I am in this situation that there is a large dictionary that is randomly updated by one thread at a fairly high frequency, and there is another thread that tries to take a snapshot of the dictionary to save as history. I currently using something like this:

Dictionary<string, object> dict = new Dictionary<string, object>();
var items = dict.Values.ToList();

This works fine for most of the time, except it occasionally throws:

System.InvalidOperationException: Collection was modified; enumeration operation may not execute.

I understand why this happen, but I don't know what can I do to avoid the collection modified error.

What is the best approach to iterate such collection?

I also tried ConcurrentDictionary, but no luck. Why? Is ConcurrentDictionary thread safe only at item level?

回答1:

According to the docs you should be able to use the GetEnumerator() method of ConcurrentDictionary to get a thread-safe iterator.

The enumerator returned from the dictionary is safe to use concurrently with reads and writes to the dictionary, however it does not represent a moment-in-time snapshot of the dictionary. The contents exposed through the enumerator may contain modifications made to the dictionary after GetEnumerator was called.

Since you're dealing with concurrent threads, it's not surprising to have some tradeoffs with consistency, but I would expect this approach to block less than the brute force approach given in other answers. This wouldn't have worked if you tried:

var items = concurrentDict.Items.ToList();

but it's supposed to work for

var items = concurrentDict.GetEnumerator();

or you could simply reference the iterator directly:

foreach(var item in concurrentDict)
{
    valueList.Add(item.Value);
}

回答2:

An ImmutableDictionary might be appropriate for you, as it supports scalable multi-threading and snapshotting as part of its basic feature-set.

// initialize.
ImmutableDictionary<string, int> dict = ImmutableDictionary.Create<string,int>();

// create a new dictionary with "foo" key added.
ImmutableDictionary<string, int> newdict = dict.Add("foo", 0);

// replace dict, thread-safe, with a new dictionary with "bar" added.
// note this is using dict, not newdict, so there is no "foo" in it.
ImmutableInterlocked.TryAdd(ref dict, "bar", 1);

// take a snapshot, thread-safe.
ImmutableDictionary<string,int> snapshot = dict;

The immutable nature means that the dictionary can never change -- you can only add a value by creating a new dictionary. And because of this property, you take a "snapshot" of it by simply keeping a reference around from the point you want to snapshot.

It is optimized under the hood to be efficient, not copying the entire thing for every operation. That said, for other operations it isn't as efficient as ConcurrentDictionary, but it's all a trade-off in what you want. For instance, a ConcurrentDictionary can be concurrently enumerated but it's impossible to enumerate a snapshot of it.

回答3:

You can use a monitor with lock keyword to ensure that only reading or only writing is executing at this moment.

public class SnapshotDictionary<TKey, TValue> : IEnumerable<KeyValuePair<TKey, TValue>>
{
    private readonly Dictionary<TKey, TValue> _dictionary = new Dictionary<TKey, TValue>();
    private readonly object _lock = new object();

    public void Add(TKey key, TValue value)
    {
        lock (_lock)
        {
            _dictionary.Add(key, value);
        }
    }

    // TODO: Other necessary IDictionary methods

    public Dictionary<TKey, TValue> GetSnaphot()
    {
        lock (_lock)
        {
            return new Dictionary<TKey, TValue>(_dictionary);
        }
    }

    public IEnumerator<KeyValuePair<TKey, TValue>> GetEnumerator()
    {
        return GetSnaphot().GetEnumerator();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

GetSnapshot method returns a snapshot of your dictionary.
I have also overriden GetEnumerator so that it creates a snapshot and then returns snapshot's enumerator.

So, this will work because will be executed on a snapshot:

var items = snapshotDictionary.GetSnapshot().Values.ToList();

// or 

foreach (var item in snapshotDictionary)
{
    // ...
}

However, this approach does not allow multithreading writing.

来源：https://stackoverflow.com/questions/44403020/in-c-sharp-how-to-iterate-ienumerable-in-multithreading-environment

标签

multithreading

dictionary

concurrency

ienumerable