Immutable Dictionary Vs Dictionary Vs C5 Vs F# - performance

Question

Our application uses plenty of dictionaries which have multi level lookup that are not frequently changing. We are investigating at converting some of the critical code that

Answer 1

Your presumption that immutable dictionaries allow faster lookup is wrong, because the way almost all immutable collections manage to avoid copying the whole structure on 'modification' is by storing data in a tree and only copying some of the nodes on 'modification', sharing all other nodes. And tree access will generally be slower than accessing a flat array by index, as the mutable cousins do.

I compared the single-thread read performance of Dictionary<,>, ConcurrentDictionary<,>, and ImmutableDictionary<,> based on your code.

The average results of 30 runs, after warmup:

To get a feel for write performance, I also ran a test which adds 50 more entries to the dictionaries. Again, the average results of 30 runs, after warmup:

Tested on

• .net 4.5.1
• Microsoft Bcl Immutable 1.0.34.0

N.B. It should be noted that immutable dictionaries are so much faster and/or allow for higher levels of concurrency in many real life multi-threaded applications that otherwise would have to resort to slow or error-prone techniques like defensive copying, locking and the likes, in order to cope with mutability in the face of threads. This is especially true if you need snapshot-abilty such as for optimistic concurrency, MVCC.

Btw, if you run your sample code as is, the value for at least the immutable dictionary will be highly untypical in a normal (longer running) application, because for some reason the immutable dictionary apparently needs time to warm up. The difference in performance is enormous. Just have a look at the output of the first 3 runs:

Items    Dict   Conc   Immu
===========================
   100   1.90   1.00 361.81
  1000   1.07   1.00   4.33
 10000   1.24   1.00   1.74
100000   1.00   1.33   2.71
---------------------------
   100   1.06   1.00   2.56
  1000   1.03   1.00   4.34
 10000   1.00   1.06   3.54
100000   1.00   1.17   2.76
---------------------------
   100   1.06   1.00   2.50
  1000   1.66   1.00   4.16
 10000   1.00   1.02   3.67
100000   1.00   1.26   3.13

Your question was about read performance (of frozen dictionaries), but the tree characteristics of immutable collections show similarly in the write performance:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                       Mutable (amortized)  Mutable (worst)  Immutable 
───────────────────────────────────────────────────────────────────────
 Stack.Push            O(1)                 O(n)             O(1)      
 Queue.Enqueue         O(1)                 O(n)             O(1)      
 List.Add              O(1)                 O(n)             O(log n)  
 HashSet.Add           O(1)                 O(n)             O(log n)  
 SortedSet.Add         O(log n)             O(n)             O(log n)  
 Dictionary.Add        O(1)                 O(n)             O(log n)  
 SortedDictionary.Add  O(log n)             O(n log n)       O(log n)  
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━