.Net Dictionary out of memory exception at around 6,000,000 entries

Question

I am using a Dictionary to store the frequency of colors in an image, where the key is the the color (as an int), and the value is the number of

Answer 1

In the 32 bit runtime, the maximum number of items you can have in a Dictionary is in the neighborhood of 61.7 million. See my old article for more info.

If you're running in 32 bit mode, then your entire application plus whatever bits of ASP.NET and the underlying machinery is required all have to fit within the memory available to your process: normally 2 GB in the 32-bit runtime.

By the way, a really wacky way to solve your problem (but one I wouldn't recommend unless you're really hurting for memory), would be the following (assuming a 24-bit image):

1. Call LockBits to get a pointer to the raw image data
2. Compress the per-scan-line padding by moving the data for each scan line to fill the previous row's padding. You end up with an array of 3-byte values followed by a bunch of empty space (to equal the padding).
3. Sort the image data. That is, sort the 3-byte values. You'd have to write a custom sort, but it wouldn't be too bad.
4. Go sequentially through the array and count the number of unique values.
5. Allocate a 2-dimensional array: int[count,2] to hold the values and their occurrence counts.
6. Go sequentially through the array again to count occurrences of each unique value and populate the counts array.

I wouldn't honestly suggest using this method. Just got a little laugh when I thought of it.