OutOfMemoryException with gcAllowVeryLargeObjects

Question

I'm using a BinarySerializer with a pretty big (althought not very deep) graph of items. I have 8GB of ram backed by 12Gig of swap and i'm getting an OutOfMemoryException when serializing which is expected ( it's possible the graph could go near or over 2Gb).

However when i use gcAllowVeryLargeObjects it's no better, i still get the same exception and i'm definately working on something that should hold in memory (at least with the swap).

Is there anything i could do to support serializing this / a way to get the same feature set but getting the result in chuncks maybe?

There's nothing special about my serialization code :

    public static byte[] Serialize(this object o)
    {
        var ms = new MemoryStream();
        var bf = new BinaryFormatter();
        bf.Serialize(ms, o);
        ms.Position = 0;
        return ms.ToArray();
    }

The object i'm serializing contains arrays of items that themselves contains array etc, but the full graph itself isn't "that" large (it's the result of indexing data that, at the source, is already only around 1GB in size).

It's not due to GC fragmentation either (compacting the large heap didn't help).

Answer 1

By default AnyCPU runs as 32 bit process on both x86 and x64 OS. So even with gcAllowVeryLargeObjects set on x64 OS you run into 4GB limit of address space (2GB on x86).

To change uncheck "prefer 32 bit" property on solution properties -> "build" tab.

The details and history can be found in following answer: What is the purpose of the "Prefer 32-bit" setting in Visual Studio 2012 and how does it actually work?