How to analyse contents of binary serialization stream?

Question

I\'m using binary serialization (BinaryFormatter) as a temporary mechanism to store state information in a file for a relatively complex (game) object structure; the files a

Answer 1

Our application operates massive data. It can take up to 1-2 GB of RAM, like your game. We met same "storing multiple copies of the same objects" problem. Also binary serialization stores too much meta data. When it was first implemented the serialized file took about 1-2 GB. Nowadays I managed to decrease the value - 50-100 MB. What did we do.

The short answer - do not use the .Net binary serialization, create your own binary serialization mechanism. We have own BinaryFormatter class, and ISerializable interface (with two methods Serialize, Deserialize).

Same object should not be serialized more than once. We save it's unique ID and restore the object from cache.

I can share some code if you ask.

EDIT: It seems you are correct. See the following code - it proves I was wrong.

[Serializable]
public class Item
{
    public string Data { get; set; }
}

[Serializable]
public class ItemHolder
{
    public Item Item1 { get; set; }

    public Item Item2 { get; set; }
}

public class Program
{
    public static void Main(params string[] args)
    {
        {
            Item item0 = new Item() { Data = "0000000000" };
            ItemHolder holderOneInstance = new ItemHolder() { Item1 = item0, Item2 = item0 };

            var fs0 = File.Create("temp-file0.txt");
            var formatter0 = new BinaryFormatter();
            formatter0.Serialize(fs0, holderOneInstance);
            fs0.Close();
            Console.WriteLine("One instance: " + new FileInfo(fs0.Name).Length); // 335
            //File.Delete(fs0.Name);
        }

        {
            Item item1 = new Item() { Data = "1111111111" };
            Item item2 = new Item() { Data = "2222222222" };
            ItemHolder holderTwoInstances = new ItemHolder() { Item1 = item1, Item2 = item2 };

            var fs1 = File.Create("temp-file1.txt");
            var formatter1 = new BinaryFormatter();
            formatter1.Serialize(fs1, holderTwoInstances);
            fs1.Close();
            Console.WriteLine("Two instances: " + new FileInfo(fs1.Name).Length); // 360
            //File.Delete(fs1.Name);
        }
    }
}

Looks like BinaryFormatter uses object.Equals to find same objects.

Have you ever looked inside the generated files? If you open "temp-file0.txt" and "temp-file1.txt" from the code example you'll see it has lots of meta data. That's why I recommended you to create your own serialization mechanism.

Sorry for being cofusing.