What's a good memory-conservative serializer to replace BinaryFormatter? [closed]

问题

I'm working with a large application using .Net v3.5 with C# (Visual Studio 2008) that uses the BinaryFormatter to create a data file.

Stream stream = File.Open(filePath, FileMode.Create, FileAccess.Write, FileShare.None);
BinaryFormatter formatter = new BinaryFormatter(null, (new StreamingContext(StreamingContextStates.All, false)));
formatter.Serialize(stream, data);
stream.Flush();
stream.Close();

Unfortunately, I frequently get OutOfMemoryException from this implementation. I'm looking for some sort of alternative to BinaryFormatter that I can quickly transition to.

It's worth noting that this application mostly relies on ISerializable, rather than [Serializable] attributes to retain versioning (of sorts). Additionally, the data we're serializing has multiple variables that points to the same object. Lastly, we also serialize Lists and Dictionaries as well, making the data contain a rather deep hierarchy of ISerializables.

Thus, I would prefer an alternative that takes advantage of ISerializable.GetObjectData capable of handling repeating pointers to the same object.

Edit: in reply to dbc, you ask a very good question. After replicating the issue, the error I get is:

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
    at System.Runtime.Serialization.ObjectIDGenerator.Rehash()
    at System.Runtime.Serialization.ObjectIDGenerator.GetId(Object obj, Boolean& firstTime)
    at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.InternalGetId(Object obj, Boolean assignUniqueIdToValueType, Type type, Boolean& isNew)
    at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteString(NameInfo memberNameInfo, NameInfo typeNameInfo, Object stringObject)
    at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteKnownValueClass(NameInfo memberNameInfo, NameInfo typeNameInfo, Object data)
    at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteMembers(NameInfo memberNameInfo, NameInfo memberTypeNameInfo, Object memberData, WriteObjectInfo objectInfo, NameInfo typeNameInfo, WriteObjectInfo memberObjectInfo)
    at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteMemberSetup(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo, String memberName, Type memberType, Object memberData, WriteObjectInfo memberObjectInfo)
    at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo, String[] memberNames, Type[] memberTypes, Object[] memberData, WriteObjectInfo[] memberObjectInfos)
    at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo)
    at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize(Object graph, Header[] inHeaders, __BinaryWriter serWriter, Boolean fCheck)
    at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph, Header[] headers, Boolean fCheck)
    at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph)

Quick Googling tells me that this is due to serializing an object that is too large. I'm pretty sure I'm not storing any binary information like images or audio, so this strikes me as a little strange. My guess would be that perhaps a list is too large to serialize.

Incidentally, I tried the code below using the same procedure, and they don't produce an exception.

SerializationInfo info = new SerializationInfo(typeof(Data), new FormatterConverter());
StreamingContext context = new StreamingContext(StreamingContextStates.All, false);
data.GetObjectData(info, context);
foreach (SerializationEntry e in info)
{
    Debug.WriteLine("Name: " + e.Name);
    Debug.WriteLine("Type: " + e.ObjectType.ToString());
    Debug.WriteLine("Value: " + e.Value.ToString());
}

回答1:

That particular exception isn't thrown because you are serializing an object that is too large. It is thrown because you are serializing an object graph with so many objects that the ObjectIDGenerator inside BinaryFormatter cannot allocate a large enough hash table to assign unique IDs to each pointer. The serializer uses ObjectIDGenerator to generate a run-time unique ID for each reference class serialized so as to correctly serialize multiple references to the same class instance as a single id-based indirect reference. Any graph serializer you choose will need to do something like this.

Rather than adopting a new graph serializer technology, which will be quite burdensome, is it possible for you to reduce the number of class instances you serialize simultaneously? For instance, can you break your object graph into disconnected segments, and serialize each one sequentially into the stream? (One tutorial on how to do this is here: Serializing lots of different objects into a single file.) Or do you have some classes that contain multiple small leaf classes that are never shared and could be replaced with a single proxy for all of them?

来源：https://stackoverflow.com/questions/25148793/whats-a-good-memory-conservative-serializer-to-replace-binaryformatter

标签

.net

serialization