Memory usage serializing chunked byte arrays with Protobuf-net

Question

In our application we have some data structures which amongst other things contain a chunked list of bytes (currently exposed as a List). We chun

Answer 1

Additional re your edit; the [ProtoInclude(..., DataFormat=...)] looks like it simply wasn't being processed. I have added a test for this in my current local build, and it now passes:

[Test]
public void Execute()
{

    var a = new A();
    var b = new B();
    a.B = b;

    b.Data = new List
    {
        Enumerable.Range(0, 1999).Select(v => (byte)v).ToArray(),
        Enumerable.Range(2000, 3999).Select(v => (byte)v).ToArray(),
    };

    var stream = new MemoryStream();
    var model = TypeModel.Create();
    model.AutoCompile = false;
#if DEBUG // this is only available in debug builds; if set, an exception is
  // thrown if the stream tries to buffer
    model.ForwardsOnly = true;
#endif
    CheckClone(model, a);
    model.CompileInPlace();
    CheckClone(model, a);
    CheckClone(model.Compile(), a);
}
void CheckClone(TypeModel model, A original)
{
    int sum = original.B.Data.Sum(x => x.Sum(b => (int)b));
    var clone = (A)model.DeepClone(original);
    Assert.IsInstanceOfType(typeof(A), clone);
    Assert.IsInstanceOfType(typeof(B), clone.B);
    Assert.AreEqual(sum, clone.B.Data.Sum(x => x.Sum(b => (int)b)));
}

This commit is tied into some other, unrelated refactorings (some rework for WinRT / IKVM), but should be committed ASAP.