Deserialize unknown type with protobuf-net

Question

I have 2 networked apps that should send serialized protobuf-net messages to each other. I can serialize the objects and send them, however, I cannot figure out how to deserialize the received bytes.

I tried to deserialize with this and it failed with a NullReferenceException.

// Where "ms" is a memorystream containing the serialized
// byte array from the network.
Messages.BaseMessage message =
  ProtoBuf.Serializer.Deserialize<Messages.BaseMessage>(ms);

I am passing a header before the serialized bytes that contains message type ID, which I can use in a giant switch statement to return the expected sublcass Type. With the block below, I receive the error: System.Reflection.TargetInvocationException ---> System.NullReferenceException.

//Where "ms" is a memorystream and "messageType" is a
//Uint16.
Type t = Messages.Helper.GetMessageType(messageType);
System.Reflection.MethodInfo method =
  typeof(ProtoBuf.Serializer).GetMethod("Deserialize").MakeGenericMethod(t);
message = method.Invoke(null, new object[] { ms }) as Messages.BaseMessage;

Here's the function I use to send a message over the network:

internal void Send(Messages.BaseMessage message){
  using (System.IO.MemoryStream ms = new System.IO.MemoryStream()){
    ProtoBuf.Serializer.Serialize(ms, message);
    byte[] messageTypeAndLength = new byte[4];
    Buffer.BlockCopy(BitConverter.GetBytes(message.messageType), 0, messageTypeAndLength, 0, 2);
    Buffer.BlockCopy(BitConverter.GetBytes((UInt16)ms.Length), 0, messageTypeAndLength, 2, 2);
    this.networkStream.Write(messageTypeAndLength);
    this.networkStream.Write(ms.ToArray());
  }
}

This the class, with base class, I'm serializing:

[Serializable,
ProtoContract,
ProtoInclude(50, typeof(BeginRequest))]
abstract internal class BaseMessage
{
  [ProtoMember(1)]
  abstract public UInt16 messageType { get; }
}

[Serializable,
ProtoContract]
internal class BeginRequest : BaseMessage
{
    [ProtoMember(1)]
    public override UInt16 messageType
    {
        get { return 1; }
    }
}

---

Fixed using Marc Gravell's suggestion. I removed the ProtoMember attribute from the readonly properties. Also switched to using SerializeWithLengthPrefix. Here's what I have now:

[Serializable,
ProtoContract,
ProtoInclude(50, typeof(BeginRequest))]
abstract internal class BaseMessage
{
  abstract public UInt16 messageType { get; }
}

[Serializable,
ProtoContract]
internal class BeginRequest : BaseMessage
{
    public override UInt16 messageType
    {
        get { return 1; }
    }
}

To receive an object:

//where "this.Ssl" is an SslStream.
BaseMessage message =
  ProtoBuf.Serializer.DeserializeWithLengthPrefix<BaseMessage>(
    this.Ssl, ProtoBuf.PrefixStyle.Base128);

To send an object:

//where "this.Ssl" is an SslStream and "message" can be anything that
// inherits from BaseMessage.
ProtoBuf.Serializer.SerializeWithLengthPrefix<BaseMessage>(
  this.Ssl, message, ProtoBuf.PrefixStyle.Base128);

Answer 1

First; for network usage, there is SerializeWithLengthPrefix and DeserializeWithLengthPrefix which handle length for you (optionally with a tag). The MakeGenericMethod looks OK at first glance; and this actually ties in very closely to the pending commit of the work I've been doing to implement an RPC stack: the pending code has an override of DeserializeWithLengthPrefix that takes (essentially) a Func<int,Type>, to resolve a tag to a type to make it easier to deserialize unexpected data on the fly.

If the message type actually relates to the inheritance between BaseMessage and BeginRequest, then you don't need this; it always goes to the top-most contract type in the hierarchy and works its way down (due to some wire details).

Also - I haven't had chance to test it, but the following might be upsetting it:

[ProtoMember(1)]
public override UInt16 messageType
{
    get { return 1; }
}

It is marked for serialization, but has no mechanism for setting the value. Maybe this is the issue? Try removing the [ProtoMember] here, since I don't this is useful - it is (as far as serialization is concerned), largely a duplicate of the [ProtoInclude(...)] marker.

Answer 2

Serializer.NonGeneric.Deserialize(Type, Stream); //Thanks,  Marc.

or

RuntimeTypeModel.Default.Deserialize(Stream, null, Type); 

Answer 3

Another way to handle this is to use protobuf-net for the "heavy lifting", but to use your own message header. The problem with processing network messages is that they can be broken across boundaries. This typically requires using a buffer to accumulate reads. If you use your own header, you can be sure that the message is there in its entirety before handing it off to protobuf-net.

As an example:

To send

using (System.IO.MemoryStream ms = new System.IO.MemoryStream())
{
    MyMessage message = new MyMessage();
    ProtoBuf.Serializer.Serialize<BaseMessage>(ms, message);
    byte[] buffer = ms.ToArray();

    int messageType = (int)MessageType.MyMessage;
    _socket.Send(BitConverter.GetBytes(messageType));
    _socket.Send(BitConverter.GetBytes(buffer.Length));
    _socket.Send(buffer);
}

To receive

protected bool EvaluateBuffer(byte[] buffer, int length)
{
    if (length < 8)
    {
        return false;
    }

    MessageType messageType = (MessageType)BitConverter.ToInt32(buffer, 0);
    int size = BitConverter.ToInt32(buffer, 4);
    if (length < size + 8)
    {
        return false;
    }

    using (MemoryStream memoryStream = new MemoryStream(buffer))
    {
        memoryStream.Seek(8, SeekOrigin.Begin);
        if (messageType == MessageType.MyMessage)
        {
            MyMessage message = 
                ProtoBuf.Serializer.Deserialize<MyMessage>(memoryStream);
        }
    }
}

The latter method would be "tried" on an accumulator buffer until there was enough data. Once the size requirement is met, the message can be deserialized.