Binary communications protocol parser design for serial data

Question

I'm revisiting a communications protocol parser design for a stream of bytes (serial data, received 1 byte at a time).

The packet structure (can't be changed) is:

|| Start Delimiter (1 byte) | Message ID (1 byte) | Length (1 byte) | Payload (n bytes) | Checksum (1 byte) ||

In the past I have implemented such systems in a procedural state-machine approach. As each byte of data arrives, the state machine is driven to see where/if the incoming data fits into a valid packet a byte at a time, and once a whole packet has been assembled, a switch statement based on the Message ID executes the appropriate handler for the message. In some implementations, the parser/state machine/message handler loop sits in its own thread so as not to burden the serial data received event handler, and is triggered by a semaphore indicating bytes have been read.

I'm wondering if there is a more elegant solution to this common problem, exploiting some of the more modern language features of C# and OO design. Any design patterns that would solve this problem? Event-driven vs polled vs combination?

I'm interested to hear your ideas. Thanks.

Prembo.

Answer 1

First of all I would separate the packet parser from the data stream reader (so that I could write tests without dealing with the stream). Then consider a base class which provides a method to read in a packet and one to write a packet.

Additionally I would build a dictionary (one time only then reuse it for future calls) like the following:

class Program {
    static void Main(string[] args) {
        var assembly = Assembly.GetExecutingAssembly();
        IDictionary<byte, Func<Message>> messages = assembly
            .GetTypes()
            .Where(t => typeof(Message).IsAssignableFrom(t) && !t.IsAbstract)
            .Select(t => new {
                Keys = t.GetCustomAttributes(typeof(AcceptsAttribute), true)
                       .Cast<AcceptsAttribute>().Select(attr => attr.MessageId),
                Value = (Func<Message>)Expression.Lambda(
                        Expression.Convert(Expression.New(t), typeof(Message)))
                        .Compile()
            })
            .SelectMany(o => o.Keys.Select(key => new { Key = key, o.Value }))
            .ToDictionary(o => o.Key, v => v.Value); 
            //will give you a runtime error when created if more 
            //than one class accepts the same message id, <= useful test case?
        var m = messages[5](); // consider a TryGetValue here instead
        m.Accept(new Packet());
        Console.ReadKey();
    }
}

[Accepts(5)]
public class FooMessage : Message {
    public override void Accept(Packet packet) {
        Console.WriteLine("here");
    }
}

//turned off for the moment by not accepting any message ids
public class BarMessage : Message {
    public override void Accept(Packet packet) {
        Console.WriteLine("here2");
    }
}

public class Packet {}

public class AcceptsAttribute : Attribute {
    public AcceptsAttribute(byte messageId) { MessageId = messageId; }

    public byte MessageId { get; private set; }
}

public abstract class Message {
    public abstract void Accept(Packet packet);
    public virtual Packet Create() { return new Packet(); }
}

---

Edit: Some explanations of what is going on here:

First:

[Accepts(5)]

This line is a C# attribute (defined by AcceptsAttribute) says the the FooMessage class accepts the message id of 5.

Second:

Yes the dictionary is being built at runtime via reflection. You need only to do this once (I would put it into a singleton class that you can put a test case on it that can be run to ensure that the dictionary builds correctly).

Third:

var m = messages[5]();

This line gets the following compiled lambda expression out of the dictionary and executes it:

()=>(Message)new FooMessage();

(The cast is necessary in .NET 3.5 but not in 4.0 due to the covariant changes in how delagates work, in 4.0 an object of type Func<FooMessage> can be assigned to an object of the type Func<Message>.)

This lambda expression is built by the Value assignment line during dictionary creation:

Value = (Func<Message>)Expression.Lambda(Expression.Convert(Expression.New(t), typeof(Message))).Compile()

(The cast here is necessary to cast the compiled lambda expression to Func<Message>.)

I did that this way because I happen to already have the type available to me at that point. You could also use:

Value = ()=>(Message)Activator.CreateInstance(t)

But I believe that would be slower (and the cast here is necessary to change Func<object> into Func<Message>).

Fourth:

.SelectMany(o => o.Keys.Select(key => new { Key = key, o.Value }))

This was done because I felt that you might have value in placing the AcceptsAttribute more than once on a class(to accept more than one message id per class). This also has the nice side affect of ignoring message classes that do not have a message id attribute (otherwise the Where method would need to have the complexity of determining if the attribute is present).

Answer 2

I'm a little late to the party but I wrote a framework that I think could do this. Without knowing more about your protocol, it's hard for me to write the object model but I would think it wouldn't be too hard. Take a look at binaryserializer.com.

Answer 3

What I generally do is define an abstract base message class and derive sealed messages from that class. Then have a message parser object that contains the state machine to interpret the bytes and build an appropriate message object. The message parser object just has a method (to pass it the incoming bytes) and optionally an event (invoked when a full message has arrived).

You then have two options for handling the actual messages:

• Define an abstract method on the base message class, overriding it in each of the derived message classes. Have the message parser invoke this method after the message has arrived completely.
• The second option is less object-oriented, but may be easier to work with: leave the message classes as just data. When the message is complete, send it out via an event that takes the abstract base message class as its parameter. Instead of a switch statement, the handler usually as-casts them to the derived types.

Both of those options are useful in different scenarios.