What are the key differences between Apache Thrift, Google Protocol Buffers, MessagePack, ASN.1 and Apache Avro?

Question

All of these provide binary serialization, RPC frameworks and IDL. I\'m interested in key differences between them and characteristics (performance, ease of use, programming

Answer 1

Adding to the performance perspective, Uber recently evaluated several of these libraries on their engineering blog:

https://eng.uber.com/trip-data-squeeze/

The winner for them? MessagePack + zlib for compression

Our goal was to find the combination of encoding protocol and compression algorithm with the most compact result at the highest speed. We tested encoding protocol and compression algorithm combinations on 2,219 pseudorandom anonymized trips from Uber New York City (put in a text file as JSON).

The lesson here is that your requirements drive which library is right for you. For Uber they couldn't use an IDL based protocol due to the schemaless nature of message passing they have. This eliminated a bunch of options. Also for them it's not only raw encoding/decoding time that comes into play, but the size of data at rest.

Size Results

Speed Results