Is there a standard mapping between JSON and Protocol Buffers?

Question

From a comment on the announcement blog post:

Regarding JSON: JSON is structured similarly to Protocol Buffers, but protocol buffer binary format is still smaller and faster to encode. JSON makes a great text encoding for protocol buffers, though -- it's trivial to write an encoder/decoder that converts arbitrary protocol messages to and from JSON, using protobuf reflection. This is a good way to communicate with AJAX apps, since making the user download a full protobuf decoder when they visit your page might be too much.

It may be trivial to cook up a mapping, but is there a single "obvious" mapping between the two that any two separate dev teams would naturally settle on? If two products supported PB data and could interoperate because they shared the same .proto spec, I wonder if they would still be able to interoperate if they independently introduced a JSON reflection of the same spec. There might be some arbitrary decisions to be made, e.g. should enum values be represented by a string (to be human-readable a la typical JSON) or by their integer value?

So is there an established mapping, and any open source implementations for generating JSON encoder/decoders from .proto specs?

Answer 1

Yes, since Protocol Buffers version 3.0.0 (released July 28, 2016) there is "A well-defined encoding in JSON as an alternative to binary proto encoding" as mentioned in the release notes

https://github.com/google/protobuf/releases/tag/v3.0.0

Answer 2

May be this is helpful http://code.google.com/p/protobuf-java-format/

Answer 3

From what I have seen, Protostuff is the project to use for any PB work on Java, including serializing it as JSON, based on protocol definition. I have not used it myself, just heard good things.

Answer 4

I needed to marshal from GeneratedMessageLite to a JSON object but did not need to unmarshal. I couldn't use the protobuf library in Pangea's answer because it doesn't work with the LITE_RUNTIME option. I also didn't want to burden our already large legacy system with generating more compiled code for the existing protocol buffers. For mashalling to JSON, I went with this simple solution to marshal

    final Person gpb = Person.newBuilder().setName("Bill Monroe").build();
    final Gson gson = new Gson();
    final String jsonString = gson.toJson(gpb);

Answer 5

One further thought: if protobuf objects have getters/setters, or appropriately named fields, one could simply use Jackson JSON processor's data binding. By default it handles public getters, any setters and public fields, but these are just default visibility levels and can be changed. If so, Jackson can serialize/deserialize protobuf generated POJOs without problems.

I have actually used this approach with Thrift-generated objects; the only thing I had to configure there was to disable serialization of various "isXXX()" methods that Thrift adds for checking if a field has been explicitly assigned or not.

Answer 6

First of all I think one should reason very carefully on putting an effort into converting a dataset to protobuffs. Here my reasons to convert a data-set to protobuffs

1. Type Safety: guarantee on the format of the data being considered.
2. uncompressed memory foot-print of the data. The reason I mention un-compressed is because post compression there isn't much of a difference in the size of JSON compressed and proto compressed but compression has a cost associated with it. Also, the serialization/de-serialization speed is almost similar, infact Jackson json is faster than protobuffs. Please checkout the following link for more information http://technicalrex.com/2014/06/23/performance-playground-jackson-vs-protocol-buffers/
3. The protobuffs needs to be transferred over the network a lot.

Saying that once you convert your data-set to Jackson JSON format in the way that the ProtoBuff definition is defined then it can very easily be directly mapped to ProtoBuff format using the Protostuff:JsonIoUtil:mergeFrom function. Signature of the function :

public static <T> void mergeFrom(JsonParser parser, T message, Schema<T> schema,  boolean numeric) 

Reference to protostuff