How to know which protobuf message the byte array is?

问题

I want to use protobuf instead of Json to communicate between message queue.

I know how to deal with it when there is only one proto message.

Assume that the proto file is:

//person.proto
syntax = "proto3";

option java_outer_classname = "PersonProto";

message Person {
    int32 id = 2;
    string name = 1;
    string email = 3;
}

Now, i can deal with it with the approach below:

PersonProto.Person person = PersonProto.Person.newBuilder()
        .setEmail("123@test.com")
        .setId(1)
        .setName("name-test")
        .build();

byte[] bytes = person.toByteArray();

//Transfer from publisher to consumer between message queue.

//I can deserialise it, because i know the proto message is Person.
PersonProto.Person.parseFrom(bytes);

But what if there are multiple proto messages?

Assume that there is another proto message called Address.

syntax = "proto3";

option java_outer_classname = "PersonProto";

message Person {
    int32 id = 2;
    string name = 1;
    string email = 3;
}

message Address {
    string address = 1;
}

When consumer received byte array from message queue, how to know which proto message it is? and how to deserialise the byte array?

回答1:

Protobuf 3 introduced the concept of Any that can act in a similar way to the top-level-message pattern explained by @AdamCozzette.

On the write-side you pack your message in an Any:

Person person = ...
Any any = Any.pack(person);

out.write(any.toByteArray());

Then on the read-side you read into an Any and switch on the types you are interested in:

Any any = Any.parseFrom(in);

if (any.is(Person.class)
{
  Person person = any.unpack(Person.class);
  ...
}
else if (any.is(Address.class);
{
  Address address = any.unpack(Address.class);
  ...
}
else
{
  //Handle unknown message
}

Using Any removes the need for special a message type (the top-level-message), but also removes an element of type-safety in as much as you may receive messages that the consuming code does not know how to handle.

回答2:

Protocol buffers are not self-describing, so in general when you get a serialized protobuf there is no way to interpret its contents without knowing what schema to expect.

In your case I would recommend using a oneof field. You can have a single top-level message type for your queue messages and let that contain a oneof field that contains either a Person or an Address:

message TopLevelMessage {
  oneof inner_message {
    Person person = 1;
    Address address = 2;
  }
}

The consuming code would then need a switch statement like this in order to retrieve the inner-message:

TopLevelMessage topLevelMessage = TopLevelMessage.parseFrom(...);

switch (topLevelMessage.getInnerMessageCase()) 
{
  case PERSON:
    Person person = topLevelMessage.getPerson();
    ...
    break;

  case ADDRESS:
    Address address = topLevelMessage.getAddress();
    ...
    break;

  default:
     ... 
}

来源：https://stackoverflow.com/questions/40450556/how-to-know-which-protobuf-message-the-byte-array-is

标签

message-queue

protocol-buffers