Serialize and deserialize protobufs through CLI?

问题

I am trying to deserialize a file saved as a protobuf through the CLI (seems like the easiest thing to do). I would prefer not to use protoc to compile, import it into a programming language and then read the result.

My use case: A TensorFlow lite tool has output some data in a protobuf format. I've found the protobuf message definition in the TensorFlow repo too. I just want to read the output quickly. Specifically, I am getting back a tflite::evaluation::EvaluationStageMetrics message from the inference_diff tool.

回答1:

I assume that the tool outputs a protobuf message in binary format.

protoc can decode the message and output in text format. See this option:

  --decode=MESSAGE_TYPE       Read a binary message of the given type from
                              standard input and write it in text format
                              to standard output.  The message type must
                              be defined in PROTO_FILES or their imports.

回答2:

While Timo Stamms answer was instrumental, I still struggled with the paths to get protoc to work in a complex repo (e.g. TensorFlow).

In the end, this worked for me:

cat inference_diff.txt | \
    protoc --proto_path="/Users/ben/butter/repos/tensorflow/" \
    --decode tflite.evaluation.EvaluationStageMetrics \ 
    $(pwd)/evaluation_config.proto

Here I pipe the binary contents of the file containing protobuf (inference_diff.txt in my case, generated by following this guide), and specify the fully qualified protobuf message (which I got by combining the package tflite.evaluation; and the message name, EvaluationStageMetrics), the absolute path of the project for the proto_path (which is the project root/ TensorFlow repo), and also the absolute path for the file which actually contains the message. proto_path is just used for resolving imports, where as the PROTO_FILE (in this case, evaluation_config.proto), is used to decode the file.

Example Output

num_runs: 50
process_metrics {
  inference_profiler_metrics {
    reference_latency {
      last_us: 455818
      max_us: 577312
      min_us: 453121
      sum_us: 72573828
      avg_us: 483825.52
      std_deviation_us: 37940
    }
    test_latency {
      last_us: 59503
      max_us: 66746
      min_us: 57828
      sum_us: 8992747
      avg_us: 59951.646666666667
      std_deviation_us: 1284
    }
    output_errors {
      max_value: 122.371696
      min_value: 83.0335922
      avg_value: 100.17548828125
      std_deviation: 8.16124535
    }
  }
}

If you just want to get the numbers in a rush and can't be bothered to fix the paths, you can do

cat inference_diff.txt | protoc --decode_raw

Example output

1: 50
2 {
  5 {
    1 {
      1: 455818
      2: 577312
      3: 453121
      4: 72573828
      5: 0x411d87c6147ae148
      6: 37940
    }
    2 {
      1: 59503
      2: 66746
      3: 57828
      4: 8992747
      5: 0x40ed45f4b17e4b18
      6: 1284
    }
    3 {
      1: 0x42f4be4f
      2: 0x42a61133
      3: 0x40590b3b33333333
      4: 0x41029476
    }
  }
}

来源：https://stackoverflow.com/questions/64959850/serialize-and-deserialize-protobufs-through-cli

标签

protocol-buffers