Cannot parse a protocol buffers file in python when using the correct .proto file

问题

(see update at bottom)

Tilemaker is an OpenStreetMap programme to generate Mapbox vector tiles (which are themselves protocol buffers (pbf) files) from an OSM pbf data file. I have compiled it and used it to create a directory of vector tiles. I cannot parse those files in Python.

I created the vector tiles with:

tilemaker input.pbf --output=tiles/

Then I created a simple python programme, based on Google's Protocol Buffers Python Tutorial in this way:

Compiling the .proto files:

mkdir py
touch py/__init__.py
protoc --proto_path=include --python_out=./py ./include/osmformat.proto
protoc --proto_path=include --python_out=./py ./include/vector_tile.proto

This python programme pyread.py doesn't work:

import sys
import py.vector_tile_pb2

with open(sys.argv[1]) as fp:
    pbf_file_contents = fp.read()

tile = py.vector_tile_pb2.Tile()
tile.ParseFromString(pbf_file_contents)

This is the error when trying to run it:

$ python pyread.py ./tiles/13/3932/2588.pbf
Traceback (most recent call last):
  File "pyread.py", line 8, in <module>
    tile.ParseFromString(pbf_file_contents)
  File "/home/rory/.local/lib/python2.7/site-packages/google/protobuf/message.py", line 186, in ParseFromString
    self.MergeFromString(serialized)
  File "/home/rory/.local/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 841, in MergeFromString
    if self._InternalParse(serialized, 0, length) != length:
  File "/home/rory/.local/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 866, in InternalParse
    new_pos = local_SkipField(buffer, new_pos, end, tag_bytes)
  File "/home/rory/.local/lib/python2.7/site-packages/google/protobuf/internal/decoder.py", line 827, in SkipField
    return WIRETYPE_TO_SKIPPER[wire_type](buffer, pos, end)
  File "/home/rory/.local/lib/python2.7/site-packages/google/protobuf/internal/decoder.py", line 797, in _RaiseInvalidWireType
    raise _DecodeError('Tag had invalid wire type.')
google.protobuf.message.DecodeError: Tag had invalid wire type.

The protoc command is from the protcol buffers library. I downloaded the latest release (2.6.1) from Google's page (which links to Github) and compiled & installed it. That protoc invocation is just like what the Tilemaker Makefile does.

What's going on? How can I read this protocol buffers file in python?

UPDATE Further investigation makes me think that one of my assumptions might be wrong. Namely, that the tilemaker command has produced a valid protobuf file. I got some vector tiles from Mapzen, which should have the same format and very similar data. But this format works with the python pyread.py command, and with protoc --decode_raw and protoc --decode=vector_tile.Tile ./include/vector_tile.proto. Hence I think the problem is with the file I was looking at.

回答1:

I think the problem is that OpenStreetMap's .pbf format is not a raw protobuf. See my answer to your other question:

https://stackoverflow.com/a/35384238/2686899

来源：https://stackoverflow.com/questions/35367800/cannot-parse-a-protocol-buffers-file-in-python-when-using-the-correct-proto-fil

标签

python

openstreetmap

protocol-buffers