问题
I was trying to run the beam Python-SDK example, but I had problem in reading the input.
https://cwiki.apache.org/confluence/display/BEAM/Usage+Guide#UsageGuide-RunaPython-SDKPipeline
when I used gs://dataflow-samples/shakespeare/kinglear.txt
as the input, the error was apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://dataflow-samples/shakespeare/kinglear.txt': TypeError("__init__() got an unexpected keyword argument 'response_encoding'",)}
when I used my local file, it seemed it didn't actually read the file, and output nothing. The result didn't include 'has_job', which I am not sure what it actually means https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/wordcount.py#L118.
回答1:
This error is because your version of google-apitools is too old. You need v0.5.23 or newer to avoid this error.
Beam python has strict version requirements for dependencies. If you are running Beam 2.11, you need to be using google-apitools v0.5.26.
来源:https://stackoverflow.com/questions/55630755/input-of-apache-beam-examples-wordcount