Input of apache_beam.examples.wordcount

℡╲_俬逩灬. 提交于 2019-12-18 09:36:41

问题


I was trying to run the beam Python-SDK example, but I had problem in reading the input.

https://cwiki.apache.org/confluence/display/BEAM/Usage+Guide#UsageGuide-RunaPython-SDKPipeline

when I used gs://dataflow-samples/shakespeare/kinglear.txt as the input, the error was apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://dataflow-samples/shakespeare/kinglear.txt': TypeError("__init__() got an unexpected keyword argument 'response_encoding'",)}

when I used my local file, it seemed it didn't actually read the file, and output nothing. The result didn't include 'has_job', which I am not sure what it actually means https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/wordcount.py#L118.


回答1:


This error is because your version of google-apitools is too old. You need v0.5.23 or newer to avoid this error.

Beam python has strict version requirements for dependencies. If you are running Beam 2.11, you need to be using google-apitools v0.5.26.



来源:https://stackoverflow.com/questions/55630755/input-of-apache-beam-examples-wordcount

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!