Unable to run a python flink application on cluster

允我心安 提交于 2019-12-13 03:57:40

问题


I am trying to run a Python Flink Application on the standalone Flink cluster. The application works fine on a single node cluster but it throws the following error on a multi-node cluster. java.lang.Exception: The user defined 'open()' method caused an exception: An error occurred while copying the file. Please help me resolve this problem. Thank you

The application I am trying to execute has the following code.

from flink.plan.Environment import get_environment
from flink.plan.Constants import INT, STRING, WriteMode

env = get_environment()

data = env.from_elements("Hello")

data.map(lambda x: list(x)).output()
env.execute()

回答1:


You have to configure "python.dc.tmp.dir" in "flink-conf.yaml" to point to a distributed filesystem (like HDFS). This directory is used to distributed the python scripts.



来源:https://stackoverflow.com/questions/49005505/unable-to-run-a-python-flink-application-on-cluster

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!