PySpark, Win10 - The system cannot find the path specified

淺唱寂寞╮ 提交于 2021-01-27 20:54:19

问题


I previously had PySpark installed as a Python package I installed through pip, I uninstalled it recently with a clean version of Python and downloaded the standalone version.

In my User variables I made a path with name: SPARK_HOME

with a value of: C:\spark-2.3.2-bin-hadoop2.7\bin

In System variables under Path I made an entry: C:\spark-2.3.2-bin-hadoop2.7\bin

When I run pyspark

I can not run spark-shell either. Any ideas?


回答1:


SPARK_HOME should be without bin folder. Hence,

Set SPARK_HOME to C:\spark-2.3.2-bin-hadoop2.7\




回答2:


Window users have to download a compatible winutils exe version and save it in your Spark's bin folder.

Find the compatible Hadoop distribution, download and save it in your Spark folder.

e.g. Download "https://github.com/steveloughran/winutils/blob/master/hadoop-2.7.1/bin/winutils.exe" and save it in your "C:\spark-2.3.2-bin-hadoop2.7\bin"

Different winutils version could be found in this link. https://github.com/steveloughran/winutils



来源:https://stackoverflow.com/questions/52726043/pyspark-win10-the-system-cannot-find-the-path-specified

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!