(null) entry in command string exception in saveAsTextFile() on Pyspark

Question

I am working in PySpark on a Jupyter notebook (Python 2.7) in windows 7. I have an RDD of type pyspark.rdd.PipelinedRDD called idSums. When attempt

Answer 1

You are missing winutils.exe a hadoop binary . Depending upon x64 bit / x32 bit System download the winutils.exe file & set your hadoop home pointing to it.

1st way :

1. Download the file
2. Create hadoop folder in Your System, ex C:
3. Create bin folder in hadoop directory, ex : C:\hadoop\bin
4. paste winutils.exe in bin, ex: C:\hadoop\bin\winuitls.exe
5. In User Variables in System Properties -> Advance System Settings

Create New Variable Name: HADOOP_HOME Path: C:\hadoop\

2nd Way :

You can set hadoop home directly in Your Java Program with the following Command like this :

System.setProperty("hadoop.home.dir","C:\hadoop" );