key not found: _PYSPARK_DRIVER_CALLBACK_HOST

Question

I\'m trying to run this code:

import pyspark
from pyspark.sql import SparkSession

spark = SparkSession.builder \\
        .master(\"local\") \\
        .app         


        

Answer 1

I have got the similar errors: java.util.NoSuchElementException: key not found: _PYSPARK_DRIVER_CALLBACK_HOST and Exception: Java gateway process exited before sending its port number

Running the command "export PYTHONPATH=$SPARK_HOME/python/:$PYTHONPATH" or setting this to .bashrc resolved the issue.

Please also check if the mapr credentails are setup.