In PySpark, a Spark dataframe can be converted to Pandas using the API df.toPandas() but when I run a job in a Spark cluster on EMR, I get the following error:
