问题
This question already has an answer here:
- Requirements for converting Spark dataframe to Pandas/R dataframe 1 answer
Is that possible to
convert from to pd.DataFrame
under %pyspark environment ?
回答1:
Try:
spark_df.toPandas()
toPandas()
Returns the contents of this DataFrame as Pandas pandas.DataFrame. This is only available if Pandas is installed and available.
And if you want the oposite:
spark_df = createDataFrame(pandas_df)
来源：https://stackoverflow.com/questions/41826553/convert-between-spark-sql-dataframe-and-pandas-dataframe