Spark - How to write a single csv file WITHOUT folder?

前端 未结 9 1201
北恋
北恋 2020-12-28 13:44

Suppose that df is a dataframe in Spark. The way to write df into a single CSV file is

df.coalesce(1).write.option(\"header\", \"tru

9条回答
  •  失恋的感觉
    2020-12-28 14:21

    A possible solution could be convert the Spark dataframe to a pandas dataframe and save it as csv:

    df.toPandas().to_csv("/")
    

提交回复
热议问题