Exporting spark dataframe to .csv with header and specific filename
问题 I am trying to export data from a spark dataframe to .csv file: df.coalesce(1)\ .write\ .format("com.databricks.spark.csv")\ .option("header", "true")\ .save(output_path) It is creating a file name "part-r-00001-512872f2-9b51-46c5-b0ee-31d626063571.csv" I want the filename to be "part-r-00000.csv" or "part-00000.csv" As the file is being created on AWS S3, I am limited in how I can use os.system commands. How can I set the file name while keeping the header in the file? Thanks! 回答1: Well,