Writing Dask partitions into single file

前端 未结 2 962
孤城傲影
孤城傲影 2020-12-29 04:04

New to dask,I have a 1GB CSV file when I read it in dask dataframe it creates around 50 partitions after my changes in the file when I

2条回答
  •  慢半拍i
    慢半拍i (楼主)
    2020-12-29 04:43

    you can convert your dask dataframe to a pandas dataframe with the compute function and then use the to_csv. something like this:

    df_dask.compute().to_csv('csv_path_file.csv')

提交回复
热议问题