Write a Pandas DataFrame to Google Cloud Storage or BigQuery

前端 未结 8 1192
予麋鹿
予麋鹿 2020-12-02 13:16

Hello and thanks for your time and consideration. I am developing a Jupyter Notebook in the Google Cloud Platform / Datalab. I have created a Pandas DataFrame and would like

8条回答
  •  执笔经年
    2020-12-02 14:13

    I have a little bit simpler solution for the task using Dask. You can convert your DataFrame to Dask DataFrame, which can be written to csv on Cloud Storage

    import dask.dataframe as dd
    import pandas
    df # your Pandas DataFrame
    ddf = dd.from_pandas(df,npartitions=1, sort=True)
    dd.to_csv('gs://YOUR_BUCKET/ddf-*.csv', index=False, sep=',', header=False,  
                                   storage_options={'token': gcs.session.credentials})  
    

提交回复
热议问题