Reading a file from a private S3 bucket to a pandas dataframe

前端 未结 8 760
猫巷女王i
猫巷女王i 2020-12-08 10:19

I\'m trying to read a CSV file from a private S3 bucket to a pandas dataframe:

df = pandas.read_csv(\'s3://mybucket/file.csv\')

I can read

8条回答
  •  抹茶落季
    2020-12-08 10:47

    import pandas as pd
    import boto3
    from io import StringIO
    
    # Read CSV
    s3 = boto3.client('s3',endpoint_url,aws_access_key_id=,aws_secret_access_key)
    read_file = s3.get_object(Bucket, Key)
    df = pd.read_csv(read_file['Body'],sep=',')
    
    # Write CSV
    csv_buffer = StringIO()
    df.to_csv(csv_buffer)
    s3.put_object(Bucket, Key,Body=csv_buffer.getvalue())
    

提交回复
热议问题