Read a file line by line from S3 using boto?

前端 未结 10 1031
刺人心
刺人心 2020-11-29 07:32

I have a csv file in S3 and I\'m trying to read the header line to get the size (these files are created by our users so they could be almost any size). Is there a way to do

10条回答
  •  旧巷少年郎
    2020-11-29 07:50

    With boto3 you can access a raw stream and read line by line. Just note raw stream is a private property for some reason

    s3 = boto3.resource('s3', aws_access_key_id='xxx', aws_secret_access_key='xxx')
    obj = s3.Object('bucket name', 'file key')
    
    obj.get()['Body']._raw_stream.readline() # line 1
    obj.get()['Body']._raw_stream.readline() # line 2
    obj.get()['Body']._raw_stream.readline() # line 3...
    

提交回复
热议问题