gzipfile

How to gzip while uploading into s3 using boto

ぐ巨炮叔叔 提交于 2020-12-01 09:25:29
问题 I have a large local file. I want to upload a gzipped version of that file into S3 using the boto library. The file is too large to gzip it efficiently on disk prior to uploading, so it should be gzipped in a streamed way during the upload. The boto library knows a function set_contents_from_file() which expects a file-like object it will read from. The gzip library knows the class GzipFile which can get an object via the parameter named fileobj ; it will write to this object when compressing