Using IO library to load string variable as a txt file to/from s3

泪湿孤枕 提交于 2020-01-15 09:27:28

问题


I have old code below that gzips a file and stores it as json into S3, using the IO library ( so a file does not save locally). I am having trouble converting this same approach (ie using IO library for a buffer) to create a .txt file and push into S3 and later retrieve. I know how to create txt files and push into s3 is as well, but not how to use IO in the process.

The value I would want to be stored in the text value would just be a variable with a string value of 'test'

Goal: Use IO library and save string variable as a text file into S3 and be able to pull it down again.

x = 'test'
inmemory = io.BytesIO() 
    with gzip.GzipFile(fileobj=inmemory, mode='wb') as fh:

        with io.TextIOWrapper(fh, encoding='utf-8',errors='replace') as wrapper:
            wrapper.write(json.dumps(x, ensure_ascii=False,indent=2))
    inmemory.seek(0)
    s3_resource.Object(s3bucket, s3path + '.json.gz').upload_fileobj(inmemory)  
    inmemory.close()

Also any documentation with that anyone likes with specific respect to the IO library and writing to files, because the actual documentation ( f = io.StringIO("some initial text data") ect..https://docs.python.org/3/library/io.html ) It just did not give me enough at my current level.


回答1:


Duplicate.

For sake of brevity, it turns out there's a way to override the putObject call so that it takes in a string of text instead of a file.

The original post is answered in Java, but this additional thread should be sufficient for a Python-specific answer.



来源:https://stackoverflow.com/questions/59539656/using-io-library-to-load-string-variable-as-a-txt-file-to-from-s3

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!