Python decompress gzip data in memory without file

问题

I have gzipped data from HTTP reply. I have following code:

def gzipDecode(self, content):
    import StringIO
    import gzip

    outFilePath = 'test'

    compressedFile = StringIO.StringIO(content)
    decompressedFile = gzip.GzipFile(fileobj=compressedFile)
    with open(outFilePath, 'w') as outfile:
        outfile.write(decompressedFile.read())

    data = ''
    with open(outFilePath, 'r') as myfile:
        data=myfile.read().replace('\n', '')

    return data

which decompress input gzipped content and return string (http reply is gzipped json). - It works.

But I need it without creating test file - all in memory.

I modified it to:

def gzipDecode(self, content):
    import StringIO
    from io import BytesIO
    import gzip

    outFile = StringIO.StringIO()

    compressedFile = StringIO.StringIO(content)
    decompressedFile = gzip.GzipFile(fileobj=compressedFile)

    outFile.write(decompressedFile.read())
    outFile.flush()

    data = outFile.read().replace('\n', '')
    print "_" + data + "_"
    return data

but it crash (gzipDecode produce empty output) in parsing json:

Traceback (most recent call last):
__
  File "/usr/lib/python2.7/SocketServer.py", line 593, in process_request_thread

    self.finish_request(request, client_address)
----------------------------------------
  File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
Exception happened during processing of request from ('10.123.66.3', 39853)
    self.RequestHandlerClass(request, client_address, self)
----------------------------------------
  File "/usr/lib/python2.7/SocketServer.py", line 649, in __init__
    self.handle()
  File "/usr/lib/python2.7/BaseHTTPServer.py", line 340, in handle
    self.handle_one_request()
  File "/usr/lib/python2.7/BaseHTTPServer.py", line 328, in handle_one_request
    method()
  File "/tmp/test_server.py", line 92, in do_POST
    data = json.loads(file_content)
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

What I do bad?

回答1:

You need to seek back to the start before you can read:

outFile.write(decompressedFile.read())
outFile.flush()
outFile.seek(0)

data = outFile.read().replace('\n', '')

来源：https://stackoverflow.com/questions/36282037/python-decompress-gzip-data-in-memory-without-file

标签

python

json

gzip

compression

stringio