Can mmap and gzip collaborate?

Question

I'm trying to figure how to use mmap with a gzip compressed file. Is that even possible ?

import mmap
import os
import gzip

filename = r'C:\temp\data.gz'

file = gzip.open(filename, "rb+")
size = os.path.getsize(filename)

file = mmap.mmap(file.fileno(), size)

print file.read(8)

The output data is compressed.

Answer 1

Well, not the way you want.

mmap() can be used to access the gzipped file if the compressed data is what you want.

mmap() is a system call for mapping disk blocks into RAM almost as if you were adding swap.

You can't map the uncompressed data into RAM with mmap() as it is not on the disk.

Answer 2

You can do easilly. Indeed the gzip module gets as optional argument a file-like object.

import mmap
import gzip

filename = "a.gz"
handle = open(filename, "rb")
mapped = mmap.mmap(handle.fileno(), 0, access=mmap.ACCESS_READ)
gzfile = gzip.GzipFile(mode="r", fileobj=mapped)

print gzfile.read()

The same applies to tarfile module:

import sys
import mmap
import tarfile

f = open(sys.argv[1], 'rb')
fo = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
tf = tarfile.open(mode='r:gz', fileobj=fo)

print tf.getnames()