How to pipe tar.extractall from python

问题

I'm extracting a tarball using the tarfile module of python. I don't want the extracted files to be written on the disk, but rather get piped directly to another program, specifically bgzip. I'm also trying to use StringIO for that matter, but I get stuck even on that stage - the tarball gets extracted on the disk.

#!/usr/bin/env python
import tarfile, StringIO
tar = tarfile.open("6genomes.tgz", "r:gz")
def enafun(members):
    for tarkati in tar:
        if tarkati.isreg():
            yield tarkati
reles = StringIO.StringIO()
reles.write(tar.extractall(members=enafun(tar)))
tar.close()

How then do I pipe correctly the output of tar.extractall?

回答1:

You cannot use extractall method, but you can use getmembers and extractfile methods instead:

#!/usr/bin/env python
import tarfile, StringIO
reles = StringIO.StringIO()
with tarfile.open("6genomes.tgz", "r:gz") as tar:
    for m in tar.members():
        if m.isreg():
            reles.write(tar.extractfile(m).read())
# do what you want with "reles".

According to the documentation, extractfile() method can take a TarInfo and will return a file-like object. You can then get the content of that file with read().

[EDIT] I add what you asked me in comment as formatting in comment seems not to render properly.

#!/usr/bin/env python
import tarfile
import subprocess
with tarfile.open("6genomes.tgz", "r:gz") as tar:
    for m in tar.members():
        if m.isreg():
            f = tar.extractfile(m)
            new_filename = generate_new_filename(f.name)
            with open(new_filename, 'wb') as new_file:
                proc = subprocess.Popen(['bgzip', '-c'], stdin=subprocess.PIPE, stdout=new_file)
                proc.stdin.write(f.read())
                proc.stdin.close()
                proc.wait()
            f.close()

来源：https://stackoverflow.com/questions/30486077/how-to-pipe-tar-extractall-from-python