Python's subprocess.Popen object hangs gathering child output when child process does not exit

Question

When a process exits abnormally or not at all, I still want to be able to gather what output it may have generated up until that point.

The obvious solution to this

Answer 1

Here's a POSIX way of doing it without the temporary file. I realize that subprocess is a little superfluous here, but since the original question used it...

import subprocess
import os
import time
import signal
import sys

pr, pw = os.pipe()
pid = os.fork () 

if pid: #parent
    os.close(pw)
    cmd = ["bash"]
    finish = time.time() + 3
    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=pr, close_fds=True)
    while p.poll() is None:
        time.sleep(0.05)
        if finish < time.time():
            os.kill(p.pid, signal.SIGTERM)
            print "timed out and killed child, collecting what output exists so far"
            out, err = p.communicate()
            print "got it: ", out
            sys.exit(0)

else: #child
    os.close(pr)
    child_script = """
    #!/bin/bash
    while [ 1 ]; do
        ((++i))
        echo "output line $i"
        sleep 1
    done
    """
    os.write(pw, child_script)