Making sure a Python script with subprocesses dies on SIGINT

Question

I\'ve got a command that I\'m wrapping in script and spawning from a Python script using subprocess.Popen. I\'m trying to make sure it dies if the

Answer 1

Fist thing; there is a send_signal() method on the Popen object. If you want to send a signal to one you've launched, use this method to send it.

Second thing; a deeper problem with the way you're setting up communication with your subprocess and then, um, not communicating with it. You cannot safely tell the subprocess to send its output to subprocess.PIPE and then not read from the pipes. UNIX pipes are buffered (typically a 4K buffer?), and if the subprocess fills up the buffer and the process on the other end of the pipe doesn't read the buffered data, the subprocess will pend (locking up, from an observer's perspective) on its next write to the pipe. So, the usual pattern when using subprocess.PIPE is to call communicate() on the Popen object.

It is not mandatory to use subprocess.PIPE if you want data back from the subprocess. A cool trick is to use the tempfile.TemporaryFile class to make an unnamed temp file (really it opens a file and immediately deletes the inode from the file system, so you have access to the file but no-one else can open one. You can do something like:

with tempfile.TemporaryFile() as iofile:
    p = Popen(cmd, stdout=iofile, stderr=iofile)
    while True:
        if p.poll() is not None:
            break
        else:
            time.sleep(0.1) # without some sleep, this polling is VERY busy...

Then you can read the contents of your temporary file (seek to the beginning of it before you do, to be sure you're at the beginning) when you know the subprocess has exited, instead of using pipes. The pipe buffering problem won't be a problem if the subprocess's output is going to a file (temporary or not).

Here is a riff on your code sample that I think does what you want. The signal handler just repeats the signals being trapped by the parent process (in this example, SIGINT and SIGTERM) to all current subprocesses (there should only ever be one in this program) and sets a module-level flag saying to shutdown at the next opportunity. Since I'm using subprocess.PIPE I/O, I call communicate() on the Popen object.

#!/usr/bin/env python

from subprocess import Popen, PIPE
import signal
import sys

current_subprocs = set()
shutdown = False

def handle_signal(signum, frame):
    # send signal recieved to subprocesses
    global shutdown
    shutdown = True
    for proc in current_subprocs:
        if proc.poll() is None:
            proc.send_signal(signum)


signal.signal(signal.SIGINT, handle_signal)
signal.signal(signal.SIGTERM, handle_signal)

for _ in range(10):
    if shutdown:
        break
    cmd = ["sleep", "2"]
    p = Popen(cmd, stdout=PIPE, stderr=PIPE)
    current_subprocs.add(p)
    out, err = p.communicate()
    current_subprocs.remove(p)
    print "subproc returncode", p.returncode

And calling it (with a Ctrl-C in the third 2 second interval):

% python /tmp/proctest.py 
subproc returncode 0
subproc returncode 0
^Csubproc returncode -2