Run background process in Python and do NOT wait

Question

My goal is simple: kick off rsync and DO NOT WAIT.

Python 2.7.9 on Debian

Sample code:

rsync_cmd = \"/usr/bin/rsync -a -e \'ssh -i /home/myus         


        

Answer 1

Popen() starts a child process—it does not wait for it to exit. You have to call .wait() method explicitly if you want to wait for the child process. In that sense, all subprocesses are background processes.

On the other hand, the child process may inherit various properties/resources from the parent such as open file descriptors, the process group, its control terminal, some signal configuration, etc—it may lead to preventing ancestors processes to exit e.g., Python subprocess .check_call vs .check_output or the child may die prematurely on Ctrl-C (SIGINT signal is sent to the foreground process group) or if the terminal session is closed (SIGHUP).

To disassociate the child process completely, you should make it a daemon. Sometimes something in between could be enough e.g., it is enough to redirect the inherited stdout in a grandchild so that .communicate() in the parent would return when its immediate child exits.

Answer 2

I encountered a similar issue while working with qnx devices and wanted a sub-process that runs independently of the main process and even runs after the main process terminates. Here's the solution I found that actually works 'creationflags=subprocess.DETACHED_PROCESS':

import subprocess
import time

pid = subprocess.Popen(["python", "path_to_script\turn_ecu_on.py"], creationflags=subprocess.DETACHED_PROCESS)

time.sleep(15)
print("Done")

Link to the doc: https://docs.python.org/3/library/subprocess.html#subprocess.Popen

Answer 3

Here is verified example for Python REPL:

>>> import subprocess
>>> import sys
>>> p = subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(100)'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT); print('finished')
finished

How to verify that via another terminal window:

$ ps aux | grep python

Output:

user           32820   0.0  0.0  2447684   3972 s003  S+   10:11PM   0:00.01 /Users/user/venv/bin/python -c import time; time.sleep(100)