Why child process (daemon=True) not exiting when main process exit in python?

问题

Here is the official explanation of daemon flag in python multiprocessing:

When a process exits, it attempts to terminate all of its daemonic child processes.

By my understanding, the parent process will kill its children whose daemon flag is set to be True when it exits.

Below is the code I used to prove my guess. But the result is different.

import multiprocessing


def child():
    while True:
        pass


for x in xrange(1, 4):
    proc = multiprocessing.Process(target=child, args=())
    proc.daemon=True
    proc.start()


while True:
    pass

The above starts 4 child processes and one main process. I killed the main process but the 4 children did not exit.

So why are they not terminated by main since the daemon is set to be true?

回答1:

Notes:

The use of xrange the implies Python 2
xrange(1, 4) will yield 3 values not 4 (so, there will only be 3 children)

This is not quite how things work. The doc ([Python 2.Docs]: multiprocessing - daemon) should probably be more specific.

The thing is that multiprocessing registers a cleanup function to kill all its deamonic children when exiting. That is done via [Python 2.Docs]: atexit - Exit handlers:

Note: The functions registered via this module are not called when the program is killed by a signal not handled by Python, when a Python fatal internal error is detected, or when os._exit() is called.

You don't handle the TERM signal (sent by default by the kill command), therefore the cleanup function is not called by the main process (leaving its children running).

I modified your code to better illustrate the behavior.

code00.py:

#!/usr/bin/env python2

import sys
import multiprocessing
import os
import time


print_text_pattern = "Output from process {0:s} - pid: {1:d}, ppid: {2:d}"


def child(name):
    while True:
        print(print_text_pattern.format(name, os.getpid(), os.getppid()))
        time.sleep(1)


def main():
    procs = list()
    for x in xrange(1, 3):
        proc_name = "Child{0:d}".format(x)
        proc = multiprocessing.Process(target=child, args=(proc_name,))
        proc.daemon = True #x % 2 == 0
        print("Process {0:s} daemon: {1:}".format(proc_name, proc.daemon))
        procs.append(proc)

    for proc in procs:
        proc.start()

    counter = 0
    while counter < 3:
        print(print_text_pattern.format("Main", os.getpid(), os.getppid()))
        time.sleep(1)
        counter += 1


if __name__ == "__main__":
    print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    main()
    print("\nDone.")

Notes:

Changed the way how children processes are spawned a bit: all of them are created 1^st, and only then started
Added some print calls from each process, to track their activity in the stdout - also added some time.sleep calls (1 second), to avoid producing too much output
Most important - the main process no longer runs forever. At some point it exits gracefully (after 3 cycles - due to counter variable), and there's when the behavior that I mentioned earlier kicks in.
This could also have been possible by intercepting the TERM signal (and others that can be explicitly be sent by the kill command) and performing the cleanup then - in that way the children would be killed as well when killing the main process - but that's more complicated
I simplified things a bit so that only 2 children are spawned
Moved everything in a main function (for structure) enclosed in a if __name__ == "__main__": conditional, so the processes are not spawned if you import the module
Give different values proc.daemon for each child then monitor the output and ps -ef | grep "code00.py" output
Added an argument (name) to child func, but that's only for display purposes

Output:

[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow]> python2 code00.py
Python 2.7.12 (default, Oct  8 2019, 14:14:10) [GCC 5.4.0 20160609] 64bit on linux2

Process Child1 daemon: True
Process Child2 daemon: True
Output from process Main - pid: 1433, ppid: 1209
Output from process Child1 - pid: 1434, ppid: 1433
Output from process Child2 - pid: 1435, ppid: 1433
Output from process Main - pid: 1433, ppid: 1209
Output from process Child2 - pid: 1435, ppid: 1433
Output from process Child1 - pid: 1434, ppid: 1433
Output from process Main - pid: 1433, ppid: 1209
Output from process Child1 - pid: 1434, ppid: 1433
Output from process Child2 - pid: 1435, ppid: 1433
Output from process Child1 - pid: 1434, ppid: 1433
Output from process Child2 - pid: 1435, ppid: 1433

Done.

回答2:

Yes, your understanding is correct and your code to test this also works.

I just added some sleep statements to debug the output (without sleep, it is difficult to infer from the huge output of prints):

import multiprocessing
import time
import sys

print("main")

def child():
    while True:
        print("child")
        time.sleep(3)

for x in xrange(1, 4):
    proc = multiprocessing.Process(target=child, args=())
    proc.daemon=True
    proc.start()

time.sleep(7)
print("exit")
sys.exit() # this exits the main process

Now, when I ran this script, and when it was running, I did a ps aux and could see four processes running from this script. After 7 seconds, when I did ps aux again, I could no more see those processes running - which means:

When the main process exited, it terminated all of its daemonic child processes.

After that, I also set the proc.daemon to False, and ran the script once again. This time, even after 7 seconds, when I did a ps aux, I could still see the child processes running (since they are non-daemonic now, they do not exit even after main process terminates).

So this works as expected - let me know if you still have an issue here.

Edit 1: Thanks to @CristiFati for pointing out the original cleanup issue. This code works because calling sys.exit() also registers the atexit callbacks as elaborated here.

来源：https://stackoverflow.com/questions/49604997/why-child-process-daemon-true-not-exiting-when-main-process-exit-in-python

标签

python

process

daemon