Python ThreadPoolExecutor 中的假守护线程

现象

观察 ThreadPoolExecutor的submit代码：

    def _adjust_thread_count(self):
        # When the executor gets lost, the weakref callback will wake up
        # the worker threads.
        def weakref_cb(_, q=self._work_queue):
            q.put(None)
        # TODO(bquinlan): Should avoid creating new threads if there are more
        # idle threads than items in the work queue.
        if len(self._threads) < self._max_workers:
            t = threading.Thread(target=_worker,
                                 args=(weakref.ref(self, weakref_cb),
                                       self._work_queue))
            t.daemon = True   #确实是守护线程
            t.start()
            self._threads.add(t)
            _threads_queues[t] = self._work_queue

在线程池里面启动的线程确实都是守护线程，但是主线程退出后，进程并没有退出，而是还在等子线程结束。接着深入代码以后发现是ThreadPool设计上为了不引起突然中断造成线程结束的其他坏影响，比如文件写到一半等，注册了atexit退出方法。简单的说就是在调用线程退出时，并没有真正退出，而是会去调用这个注册在ateixt上的方法，而线程池这个文件的退出方法就是等线程池中的所有线程结束后join退出。

总之，挺迷的设计。但python handle不了的异常或退出，还是能正确退出的。

代码如下

代码

# Workers are created as daemon threads. This is done to allow the interpreter
# to exit when there are still idle threads in a ThreadPoolExecutor's thread
# pool (i.e. shutdown() was not called). However, allowing workers to die with
# the interpreter has two undesirable properties:
#   - The workers would still be running during interpreter shutdown,
#     meaning that they would fail in unpredictable ways.
#   - The workers could be killed while evaluating a work item, which could
#     be bad if the callable being evaluated has external side-effects e.g.
#     writing to a file.
#
# To work around this problem, an exit handler is installed which tells the
# workers to exit when their work queues are empty and then waits until the
# threads finish.

_threads_queues = weakref.WeakKeyDictionary()
_shutdown = False

def _python_exit():
    global _shutdown
    _shutdown = True
    items = list(_threads_queues.items())
    for t, q in items:
        q.put(None)
    for t, q in items:
        t.join()

atexit.register(_python_exit)

来源：oschina

链接：https://my.oschina.net/u/3534184/blog/4258747

标签

def

python

around

threadpool