In Python threading, how I can I track a thread's completion?

Question

I\'ve a python program that spawns a number of threads. These threads last anywhere between 2 seconds to 30 seconds. In the main thread I want to track whenever each thread

Answer 1

Why not just have the threads themselves print a completion message, or call some other completion callback when done?

You can the just join these threads from your main program, so you'll see a bunch of completion messages and your program will terminate when they're all done, as required.

Here's a quick and simple demonstration:

#!/usr/bin/python

import threading
import time

def really_simple_callback(message):
    """
    This is a really simple callback. `sys.stdout` already has a lock built-in,
    so this is fine to do.
    """    
    print message

def threaded_target(sleeptime, callback):
    """
    Target for the threads: sleep and call back with completion message.
    """
    time.sleep(sleeptime)
    callback("%s completed!" % threading.current_thread())

if __name__ == '__main__':
    # Keep track of the threads we create
    threads = []

    # callback_when_done is effectively a function
    callback_when_done = really_simple_callback

    for idx in xrange(0, 10):
        threads.append(
            threading.Thread(
                target=threaded_target,
                name="Thread #%d" % idx,
                args=(10 - idx, callback_when_done)
            )
        )

    [t.start() for t in threads]
    [t.join() for t in threads]

    # Note that thread #0 runs for the longest, but we'll see its message first!

Answer 2

What I would suggest is loop like this

while len(threadSet) > 0:
    time.sleep(1)
    for thread in theadSet:
        if not thread.isAlive()
            print "Thread "+thread.getName()+" terminated"
            threadSet.remove(thread)

There is a 1 second sleep, so there will be a slight delay between the thread termination and the message being printed. If you can live with this delay, then I think this is a simpler solution than the one you proposed in your question.

Answer 3

You can let the threads push their results into a threading.Queue. Have another thread wait on this queue and print the message as soon as a new item appears.

Answer 4

The thread needs to be checked using the Thread.is_alive() call.

Answer 5

I use a slightly different technique because of the nature of the threads I used in my application. To illustrate, this is a fragment of a test-strap program I wrote to scaffold a barrier class for my threading class:

   while threads:
        finished = set(threads) - set(threading.enumerate())
        while finished:
            ttt = finished.pop()
            threads.remove(ttt)
        time.sleep(0.5)

Why do I do it this way? In my production code, I have a time limit, so the first line actually reads "while threads and time.time() < cutoff_time". If I reach the cut-off, I then have code to tell the threads to shut down.

Answer 6

Here's a variation on @detly's answer that lets you specify the messages from your main thread, instead of printing them from your target functions. This creates a wrapper function which calls your target and then prints a message before terminating. You could modify this to perform any kind of standard cleanup after each thread completes.

#!/usr/bin/python

import threading
import time

def target1():
    time.sleep(0.1)
    print "target1 running"
    time.sleep(4)

def target2():
    time.sleep(0.1)
    print "target2 running"
    time.sleep(2)

def launch_thread_with_message(target, message, args=[], kwargs={}):
    def target_with_msg(*args, **kwargs):
        target(*args, **kwargs)
        print message
    thread = threading.Thread(target=target_with_msg, args=args, kwargs=kwargs)
    thread.start()
    return thread

if __name__ == '__main__':
    thread1 = launch_thread_with_message(target1, "finished target1")
    thread2 = launch_thread_with_message(target2, "finished target2")

    print "main: launched all threads"

    thread1.join()
    thread2.join()

    print "main: finished all threads"