pthread_exit vs. return

Question

I have a joinable pthread runner function defined as below:

void *sumOfProducts(void *param)
{
...
pthread_exit(0);
}

This thread is supposed to join the main thread.

Whenever I ran my program through Valgrind I would get the following leaks:

LEAK SUMMARY:
   definitely lost: 0 bytes in 0 blocks
   indirectly lost: 0 bytes in 0 blocks
     possibly lost: 0 bytes in 0 blocks
   still reachable: 968 bytes in 5 blocks
        suppressed: 0 bytes in 0 blocks

ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 15 from 10)

I checked the man page for pthreads which said:

  The new thread terminates in one of the following ways:

   * It  calls  pthread_exit(3),  specifying  an exit status value that is
     available  to  another  thread  in  the  same  process   that   calls
     pthread_join(3).

   * It  returns  from  start_routine().   This  is  equivalent to calling
     pthread_exit(3) with the value supplied in the return statement.

   * It is canceled (see pthread_cancel(3)).

   * Any of the threads in the process calls exit(3), or the  main  thread
     performs  a  return  from main().  This causes the termination of all
     threads in the process.

Miraculously, when I replaced the pthread_exit() with a return statement, the leaks disappeared.

return(NULL);

My actual question is three-pronged:

1. Can someone explain why the return statement gave no leaks?
2. Is there some fundamental difference between both statements, in relation to exiting from threads?
3. If so, when should one be preferred over the other?

Answer 1

The following minimal test case exhibits the behaviour you describe:

#include <pthread.h>
#include <unistd.h>

void *app1(void *x)
{
    sleep(1);
    pthread_exit(0);
}

int main()
{
    pthread_t t1;

    pthread_create(&t1, NULL, app1, NULL);
    pthread_join(t1, NULL);

    return 0;
}

valgrind --leak-check=full --show-reachable=yes shows 5 blocks allocated from functions called by pthread_exit() that is unfreed but still reachable at process exit. If the pthread_exit(0); is replaced by return 0;, the 5 blocks are not allocated.

However, if you test creating and joining large numbers of threads, you will find that the amount of unfreed memory in use at exit does not increase. This, and the fact that it is still reachable, indicates that you're just seeing an oddity of the glibc implementation. Several glibc functions allocate memory with malloc() the first time they're called, which they keep allocated for the remainder of the process lifetime. glibc doesn't bother to free this memory at process exit, since it knows that the process is being torn down anyway - it'd just be a waste of CPU cycles.

Answer 2

Not sure if you're still interested in this, but I am currently debugging a similar situation. Threads that use pthread_exit cause valgrind to report reachable blocks. The reason seems to be fairly well explained here:

https://bugzilla.redhat.com/show_bug.cgi?id=483821

Essentially it seems pthread_exit causes a dlopen which is never cleaned up explicitly when the process exits.

Answer 3

It looks like calling exit() (and, apparently, pthread_exit()) leaves automatically-allocated variables allocated. You must either return or throw in order to properly unwind.

Per C++ valgrind possible leaks on STL string:

@Klaim: I don't see where that document says that I am wrong, but if it does then it is wrong. To quote the C++ standard (§18.3/8): "Automatic objects are not destroyed as a result of calling exit()." – James McNellis Sep 10 '10 at 19:11

Since doing a "return 0" instead of "pthread_exit(0)" seemed to solve your problem (and mine.. thanks), I'm assuming that the behavior is similar between the two.

Answer 4

Are you actually using C++, by any chance? To clarify - your source file ends with a .c extension, and you are compiling it with gcc, not g++?

It seems reasonably likely that your function is allocating resources that you expect to be cleaned up automatically when the function returns. Local C++ objects like std::vector or std::string do this, and their destructors probably won't be run if you call pthread_exit, but would be cleaned up if you just return.

My preference is to avoid low-level APIs such as pthread_exit, and always just return from the thread function, where possible. They're equivalent, except that pthread_exit is a de-facto flow-control construct that bypasses the language you're using, but return doesn't.

Answer 5

I have the experience that valgrind has difficulties of tracking the storage that is allocated for the state of joinable threads. (This goes in the same direction as caf indicates.)

Since it seems that you always return a value of 0 I guess that you perhaps need to join your threads from an application point of view? If so consider of launching them detached from the start, this avoids the allocation of that memory.

The downside is that you either have:

1. to implement your own barrier at the end of your main. If you know the number of threads beforehand, a simple statically allocated pthread_barrier would do.
2. or to exit you main with pthread_exit such that you don't kill the rest of the running threads that might not yet be finished.