How to allocate a new TLS area with clone system call

Question

Short version of question: What parameter do I need to pass to the clone system call on x86_64 Linux system if I want to allocate a new TLS area for the thread that I am creating.

Long version:

I am working on a research project and for something I am experimenting with I want to create threads using the clone system call instead of using pthread_create. However, I also want to be able to use thread local storage. I don't plan on creating many threads right now, so it would be fine for me to create a new TLS area for each thread that I create with the clone system call.

I was looking at the man page for clone and it has the following information about the flag for the TLS parameter:

CLONE_SETTLS (since Linux 2.5.32)
   The newtls argument is the new TLS (Thread Local Storage) descriptor.
   (See set_thread_area(2).)

So I looked at the man page for set_thread_area and noticed the following which looked promising:

 When  set_thread_area()  is  passed  an  entry_number  of -1, it uses a 
 free TLS entry. If set_thread_area() finds a free TLS entry, the value of
 u_info->entry_number is set upon return to show which entry was changed.

However, after experimenting with this some it appears that set_thread_area is not implemented in my system (Ubunut 10.04 on an x86_64 platform). When I run the following code I get an error that says: set_thread_area() failed: Function not implemented

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h> 
#include <sys/syscall.h>
#include <sys/types.h>
#include <linux/unistd.h>

 #include <asm/ldt.h>

int main()
{
  struct user_desc u_info;
  u_info.entry_number = -1; 
  int rc = syscall(SYS_set_thread_area,&u_info);
  if(rc < 0) {
    perror("set_thread_area() failed");
    exit(-1);
  }

  printf("entry_number is %d",u_info.entry_number);
}

I also saw that when I use strace the see what happens when pthread_create is called that I don't see any calls to set_thread_area. I have also been looking at the nptl pthread source code to try to understand what they do when creating threads. But I don't completely understand it yet and I think it is more complex than what I'm trying to do since I don't need something that is as robust at the pthread implementation. I'm assuming that the set_thread_area system call is for x86 and that there is a different mechanism used for x86_64. But for the moment I have not been able to figure out what it is so I'm hoping this question will help me get some ideas about what I need to look at.

Answer 1

I am working on a research project and for something I am experimenting with I want to create threads using the clone system call instead of using pthread_create

In the exceedingly unlikely scenario where your new thread never calls any libc functions (either directly, or by calling something else which calls libc; this also includes dynamic symbol resolution via PLT), then you can pass whatever TLS storage you desire as the the new_tls parameter to clone.

You should ignore all references to set_thread_area -- they only apply to 32-bit/ix86 case.

If you are planning to use libc in your newly-created thread, you should abandon your approach: libc expects TLS to be set up a certain way, and there is no way for you to arrange for such setup when you call clone directly. Your new thread will intermittently crash when libc discovers that you didn't set up TLS properly. Debugging such crashes is exceedingly difficult, and the only reliable solution is ... to use pthread_create.

Answer 2

The other answer is absolutely correct in that setting up a thread outside of libc's control is guaranteed to cause trouble at a certain point. You can do it, but you can no longer rely on libc's services, definitely not on any of the pthread_* functions or thread-local variables (defined as such using __thread or thread_local).

That being said, you can set one of the segment registers used for TLS (GS and FS) even on x86-64. The system call to look for is prctl(ARCH_SET_GS, ...).

You can see an example comparing setting up TLS registers on i386 and x86-64 in this piece of code.