Does executing an int 3 interrupt stop the entire process on Linux or just the current thread?

Question

Suppose the architecture is x86. And the OS is Linux based. Given a multithreaded process in which a single thread executes an int 3 instruction, does the inter

Answer 1

Since the question is Linux specific, let's dive into kernel sources! We know int 3 will generate a SIGTRAP, as we can see in do_int3. The default behaviour of SIGTRAP is to terminate the process and dump core.

do_int3 calls do_trap which, after a lot of indirection, calls complete_signal, where most of the magic happens. Following the comments, it's quite clear to see what is happening without much need for explanation:

• A thread is found to deliver the signal to. The main thread is given first crack, but any thread can get it unless explicitly stated it doesn't want to.
• SIGTRAP is fatal (and we've assumed we want to establish what the default behaviour is) and must dump core, so it is fatal to the whole group
• The loop at line 1003 wakes up all threads and delivers the signal.

EDIT: To answer the comment:

When the process is being ptraced, the behaviour is pretty well documented in the manual page (see "Signal-delivery-stop"). Basically, after the kernel selects an arbitrary thread which handles the signal, if the selected thread is traced, it enters signal-delivery-stop -- this means the signal is not yet delivered to the process, and can be suppressed by the tracer process. This is the case with a debugger: a dead process is of no use to us when debugging (that's not entirely true, but let's consider the live-debugging scenario, which is the only one which makes sense in this context), so by default we block SIGTRAP unless the user specifies otherwise. In this case it is irrelevant how the traced process handles SIGTRAP (SIG_IGN or SIG_DFL or a custom handler) because it will never know it occurred.

Note that in the case of SIGTRAP, the tracer process must account for various scenarios other than the process being stopped, as also detailed in the man page under each ptrace action.