Compiler using local variables without adjusting RSP

Question

In question Compilers: Understanding assembly code generated from small programs the compiler uses two local variables without adjusting the stack pointer.

Not adjusting RSP for the use of local variables seems not interrupt safe and so the compiler seems to rely on the hardware automatically switching to a system stack when interrupts occur. Otherwise, the first interrupt that came along would push the instruction pointer onto the stack and would overwrite the local variable.

The code from that question is:

#include <stdio.h>

int main()
{
    for(int i=0;i<10;i++){
        int k=0;
    }
}

The assembly code generated by that compiler is:

00000000004004d6 <main>:
  4004d6:       55                      push   rbp
  4004d7:       48 89 e5                mov    rbp,rsp
  4004da:       c7 45 f8 00 00 00 00    mov    DWORD PTR [rbp-0x8],0x0
  4004e1:       eb 0b                   jmp    4004ee <main+0x18>
  4004e3:       c7 45 fc 00 00 00 00    mov    DWORD PTR [rbp-0x4],0x0
  4004ea:       83 45 f8 01             add    DWORD PTR [rbp-0x8],0x1
  4004ee:       83 7d f8 09             cmp    DWORD PTR [rbp-0x8],0x9
  4004f2:       7e ef                   jle    4004e3 <main+0xd>
  4004f4:       b8 00 00 00 00          mov    eax,0x0
  4004f9:       5d                      pop    rbp
  4004fa:       c3                      ret    

The local variables are i at [rbp-0x8] and k at [rbp-0x4].

Can anyone shine light on this interrupt problem? Does the hardware indeed switch to a system stack? How? Am I wrong in my understanding?

Answer 1

This is the so called "red zone" of the x86-64 ABI. A summary from wikipedia:

In computing, a red zone is a fixed-size area in a function's stack frame beyond the current stack pointer which is not preserved by that function. The callee function may use the red zone for storing local variables without the extra overhead of modifying the stack pointer. This region of memory is not to be modified by interrupt/exception/signal handlers. The x86-64 ABI used by System V mandates a 128-byte red zone which begins directly under the current value of the stack pointer.

In 64-bit Linux user code it is OK, as long as no more than 128 bytes are used. It is an optimization used most prominently by leaf-functions, i.e. functions which don't call other functions,

---

If you were to compile the example program as a 64-bit Linux program with GCC (or compatible compiler) using the -mno-red-zone option you'd see code like this generated:

main:
        push    rbp
        mov     rbp, rsp
        sub     rsp, 16;     <<============  Observe RSP is now being adjusted.
        mov     DWORD PTR [rbp-4], 0
.L3:
        cmp     DWORD PTR [rbp-4], 9
        jg      .L2
        mov     DWORD PTR [rbp-8], 0
        add     DWORD PTR [rbp-4], 1
        jmp     .L3
.L2:
        mov     eax, 0
        leave
        ret

This code generation can be observed at this godbolt.org link.

---

For a 32-bit Linux user program it would be a bad thing not to adjust the stack pointer. If you were to compile the code in the question as 32-bit code (using -m32 option) main would appear something like the following code:

main:
        push    ebp
        mov     ebp, esp
        sub     esp, 16;     <<============  Observe ESP is being adjusted.
        mov     DWORD PTR [ebp-4], 0
.L3:
        cmp     DWORD PTR [ebp-4], 9
        jg      .L2
        mov     DWORD PTR [ebp-8], 0
        add     DWORD PTR [ebp-4], 1
        jmp     .L3
.L2:
        mov     eax, 0
        leave
        ret

This code generation can be observed at this gotbolt.org link.