Why is gcc generating an extra return address?

Question

I am currently learning the basics of assembly and came across something odd when looking at the instructions generated by gcc (6.1.1).

Here is the source:

Answer 1

Update: gcc8 simplifies this at least for normal use-cases (-fomit-frame-pointer, and no alloca or C99 VLAs that require variable-size allocation). Perhaps motivated by increasing usage of AVX leading to more functions wanting a 32-byte aligned local or array.

Also, probably a duplicate of What's up with gcc weird stack manipulation when it wants extra stack alignment?

---

This complicated prologue is fine if it only ever runs a couple times (e.g. at the start of main in 32-bit code), but the more it appears the more worthwhile it is to optimize it. GCC sometimes still over-aligns the stack in functions where all >16-byte aligned objects are optimized into registers, which is a missed optimization already but less bad when the stack alignment is cheaper.

---

gcc makes some clunky code when aligning the stack within a function, even with optimization enabled. I have a possible theory (see below) on why gcc might be copying the return address to just above where it saves ebp to make a stack frame (and yes, I agree that's what gcc is doing). It doesn't look necessary in this function, and clang doesn't do anything like that.

Besides that, the nonsense with ecx is probably just gcc not optimizing away unneeded parts of its align-the-stack boilerplate. (The pre-alignment value of esp is needed to reference args on the stack, so it makes sense that it puts the address of the first would-be arg into a register).

---

You see the same thing with optimization in 32-bit code (where gcc makes a main that doesn't assume 16B stack alignment, even though the current version of the ABI requires that at process startup, and the CRT code that calls main either aligns the stack itself or preserves the initial alignment provided by the kernel, I forget). You also see this in functions that align the stack to more than 16B (e.g. functions that use __m256 types, sometimes even if they never spill them to the stack. Or functions with an array declared with C++11 alignas(32), or any other way of requesting alignment.) In 64-bit code, gcc always seems to use r10 for this, not rcx.

There's nothing required for ABI compliance about the way gcc does it, because clang does something much simpler.

I added an aligned variable (with volatile as a simple way to force the compiler to actually reserve aligned space for it on the stack, instead of optimizing it away). I put your code on the Godbolt compiler explorer, to look at the asm with -O3. I see the same behaviour from gcc 4.9, 5.3, and 6.1, but different behaviour with clang.

int main(){
    __attribute__((aligned(32))) volatile int v = 1;
    return 0;
}

Clang3.8's -O3 -m32 output is functionally identical to its -m64 output. Note that -O3 enables -fomit-frame-pointer, but some functions make stack frames anyway.

    push    ebp
    mov     ebp, esp                # make a stack frame *before* aligning, so ebp-relative addressing can only access stack args, not aligned locals.
    and     esp, -32
    sub     esp, 32                 # esp is 32B aligned with 32 or 48B above esp reserved (depending on incoming alignment)
    mov     dword ptr [esp], 1      # store v
    xor     eax, eax                # return 0
    mov     esp, ebp                # leave
    pop     ebp
    ret

gcc's output is nearly the same between -m32 and -m64, but it puts v in the red-zone with -m64 so the -m32 output has two extra instructions:

    # gcc 6.1 -m32 -O3 -fverbose-asm.  Most of gcc's comment lines are empty.  I guess that means it has no idea why it's emitting those insns :P
    lea     ecx, [esp+4]      #,   get a pointer to where the first arg would be
    and     esp, -32  #,          align
    xor     eax, eax  #           return 0
    push    DWORD PTR [ecx-4]       #  No clue WTF this is for; this looks batshit insane, but happens even in 64bit mode.
    push    ebp     #             make a stackframe, even though -fomit-frame-pointer is on by default and we can already restore the original esp from ecx (unlike clang)
    mov     ebp, esp  #,
    push    ecx     #             save the old esp value (even though this function doesn't clobber ecx...)
    sub     esp, 52   #,          reserve space for v  (not present with -m64)
    mov     DWORD PTR [ebp-56], 1     # v,
    add     esp, 52   #,          unreserve (not present with -m64)
    pop     ecx       #           restore ecx (even though nothing clobbered it)
    pop     ebp       #           at least it knows it can just pop instead of `leave`
    lea     esp, [ecx-4]      #,  restore pre-alignment esp
    ret

It seems that gcc wants to make its stack frame (with push ebp) after aligning the stack. I guess that makes sense, so it can reference locals relative to ebp. Otherwise it would have to use esp-relative addressing, if it wanted aligned locals.

My theory on why gcc does this:

The extra copy of the return address after aligning but before pushing ebp means that the return address is copied to the expected place relative to the saved ebp value (and the value that will be in ebp when child functions are called). So this does potentially help code that wants to unwind the stack by following the linked list of stack frames, and looking at return-addresses to find out what function is involved.

I'm not sure whether this matters with modern stack-unwind info that allows stack-unwinding (backtraces / exception handling) with -fomit-frame-pointer. (It's metadata in the .eh_frame section. This is what the .cfi_* directives around every modification to esp are for.) I should look at what clang does when it has to align the stack in a non-leaf function.

---

The original value of esp would be needed inside the function to reference function args on the stack. I think gcc doesn't know how to optimize away unneeded parts of its align-the-stack method. (e.g. out main doesn't look at its args (and is declared not to take any))

This kind of code-gen is typical of what you see in a function that needs to align the stack; it's not extra weird because of using a volatile with automatic storage.