Assembly Return int to C function segfaults

问题

I am finishing up an assembly program that replaces characters in a string with a given replacement character. The assembly code calls C functions and the assembly program itself is called from main in my .c file. However, when trying to finish and return a final int value FROM the assembly program TO C, I get segfaults. My .asm file is as follows:

; File: strrepl.asm
; Implements a C function with the prototype:
;
;   int strrepl(char *str, int c, int (* isinsubset) (int c) ) ;
;
; 
; Result: chars in string are replaced with the replacement character and string is returned.

    SECTION .text
    global  strrepl


_strrepl:   nop
strrepl:
    push    ebp         ; set up stack frame
    mov ebp, esp

    push    esi         ; save registers
    push    ebx
    xor eax, eax
    mov ecx, [ebp + 8]      ;load string (char array) into ecx
    jecxz   end         ;jump if [ecx] is zero
    mov al, [ebp + 12]      ;move the replacement character into esi
    mov edx, [ebp + 16]     ;move function pointer into edx

firstLoop:

    xor eax, eax

    mov edi, [ecx]
    cmp edi, 0
    jz  end

    mov edi, ecx        ; save array


    movzx   eax, byte [ecx]     ;load single byte into eax  
    push    eax         ; parameter for (*isinsubset)
    mov edx, [ebp + 16]         
    call    edx         ; execute (*isinsubset)


    mov ecx, edi        ; restore array
    cmp eax, 0
    jne secondLoop  
    add esp, 4          ; "pop off" the parameter
    mov ebx, eax        ; store return value
    add ecx, 1
    jmp firstLoop

secondLoop:
    mov eax, [ebp+12]
    mov [ecx], al
    mov edx, [ebp+16]
    add esp, 4
    mov ebx, eax
    add ecx, 1
    jmp     firstLoop

end:
    pop ebx         ; restore registers
    pop esi
    mov esp, ebp        ; take down stack frame
    pop ebp
    mov eax, 9
    push    eax         ;test
    ret

and my c file is:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <ctype.h>

//display *((char *) $edi)
// These functions will be implemented in assembly:
//

int strrepl(char *str, int c, int (* isinsubset) (int c) ) ;


int isvowel (int c) {

   if (c == 'a' || c == 'e' || c == 'i' || c == 'o' || c == 'u') 
      return 1 ;

   if (c == 'A' || c == 'E' || c == 'I' || c == 'O' || c == 'U') 
      return 1 ;

   return 0 ;
}

int main(){
    char *str1;
    int r;


    str1 = strdup("ABC 123 779 Hello World") ;
    r = strrepl(str1, '#', &isdigit) ;
    printf("str1 = \"%s\"\n", str1) ;
    printf("%d chararcters were replaced\n", r) ;
    free(str1) ;
    return 0;
}

In my assembly code, you can see in end

mov eax, 9
push    eax 

I am simply trying to return the value 9 to the value "r" which is an int in the C file. This is just a test to see if I can return an int back to r in the c file. Eventually I will be returning the number of characters that were replaced back to r. However, I need to figure out why the following code above is segfaulting. Any ideas?

回答1:

mov     eax, 9
push    eax          ; NOT a good idea
ret

That is a big mistake. It's going to return based on the lowest thing on the stack and you've just pushed something on to the stack that's almost certainly not a valid return address.

Most functions return a code by simply placing it into eax (this depends on calling convention of course but that's a pretty common one), there's generally no need to push it on to the stack, and certainly plenty of downside to doing so.

回答2:

Return values are normally stored in EAX on X86 32 bit machines. So your pushing it on the stack after storing it in EAX is wrong, because the function it is returning to will try to use what is in EAX as a value for IP (instruction pointer)

Ret with no argument pops the return address off of the stack and jumps to it.

source

来源：https://stackoverflow.com/questions/16435888/assembly-return-int-to-c-function-segfaults