Create function in memory

Question

I learned about function pointers recently in class and I was wondering if you could assign a function pointer to a block of memory allocated by a program, fill the memory block with assembly commands (hex values of op codes), then call the memory block using the function pointer.

I don't know much about function pointers, but I'm guessing you can't assign them wherever you want in memory, they need to point to a function. If that's true, how can you create a function in memory to be called? Is that even possible?

Here's some code I typed up to show the concept. I used various opcode values to see if anything would work, and 0x90 (NOP) did not break it sometimes (but it did other times), and 0xC3 (ret) always broke it.

#include <stdlib.h> //for malloc
#include <cstring> //for memcpy

int main() //program entry
{
    void(*test)() = NULL; //create function pointer, initialize to NULL
    void* hold_address = (void*)malloc(100); //allocate memory, save the address it returns in a dummy pointer
    int asm_commands[] = {0x90}; //create array of assembly commands, hex values
    memcpy(hold_address, asm_commands, sizeof(asm_commands)); //copy the array into the reserved memory
    test = (void(*)())hold_address; //set the function pointer to start of the allocated memory
    test(); //call the function, crashes here
    return 0; //exit the program
}

Answer 1

A few points:

• your asm_commands[] should probably be unsigned char (or uint8_t) - as is, you'll be copying 3 NUL/0 characters as well as the 0x90

• some Operating Systems will just not let you execute instructions in memory you've allocated with malloc() - they'll SIGSEGV or similar instead - that's intended to prevent certain types of stack overflow and other hackery

• I suggest you write an actual function void f() { } and see what opcodes are generated for it, using g++ -S or whatever your compiler offers, as you might need to do something more than just 0xC3 to return properly (e.g. pop certain registers)

  • if "cloning" instructions from an actual C++ function to get you started on a tweaked asm function, be wary of position dependent code too... you can't just copy data and code addresses within the function as they won't be inside the malloced region. Position Independent Code (PIC) uses relative addressing opcodes to avoid this... that's what you'll need to write.

Answer 2

You can do what you describe. However, memory allocated with malloc() might not have the permission for code execution - depending on your platform.

The way to allocate executable memory differs from OS to OS. On Linux, check mmap. On Windows, see VirtualAlloc.

Answer 3

Function calling is more complex than that.

You must at least modify the base pointer. I suggest you look at the dissassembly of a function call and try to mimic it